Applets

Algorithms

From the Mathematical Perspective

What is an algorithm?

•Algorithm: A prescribed set of welldefined rules and processes forsolving a problem in a finite numberof steps

•Prescribed: known in advance

•Well defined: unambiguous, clear

•Finite: always ends

•Not every program implements analgorithm, but most do

Properties

•An algorithm should demonstrateseveral properties:

•Input – they take some type of inputdata

•Output – they produce some answer

•Definiteness – each step isunambiguously defined

•Correctness – a correct algorithmproduces correct results for eachset of input values

More Properties

•Finiteness – always finishes in afinite number of steps

–Finite may be large

–Recall the halting problem

•Effectiveness - it must be possible toperform each step exactly in a finiteamount of time

•Generality – it should solve theproblem for any set of inputs, notjust a particular set

–We are allowed to limit input domains

Expression

•How do we represent an algorithm?

•Almost every program implementsan algorithm

–We do not necessarily want to learneach programming language

•Choose a standard language

–ACM chose ALGOL as their standard

–Before that FORTRAN was common

•Other possibilities exist

Pseudo Code

•The book, among others, haschosen pseudo code

•This has greater precision thannatural language yet we do notworry about the syntactical detailsof a real programming language

•The book’s choice looks much likePascal, which is also a descendentof ALGOL

Rules(?) of Pseudo code

–No need to declare variables, except inprocedure headers

–No need for ; to terminate or separate astatement

–Usually one statement per line

–Indentation is significant

–Assignment statement uses :=

–Reserved words are bold

–An if statement has a then or then andelse

–Simplified for and while syntax

–Comments in braces

Example

•Find maximum element in sequence:procedure max(a1, a2, …an: integer) max := a1 {Assignment on name} for i=2 to n if max < ai

• then max := ai return max {All done}

Searching

•Something we do a lot of inComputer Science

•We have a collection of items

•We want to find whether a given oneexists in the collection

•Sometimes we search for a key andwant the associated data

–Account number and accountinformation

•Other times just to see if the valueexists – such as set membership

Two Searches

•A linear search looks sequentiallythrough a sequence of values

–The search length is proportional to thenumber of items

•A binary search looks through asorted sequence of values

–This works much faster

Linear

procedure linear search(x:integer,

a1, a2, …,an: distinct integers)

i := 1

while (i ≤ n and x ≠ ai)

i := i + 1

if i ≤ n

then location := i

else location := 0

return location{location is the subscript of the termthat equals x, or is 0 if x is not found}

Binary

procedure binary search(x: integer, a1,a2,…, an: increasing integers)

i := 1 {i is the left endpoint of interval}

j := n {j is right endpoint of interval}

while i < j

m := ⌊(i + j)/2⌋

if x > am then i := m + 1

else j := m

if x = ai then location := i

else location := 0

return location{location is the subscript i of the termai equal to x, or 0 if x is not found}

Sorting

•Another extremely importantoperation

•Put a sequence into ascending ordescending order

•Look at two here

•Bubble

•Insertion

Bubble Sort

•Basic idea

•Start at top

•Compare adjacent elements

•Exchange if out of order

•Repeat until a pass has noexchanges

•See the text for pseudo code

First Pass

•Small items bubble up slowly

–One element per pass

•Large items sink quickly

–Keep descending until they find alarger item or hit bottom

Commentary

•The book uses two for loops

•This is not the best approach

•However, the bubble sort is perhapsthe worst sort in existence

•The only thing it should be everconsidered for is a sequence wherethere are only slight disorderings

Insertion Sort

•Partition the array into two pieces

•The first one and all the rest

•The first part of the array is alreadysorted

•Remove the first unsorted item

•Insert into the correct location of thesorted part

How it works

Sorted part of array

Unsorted part of array

Remove 3

Insert

Greedy Algorithms

•There are a number of situationswhere we want to find an optimalsolution

–Usually a maximum of minimum

•There are typically severalparameters, that is values that canbe changed

•Exhaustively searching everycombination of every parameter istypically prohibitive

How do they work?

•Start at some point

–All parameters to function have anarbitrary or given value

•Look one step in every direction

–Vary each parameter by a smallincrement

•Choose the one that gives the bestresults and return to the first stepwith new values

•Quit when no more improvementmay be made

Discussion

•There must be a readily availablefunction that produces a measure ofthe goodness of the solution

•The results may be optimal or notdepending on the type of problem

•As the algorithm homes in on theproblem the step size may beadjusted

•Next we will consider the changeproblem giving the fewest coins

Change

•Suppose we have to give change

•The amount required is in the range1 to 99 cents

•We wish to give the fewest coins

•We have the normal coins: quarters,dimes, nickels and pennies

•Our function takes the count of eachcoin and the original change amountand returns how much change is leftto be given

Greed is good

•The starting point is no coins aregiven – fours counts of zero

•We next look at four steps:incrementing each coin

•We choose the one that gives us thesmallest value that not negative

•This is our best step – next go to thebeginning with adjusted parameters

Example

•Suppose that the amount is 78 cents

•We try one of each coin and find thatthe quarter gets us to 53

•We try one of each again and findthat the quarter gets us to 28

•We try one of each again and findthat the quarter gets us to 3

•Now the quarter, dime and nicketsends us negative, but 1 penny getsus to 2, which we repeat twice more

Commentary

•This is a simple with an easy solution

•A similar program was shown in127/160 that solves this optimallybut is not a classis greedy

•However, greedy solutions work inmuch more complicatedenvironments where the correctpath is not know in advance

•The book also observes that ifnickels are not available thisalgorithm will not necessarily beoptimal

The Halting Problem

•This is important because of thefinite number of steps requirementof an algorithm

•That it is unsolvable shows absolutelimitations to static analysis

–What can be seen without running aprogram

–It also proves the existence ofunsolvable problems

•We saw the proof in chapter 1presentations

Exercises

•3, 7, 19, 35, 53, 55