Learning to Count

Combinatorics problems are notorious for their reliance on cleverness and insight. Once you look at the problem in the right way, the answer suddenly becomes obvious.

Basic counting techniques include:

*Product Rule*- The*product rule*states that if there are possibilities from set and possibilities from set , then there are ways to combine one from and one from .*Sum Rule*- The*sum rule*states that if there are possibilities from set and possibilities from set , then there are ways for either*or*to occur - assuming the elements of and are distinct.*Inclusion-Exclusion Formula*- The sum rule is a special case of a more general formula when the two sets can overlap, namely,

The reason this works is that summing the sets double counts certain possibilities, namely, those occurring in both sets.

Double counting is a slippery aspect of combinatorics, which can make it difficult to solve problems via inclusion-exclusion.

Combinatorial Objects

A * bijection* is a one-to-one mapping between the elements of one set
and the elements of another.
Counting the size of one of the sets automatically gives you the
size of the other set.

Exploiting bijections requires us to have a repertoire of sets which we know how to count, so we can map other objects to them.

It is useful to have a feeling for how fast the number of objects grows, to know when exhaustive search breaks down as a possible technique:

*Permutations*- A*permutation*is an arrangement of items, where every item appears exactly once. There are different permutations. The permutations of three items are 123, 132, 213, 231, 312, and 321. For , .*Subsets*- A*subset*is a selection of elements from possible items. There are distinct subsets of things. Thus there are subsets of three items, namely, 1, 2, 3, 12, 13, 23, 123, and the empty set: never forget the empty set. For , .*Strings*- A*string*is a sequence of items which are drawn*with repetition*. There are distinct sequences of items drawn from items. There are 27 length-3 strings on 123. The number of binary strings of length is identical to the number of subsets of items (why?)

Recurrence Relations

Recurrence relations make it easy to count a variety of recursively defined structures. Recursively defined structures include trees, lists, well-formed formulae, and divide-and-conquer algorithms - so they lurk wherever computer scientists do.

A recurrence relation is an equation which is defined in terms of itself. They are useful because many natural functions are easily expressed as recurrences:

Polynomials:

Exponentials:

Weird but interesting functions otherwise hard to represent:

It is often easy to find a recurrence as the answer
to a counting problem.
* Solving* the recurrence to get a nice closed form can
be somewhat of an art, but
as we shall see, computer programs can easily evaluate the value of a given
recurrence
even without the existence of a nice closed form.

Binomial Coefficients

The most important class of counting numbers are the * binomial
coefficients*, where
counts the number of ways to
choose things out of possibilities.
What do they count?

*Committees*- How many ways are there to form a -member committee from people? By definition, is the answer.*Paths Across a Grid*- How many ways are there to travel from the upper-left corner of an grid to the lower-right corner by walking only down and to the right? Every path must consist of steps, downward and to the right. Every path with a different set of downward moves is distinct, so there are such sets/paths.*Coefficients of*- Observe that

What is the coefficient of the term ? Clearly , because it counts the number of ways we can choose the -terms out of possibilities.

Computing Binomial Coefficients

Since , in principle you can compute them straight from factorials. However, intermediate calculations can easily cause arithmetic overflow even when the final coefficient fits comfortably within an integer.

A more stable way to compute binomial coefficients is using the recurrence
relation implicit in the construction of Pascal's triangle, namely,
that

Why does this work? Consider whether the th element appears in one of the subsets of elements. If so, we can complete the subset by picking other items from the other . If not, we must pick all items from the remaining . There is no overlap between these cases, and all possibilities are included, so the sum counts all -subsets.

No recurrence is complete without basis cases. How many ways are there to choose 0 things from a set? Exactly one, the empty set. The right term of the sum drives us up to . How many ways are there to choose things from a -element set? Exactly one, the complete set.

Implementation

The following implementation of this recurrence is a good example of
programming techniques used in * dynamic programming*, namely
storing a table of results so we can evaluate terms of the recurrence
by looking up the appropriate values.

#define MAXN 100 /* largest n or m */ long binomial_coefficient(n,m) int n,m; /* computer n choose m */ { int i,j; /* counters */ long bc[MAXN][MAXN]; /* table of binomial coefficients */ for (i=0; i<=n; i++) bc[i][0] = 1; for (j=0; j<=n; j++) bc[j][j] = 1; for (i=1; i<=n; i++) for (j=1; j<i; j++) bc[i][j] = bc[i-1][j-1] + bc[i-1][j]; return( bc[n][m] ); }

Other Counting Sequences

Several other counting sequences which repeatedly emerge in applications, and which are easily computed using recurrence relations.

*Fibonacci numbers*- Defined by the recurrence and the initial values and , they emerge repeatedly because this is perhaps the simplest interesting recurrence relation. The first several values are 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, ...*Catalan Numbers*- The recurrence and associated closed form

defines the*Catalan numbers*, which occur in a surprising number of problems in combinatorics. The first several terms are when .How many ways are there to build a balanced formula from sets of left and right parentheses? For example, there are five ways to do it for : , , , , and . The leftmost parenthesis matches some right parenthesis , which must partition the formula into two balanced pieces, leading to the recurrence.

The exact same reasoning arises in counting the number of triangulations of a convex polygon, counting the number of rooted binary trees on leaves, and counting the number of paths across a lattice which do not rise above the main diagonal.

*Eulerian Numbers*- The*Eulerian*numbers count the number of permutations of length with exactly ascending sequences or*runs*. A recurrence can be formulated by considering each permutation of . There are places to insert element , and each either splits an existing run of or occurs immediately after the last element of an existing run, thus preserving the run count. Thus .*Integer Partitions*- An integer partition of is an unordered set of positive integers which add up to . For example, there are seven partitions of 5, namely, , , , , , , and . The easiest way to count them is to define a function giving the number of integer partitions of with largest part at most . In any acceptable partition the largest part either does or does not reach with limit, so . The basis cases are and whenever .

Assigned Problems

110603 (Counting) - How many ways can we express as the sum of 2s, 3s, and two types of 1?

110604 (Expressions) - How many ways can we build a well-formed formula from parentheses with nesting depth ?

110606 (The Priest Mathematician) - What is the best way to solve a 4-peg Tower of Hanoi puzzle?

110607 (Self-describing Sequence) - What is the th value of the sequence containing occurrences of for each , i.e. 1, 2, 2, 3, 3, 4 ,4, 4, 5, 5, 6 ...? What if is too large to construct the entire sequence in memory?

2003-05-22