Multinomial coefficients

This is the fourth post in a series of posts on combinatorial analysis. The post is opened with the following problem. This post builds on the previous post on binomial coefficients.

    Figure 1 – three choices for lunch

Multinomial Lunch Problem
Each of the twelve people have 3 choices for lunch – McDonald, Burger King and IN-N-OUT, all fast food restaurants. The choice for each person is independent of the choices of the other diners.

  1. In how many ways can the lunch choices for this 12-person group be made?
  2. In how many ways can the lunch choices for this 12-person group be made in such a way that 3 people choose McDonald, 4 people choose Burger King and 5 people choose IN-N-OUT?
  3. In how many ways can the lunch choices for this 12-person group be made in such a way that 3 people choose one restaurant, 4 people choose another restaurant and 5 people choose the remaining restaurant?

The previous posts on combinatorial analysis are: binomial coefficients, combination and permutation and multiplication principle.

___________________________________________________________________________

Multinomial Coefficients

The problem for lunch choices is a multinomial coefficient problem. The second question in the problem is equivalent to any one of the following question.

  1. How many ways can a set of 12 distinct objects be divided into 3 subgroups, one consisting of 3 objects, one consisting of 4 objects and one consisting of 5 objects?
  2. How many different 12-letter strings are there consisting of 3 M’s, 4 B’s and 5 I’s?
  3. Suppose each of the questions in a 12-question multiple choice test has three choices (M, D and I). A student chooses the answers by pure guessing. How many of the possible test answers consist of 3 M’s, 4 B’s and 5 I’s?
  4. How many ways can 12 people be assigned to 3 committees such that one committee consists of 3 people, one committee consists of 4 people and one committee consists of 5 people? Assume that each person is assigned to only one committee.
  5. An urn contains three letters M, B and I. Randomly sample 12 letters from the urn with replacement. What is the number of sample results that consist of 3 M’s, 4 B’s and 5 I’s?

The first question is basically the definition of multinomial coefficient. It is the total number of ways to divide a set of objects into several subsets such that each subset is of a pre-determined size. In the second question, the objects are the 12 positions in the letter strings. In the third question, the objects to be divided are the 12 multiple choice questions. In the fourth question, the objects to be divided are the 12 people to be assigned into committees.

The fifth question is from a random sampling perspective. In any of the question, the dividing can be thought of as a choice (or decision) for each object. For each of the 12 objects, should the object be put into group 1, group 2 or group 3 (or should choice 1 be taken, choice 2 be taken, choice 3 be taken)? In the lunch problem, each diner has 3 choices. The choices in their totality would be viewed as the results from a random sampling of the population of 3.

The answer to any of the above questions is a multinomial coefficient.

Multinomial Coefficient

If n_1+n_2+\cdots+n_k=n, the notation for the multinomial coefficient is \displaystyle \binom{n}{n_1,n_2,\cdots,n_k} and is defined by \displaystyle \binom{n}{n_1,n_2,\cdots,n_k}=\frac{n!}{n_1! n_2! \cdots n_k!}. The multinomial coefficient is the number of ways a set of n distinct objects can be divided into k subsets, one of which consists of n_1 objects, one of which consists of n_2 objects, …., one of which consists of n_k objects.

It is clear that multinomial coefficients are a generalization of binomial coefficients. Instead of having to choose from one of two choices in each random selection, there are multiple choices to choose from. Even the notation \displaystyle \binom{n}{n_1,n_2,\cdots,n_k} is a generalization of the notation \displaystyle \binom{n}{n_1}, the notation for binomial coefficients. Note that \displaystyle \binom{n}{n_1,n-n_1}=\binom{n}{n_1}.

___________________________________________________________________________

Multinomial Lunch Problem

Viewing the lunch choices as random sampling (each diner randomly chooses from M, B and I), there are 3^{12}= 531,441 many ways can the lunch choices be made for the whole 12-person group. This count contains all the possible choices including possibilities that some restaurant is not chosen. The following is one possible outcome that fall under question 2.

    B-M-B-I-I-M-B-B-M-I-I-I

The above string shows that the first person chooses Burger King, the second person chooses McDonald, the third person chooses Burger King and so on. How many other strings are like this one in the sense that there are 3 M’s, 4 B’s and 5 I’s? This would be a multinomial coefficient.

    \displaystyle \frac{12!}{3! \ 4! \ 5!}=\frac{12 \cdot 11 \cdot 10 \cdot 9 \cdot 8 \cdot 7 \cdot 6}{3 \cdot 2 \cdot 1 \cdot 4 \cdot 3 \cdot 2 \cdot 1}=\text{27,720}

The answer to the third question is more than 27,720. For example, there could be three diners choosing IN-N-OUT, 4 diners choosing Burger King and 5 diners choosing McDonald. In other words, we need to apply multinomial coefficient on the three positions (three restaurants). There are 6 ways to divide 3 positions resulting in three subsets with one each since \frac{3!}{1! \ 1! \ 1!}=6. Then there are 6 x 27,720 = 166,320 ways for 12 diners making choices so that 3 of them go to one place, 4 of them go to another place and five of them go to the remaining place. This is an example of the double use of the multinomial coefficients – the first application is on dividing the objects into subgroups and the second application on the dividing the subgroups. The following example gives another illustration.

Example 1
A fair die is rolled 15 times. How many of the outcomes consist of four 2’s, five 3’s and six 4’s? How many of the outcomes consist of four of one face, five of another face and six of another face (different from the other two faces)?

The first question is a straight application of one multinomial coefficient.

    \displaystyle \frac{15!}{4! \ 5! \ 6!}=\frac{15 \cdot 14 \cdot 13 \cdot 12 \cdot 11 \cdot 10 \cdot 9 \cdot 8 \cdot 7}{4 \cdot 3 \cdot 2 \cdot 1 \cdot 5 \cdot 4 \cdot 3 \cdot 2 \cdot 1}=\text{630,630}

To understand the second question, consider the notation (0, 4, 5, 6, 0, 0), which means that there are zero 1’s, four 2’s, five 3’s, six 4’s, zero 5’s and zero 6’s from rolling the die 15 times. Basically it shows how many rolls fall into each position (face). Now we are interested in counting other outcomes that have 4, 5 and 6 of another 3 positions. Basically we are trying to divide 6 positions into four groups, one groups with 3 positions that do not appear in the 15 rolls, one groups with one position that appear 4 times, one group with one position that appear 5 times, and one group with one position that appears 6 times. The following shows how many ways to divide the 6 positions.

    \displaystyle \frac{6!}{3! \ 1! \ 1! \ 1!}=120

For each of the 120 times, there are 630,630 outcomes. Thus the answer to the second question is 120 x 630,630 = 75,675,600. \square

___________________________________________________________________________

Multinomial Sampling

We now explore the random sampling angle of multinomial coefficients. To make it easier to follow, we continue to use the Multinomial Lunch Problem. The problem, as discussed above, is like sampling 12 times with replacement from an urn with 3 letters M, B and I. Each of the 12 random selections has 3 outcomes. Thus the experiment in total has 3^{12} = 531,441 outcomes, each of which can be written as a string of 12 letters. Three examples:

    B-M-B-I-I-M-B-B-M-I-I-I

    I-M-M-I-M-M-M-I-M-I-I-M

    M-M-M-M-M-M-M-M-M-M-M-M

These three strings can be called ordered samples for the experiment of randomly selecting from the urn 12 times with replacement. These display the results of each draw sequentially. We can also use unordered samples, which do not contain the orders but contain the number of times each letter is drawn. For the first string, the unordered sample would be (3, 4, 5) where the first number is the number of people choosing McDonald, the second number is the number of people choosing Burger King and the third number is the number of people choosing IN-N-OUT. For the second string, the unordered sample would be (7, 0, 5). For the third, everyone goes to McDonald and so it is (12, 0, 0).

Note that (3, 4, 5) is called an unordered sample because it does not tell us the order of the chosen letters. It only tells us how many times each letter is chosen. The number of ordered samples resulting in an unordered sample would be precisely the multinomial coefficient. For the unordered sample (3, 4, 5), there would be

    \displaystyle \frac{12!}{3! \ 4! \ 5!}=\frac{12 \cdot 11 \cdot 10 \cdot 9 \cdot 8 \cdot 7 \cdot 6}{3 \cdot 2 \cdot 1 \cdot 4 \cdot 3 \cdot 2 \cdot 1}=\text{27,720}

many ordered samples. In randomly selecting letters 12 times with replacement from an urn containing the letters M, B and I, how many unordered samples are there? To answer this question, it is helpful to look at unordered samples using a star-and-bar diagram. The star-and-bar diagrams for above three unordered samples are:

    * * * | * * * * | * * * * *

    * * * * * * * | \text{ } | * * * * *

    * * * * * * * * * * * * | \text{ } | \text{ }

In the random experiment in question, a star-and-bar diagram has 12 stars, representing the 12 letters selected, and 2 bars, creating three spaces for the three letters M, B and I. In each star-and-bar diagram, there are 12 + 2 = 14 positions, 12 of which are for the stars and the remaining 2 positions are for the bars. In other words, each of the 14 positions can be either a star or a bar. Thus star-and-bar diagrams are the results of a binomial experiment. How many possible star-and-bar diagrams? There are

    \displaystyle \binom{12+2}{11}=\binom{14}{11}=\binom{14}{2}=364

many different diagrams. Thus there are 364 many unordered samples from drawing letters 12 times from an urn containing the letters M, B and I.

A related question is: how many unordered samples are there in this experiments such that each letter is selected at least once? This means that each space in the star-and-bar diagram has at least one star. Then the remaining 12 – 3 = 9 stars will have to be distributed in the three spaces. There are

    \displaystyle \binom{9+2}{9}=\binom{11}{9}=55

unordered samples in which each letter is chosen at least once.

There is another way to interpret the unordered samples. What is the total number of non-negative integer solutions to the equation x + y + x = 12? The solution x = 3, y = 4, z = 5 would be the unordered sample (3, 4, 5). Thus there are 364 different solutions. What is the total number of positive integer solutions to the equation x + y + x = 12? There are 55 such solution. Each such solution would be an unordered sample where each letter is selected at least once.

___________________________________________________________________________

Multinomial Theorem

We now generalize the discussion on multinomial sampling.

Multinomial Sampling

Suppose that a multinomial experiment consisting of n independent trials with each trial resulting in exactly k outcomes is performed. For convenience, these k outcomes are labeled by the symbols x_1,x_2,\cdots,x_k. The result of the experiment can be recorded by an ordered sample, which is a string of n symbols with each symbol being one of x_1,x_2,\cdots,x_k. The result of the experiment can also be recorded by an unordered sample, which is a sequence of k numbers with the first number being the number occurrences of the outcome x_1, the second number being the number of occurrences of the outcome x_2 and so on. The number of possible ordered samples in the multinomial experiment is k^n. The number of ordered samples resulting in a given unordered sample (n_1,n_2,\cdots,n_k) is the multinomial coefficient \displaystyle \binom{n}{n_1,n_2,\cdots,n_k}=\frac{n!}{n_1! \ n_2! \cdots n_k!}. The number of unordered samples in the experiment is \displaystyle \binom{n+k-1}{n} based on a combinatorial argument using stars and bars. The number of unordered samples such that in each sample, each of the outcomes x_1,x_2,\cdots,x_k occurs at least once is \displaystyle \binom{n-k+k-1}{n-k}=\binom{n-k+k-1}{k-1}=\binom{n-1}{k-1}.

Number of Integer Solutions to Equations

The number of non-negative integer solutions to the equation x_1+x_2+\cdots+x_k=n is \displaystyle \binom{n+k-1}{n}, which is the total number of unordered samples as indicated above. The number of positive integer solutions to the equation x_1+x_2+\cdots+x+k=n is \displaystyle \binom{n-1}{k-1}, which is the number of unordered samples such that each symbol from x_1,x_2,\cdots,x_k occurs at least once in the multinomial sampling.

There is a lot to unpack in the above two paragraphs. It is helpful to follow the random sampling idea on a specific example, either the lunch problem discussed above or another example. The ideas in the above paragraphs contains all the information for stating the multinomial theorem.

Sum of all Multinomial Coefficients

In the multinomial experiment described above, the sum of all possible multinomial coefficients is k^n, i.e. \displaystyle \sum \limits_{a_1+a_2+\cdots a_k=n} \frac{n!}{a_1! \ a_2! \cdots a_k!}=k^n

Multinomial Theorem

The formula shown below shows how to raise a multinomial to a power. \displaystyle (x_1+x_2+\cdots x_k)^n =\sum \limits_{a_1+a_2+\cdots a_k=n} \frac{n!}{a_1! \ a_2! \cdots a_k!} \ x_1^{a_1} x_2^{a_2} \cdots  x_k^{a_k}

The count k^n is identical to the sum of all the possible multinomial coefficients in the experiment. This fact follows from the observations we make about the multinomial sampling. The count k^n is the total number of all ordered samples. The ordered samples can be divided into groups where each group consists of of all ordered samples associating with the same unordered sample. Note that the size of each of these groups is a multinomial coefficient.

The multinomial theorem is a compact way of describing the multinomial experiment. When it is expanded out, the left-hand-side lists out all the k^n ordered samples. A typical ordered sample consists of a_1 many x_1, a_2 many x_2 and so on, which can be represented by x_1^{a_1} x_2^{a_2} \cdots  x_k^{a_k}, which is a term on the right-hand-side. Thus a term such as x_1^{a_1} x_2^{a_2} \cdots  x_k^{a_k} is another way to notate an unordered sample and it represents \frac{n!}{a_1! \ a_2! \cdots a_k!} many ordered samples. Thus \frac{n!}{a_1! \ a_2! \cdots a_k!} x_1^{a_1} x_2^{a_2} \cdots  x_k^{a_k} is the result of aggregating all the ordered samples for a given unordered sample. Summing all these aggregated results give all the possible ordered samples in the multinomial experiment.

___________________________________________________________________________

Examples

The best way to absorb the concepts discussed here is to work excises. Three more examples are presented. Several exercises are given in the section below.

Example 2
Suppose that 11 new hires are assigned to four office locations – the headquarter, branch office North, branch office South and branch office East.

  • How many assignments can be made?
  • How many assignments can be made such that 3 new hires are assigned to the headquarter, 3 new hires are assigned to branch office North, 4 new hires are assigned to branch office South and 1 new hire is assigned to branch office East?
  • How many assignments can be made such that 3 new hires are assigned to one location, 3 new hires are assigned to another location, 4 new hires are assigned to another location and 1 new hire is assigned to the remaining location?

The answer to the first question is 4^{11} = 4,194,304. The answer to the second question is a multinomial coefficient.

    \displaystyle \frac{11!}{3! \ 3! \ 4! \ 1!}=46200

Each assignment can be viewed as a string of 11 letters, each of which is chosen from H, N, S and E (these are the ordered samples discussed above). An unordered sample is a sequence of 4 numbers. For example, the unordered sample for the second question is (3, 3, 4, 1). The third question would include other unordered samples with the numbers summing to 11, e.g. (1, 3, 4, 3). Basically we are trying to divide the 4 letters into 3 groups, one group of one letter appearing once, one group with two letters appearing 3 times each, one group with one letter appearing 4 times. This is another multinomial coefficient.

    \displaystyle \frac{4!}{1! \ 1! \ 2!}=12

Thus the answer to the third question is 12 x 46,200 = 554,400. \square

Example 3
Suppose that 16 new hires are assigned to four office locations – the headquarter, branch office North, branch office South and branch office East. Six of the new hires are engineers and only work in the headquarter or branch office East. The other ten new hires are technicians and can be assigned to any one of the four locations.

  • How many assignments can be made?
  • How many assignments can be made such that four of the engineers will work in the headquarter and two of the new technicians will work in the headquarter, 3 new technicians are assigned to branch office North, 4 new technicians are assigned to branch office South and 1 new technician is assigned to branch office East?
  • How many assignments can be made such that four of the engineers will work in either the headquarter or branch office East and two of the new technicians will work in one of the 4 locations, 3 new technicians are assigned to another location, 4 new technicians are assigned to a third location and 1 new technician is assigned to the remaining location?

There are two assignments, one for engineers and one for technicians. The answer would be obtained by multiplying the two assignment counts. The answer to the first question is 2^6 \cdot 4^{11} = 640,000. The following is the answer to the second question.

    \displaystyle \binom{6}{4} \times \frac{10!}{2! \ 3! \ 4! \ 1!}=15 \times 12600=189000

Note that there are \binom{6}{4} = 15 ways to 4 of the engineers to the headquarter (and thus assigning the other two engineers to the other office). The number of ways to assign 10 technicians to the 4 locations in the indicated numbers is the multinomial coefficient indicated above (the one resulting in 12,600). The following is the answer to the third question.

    \displaystyle \binom{6}{4} \times 2 \times \frac{10!}{2! \ 3! \ 4! \ 1!} \times \frac{4!}{1! \ 1! \ 1! \ 1!}=30 \times 302400=9072000

Note that for the third question, a second multinomial coefficient on the locations is required for each type of workers. \square

Example 4
Fifteen identical brand new mail delivery trucks are assigned to 5 post offices.

  • How many assignments can be made?
  • How many assignments can be made such that each post office is assigned at least one truck?

The problem can be done using the combinatorial argument with stars and bars discussed above. The 15 trucks are stars and there 4 bars creating 5 spaces representing the post offices. Any assignment of trucks would be like a string of 15 stars and 4 bars. The following is an example.

    * * * * | * * * * * * | | * * * * *

The above star-and-bar diagram shows that 4 trucks are assigned to the first post office, 6 trucks are assigned to the second post office, zero trucks are assigned to the third post office and 5 trucks are assigned to the fourth post office. What is more important to notice is that there are 19 positions in the diagram. The problem is to choose 15 of them to place the stars. The following is the answer to the first question.

    \displaystyle \binom{19}{15}=3876

The second question requires that each space has at least one star. We place one star in each space. There are 11 stars left. We now consider 11 stars and 3 bars. The following is the number of ways of arranging 11 stars in 14 positions.

    \displaystyle \binom{14}{11}=364

Thus there are 364 ways to assign 15 trucks to four post offices such that each office gets at least one truck. \square

___________________________________________________________________________

Exercises

Exercise 1
A father is to distribute 9 gifts to his three children.

  1. In how many ways can the gifts be distributed if the eldest child is to receive 2 gifts, the middle child is to receive 2 gifts and the youngest child is to receive 5 gifts?
  2. In how many ways can the gifts be distributed if one child is to receive 2 gifts, another child is to receive 2 gifts and the remaining child is to receive 5 gifts?

Exercise 2
Eleven job assignments are randomly distributed to four workers Marcus, Issac, Samantha and Paul. In how many ways can these jobs be assigned to the four workers such that Marcus will receive one job, Issac will receive 4 jobs, Samantha will receive 4 jobs and Paul will receive 2 jobs?

Exercise 3

  1. Ten students are to be assigned to two math classes. How many ways can the students be divided into the two math classes with 5 students in each class?
  2. Fifteen students are to be assigned to three math classes. How many ways can the students be divided into the three math classes with 5 students in each class?

Exercise 4
An investor has 25 thousand dollars to invest among 5 possible investments. The amount to invest in each investment is in the unit of one thousand dollars. Suppose that the entire amount of 25 thousand dollars is to be invested.

  1. How many different investment strategies are possible?
  2. How many different investment strategies are possible if at least one unit is put into each investment choice?

Exercise 5
Recall the Multinomial Lunch Problem. Each of the twelve people have 3 choices for lunch – McDonald, Burger King and IN-N-OUT, all fast food restaurants. The choice for each person is independent of the choices of the other diners. In how many ways can the lunch choices for this 12-person group be made in such a way that each restaurant is visited by at least 3 diners?

\text{ }

\text{ }

\text{ }

\text{ }

\text{ }

\text{ }

\text{ }

\text{ }

Answers

Exercise 1

  1. 756
  2. 2,268

Exercise 2
34,650

Exercise 3

  1. 252
  2. 756,756

Exercise 4

  1. 23,751
  2. 10,626

Exercise 5
256,410
Hint: There are three cases, each of which requires the use of two multinomial coefficients, one for the diners and one for the restaurants.

___________________________________________________________________________
\copyright 2017 – Dan Ma

Advertisements

Binomial coefficients

This is the third post in a series of posts on combinatorial analysis. The post is opened with the following problem.

    Figure 1
    path-problem

Path Problem
Point Q is 7 blocks east and 6 blocks north of Point P (see Figure 1). A person wants to walk from Point P to Point Q by walking 13 blocks. Assume that all the paths from any point to any point in the above diagram are available for walking. In how many ways can the person go from Point P to Point Q?

One way to solve this problem is to count the number of paths in a “brute force” approach by tracing all possible paths in the diagram. The total number of paths is 1,716. So counting directly is not a practical approach. It turns out that the path problem and other similar problems will become routine once the concept of binomial coefficients is clearly understood.

___________________________________________________________________________

Review

First we highlight the important concepts introduced in the previous post on permutations and combinations.

The number n!, read n factorial, is defined by the product n!=n \times (n-1) \times (n-2) \times \cdots \times 3 \times 2 \times 1. It is the number of ordered arrangements of n objects.

For convenience we define 0! = 1. An ordered arrangement of a set of objects is called a permutation. How many permutations are there of 7 objects? For example, how many ways can the following 7 people be arranged in a row for a group photo: Abby, Beth, Charlie, David, Edward, Frank, and George? An equivalent question: how many permutations are there of the letters A, B, C, D, E, F and G? The answer is 7! = 5040. The reasoning is based on the multiplication principle (see here). There are 7 people to choose for the first position, and there are 6 people to choose from for the second position after the first position is fixed, and so on. Thus there are 7 x 6 x 5 x 4 x 3 x 2 x 1 = 5040.

The number of permutations of k objects chosen from n objects is defined by \displaystyle P(n,k)=\frac{n!}{(n-k)!}.

Out of the 7 people mentioned above, how many ways can three people be randomly selected to receive three cash prizes of $1000, $500 and $100? An equivalent question: how many permutations are there of 3 letters chosen from the 7 letters A, B, C, D, E, F and G? The answer can also be obtained by the multiplication principle. There are 7 x 6 x 5 = 210 permutations, which is identical to P(7,3)=\frac{7!}{(7-3)!}=\frac{7!}{4!}=7 \times 6 \times 5.

The number of different groups of size k that can be chosen from a set of n distinct objects is \displaystyle \binom{n}{k}=\frac{n!}{k! \ (n-k)!} where n is a positive integer and k=0,1,2,\cdots,n. This number is called the number of possible combinations of n objects chosen k at a time. The values \displaystyle \binom{n}{k} are often referred to as binomial coefficients because of their connection with the binomial theorem.

Out of the 7 people mentioned above, how many ways can three people be randomly selected to receive three cash prizes, each of which is $1000? An equivalent question: how many three-letter sets can be chosen from the 7 letters A, B, C, D, E, F and G? Here, the order does not matter. The answer 210 obtained previously is too large because it contains many repeats. For example, the letters A, B and C has 3! = 6 permutations. In the answer 210, each 3-letter combination occurs 6 times. So the answer is 210 / 6 = 35, which is identical to \binom{7}{3}=35.

Other notations for the binomial coefficient \binom{n}{k} are _nC_k, C(n,k) and C_{n,k}. Permutation and combination can all be computed directly in terms of factorials. The calculation can also be done using calculator or software. Most calculators have functions for combination (binomial coefficient) and permutation. In Excel, the binomial coefficient is computed by the formula =COMBIN(n, k), which will give the result as \binom{n}{k}. The Excel formula for permutation is =PERMUT(n,k).

___________________________________________________________________________

The Path Problem

Recall that the problem is to walk 7 blocks East and 6 blocks North starting at point P. Let use E to mean walking one block East and N to mean walking one block North. The following is one specific path that the person might take.

    Figure 2
    path-problem-specific-path

The path in Figure 2 can be described by the string E-E-N-N-N-E-N-E-E-E-N-N-E. Of course there are many other possible paths that can be taken. Observe that each possible path correspond to a string of 7 E’s and 6 N’s. On the other hand, each possible 13-character string with 7 E’s and 6 N’s correspond to a path in the problem. So the problem is about finding the number of possible 13-character string with 7 E’s and 6 N’s. In other words, we are in interested in all the ways we can write 7 E’s and 6 N’s in the following 13 positions.

    \LARGE \Box \ \ \ \Box \ \ \ \Box \ \ \ \Box \ \ \ \Box \ \ \ \Box \ \ \ \Box \ \ \ \Box \ \ \ \Box \ \ \ \Box \ \ \ \Box \ \ \ \Box \ \ \ \Box

Writing 7 E’s and 6 N’s in 13 boxes is the same as choosing 7 boxes out of 13 boxes to fill with 7 E’s. The number of ways to choose 7 objects out of 13 objects is simply \binom{13}{7}=1716.

___________________________________________________________________________

Several Interpretations of the Binomial Coefficients

The path problem makes it clear that there are several ways of looking at the binomial coefficient \displaystyle \binom{n}{k}. Each of the following is a count that is computed by \displaystyle \binom{n}{k}.

  1. The number of different groups of size k that can be chosen from a set of n objects.
  2. The number of ways a set of n objects can be labeled by 0 or 1.
  3. The number of different n-character strings that can be formed by using k of one symbol and n-k of another symbol.
  4. The number of permutations of n items, of which k are alike and n-k are alike.
  5. A random experiment consists of n trials, each of which results in one of two outcomes (outcome 1 and outcome 2). The number of ways the experiment can be performed such that k of the trials result in outcome 1 and n-k of the trials result in outcome 2.
  6. An urn contains a large number of balls, each of which is labeled 0 or 1. Randomly select a ball n times from the urn with replacement. The number of ways the balls can be selected in such a way that k of the balls are labeled 1 and n-k of the balls are labeled 0.

The same theme runs through the above 6 ideas. The first one is the standard definition of binomial coefficient. The number \binom{n}{k}, according to idea 1, would be the number of all possible subsets of size k that can be chosen from a set of n objects. We can also think of choosing a subset as a random experiment of n trials where each trial has two distinct outcomes. Out of the n objects, we only want k of them with k \le n. Then for each object, there is a decision – choose it or not choose it (two distinct outcomes). So there are n trials and each trial has two outcomes (idea 5). We can also think of a random sampling process too. Perform each trial that ends in two outcomes will be like a random sampling from a population consisting of 0’s and 1’s (idea 6).

Going back to the n objects in idea 1, deciding whether to choose each object is also like labeling that object with 0 or 1 (idea 2). Since each object generates two outcomes (choose or not choose), the n decisions successively is like a string of n characters, each of which is one of two distinct symbols (ideas 3 and 4).

Though the interpretation may be nuanced, the calculation of the binomial coefficient is straightforward. Thinking of the binomial coefficient as the number of ways to making a series of two-outcome decisions is crucial to the understanding of binomial distribution. In the remainder of the post, we discuss other properties of the binomial coefficients.

___________________________________________________________________________

The Binomial Theorem

The sum of two symbols, say x+y, is called a binomial. The binomial theorem is a formula for deriving the power of a binomial, i.e. for deriving (x+y)^n for n=1,2,3,\cdots. We explain the idea behind the formula. The ideas 3 and 4 discussed above are particularly useful ways of looking at the binomial theorem. Consider the first several expansions.

    \displaystyle \begin{aligned} (x+y)^1&=x+y  \end{aligned}

    \displaystyle \begin{aligned} (x+y)^2&=(x+y) (x+y)\\&=xx+xy+yx+yy \\&=x^2+2xy+y^2 \end{aligned}

    \displaystyle \begin{aligned} (x+y)^3&=(x+y) (x+y) (x+y)\\&=xxx+xxy+xyx+yxx+xyy+yxy+yyx+yyy \\&=x^3+3x^2y^1+3xy^2+y^3  \end{aligned}

    \displaystyle \begin{aligned} (x+y)^4&=(x+y) (x+y) (x+y) (x+y)\\&=xxxx+xxxy+xxyx+xyxx+yxxx\\& \ \ +xxyy+ xyxy+xyyx+yxxy+yxyx+yyxx\\&\ \ +xyyy+yxyy+yyxy+yyyx+yyyy \ \ \ \ \ \ \ \ \ \text{(16 terms)} \\&=x^4+4x^3y^1+6x^2y^2+4xy^3+y^4  \end{aligned}

    \displaystyle \begin{aligned} (x+y)^5&=(x+y) (x+y) (x+y) (x+y) (x+y)\\&=xxxxx+xxxxy+xxxyx+xxyxx+xyxxx+yxxxx \cdots \\& \ \ +xxxyy+xxyxy+xyxxy+yxxxy+xxyyx \\& \ \ + xyxyx+yxxyx+xyyxx+yxyxx+yyxxx \ \ + \cdots (\text{32 terms}) \\&=x^5+5x^4y^1+10x^3y^2+10x^2y^3+5xy^4+y^5  \end{aligned}

The above derivations are deliberately detailed to make a point. For example, the result of (x+y)^5 is product of 5 identical factors of x+y. Multiplying out these five factors of x+y results in 2^5=32 terms. Each of these 32 items is a string of x’s and y’s. Let’s look at the strings with 3 x’s and 2 y’s. they are:

    xxxyy+xxyxy+xyxxy+yxxxy+xxyyx

    xyxyx+yxxyx+xyyxx+yxyxx+yyxxx

To find out how many such strings are possible, we need to find out the number of 5-letter strings of which 3 letters are the same and the other 2 letters are the same. That would be \binom{5}{3}=\binom{5}{2}=10.

In general, (x+y)^n is obtained by multiplying n factors of x+y. In the expansion, a typical term is x^{n-j} y^j, which can be viewed as a n-letter string of n-j of the letters are x and j of the letters are y. We can also look at the situation as choosing j of the n positions to write the letter y. There are \binom{n}{j} many ways of writing j instances of y in n positions. Thus \binom{n}{j} x^{n-j} y^j would be one of the terms in the final result of the expansion. This is the reason why \binom{n}{j} is referred to as binomial coefficients.

The binomial theorem shows how to derive the power of a binomial. The formula is: \displaystyle (x+y)^n=\binom{n}{0} x^n+\binom{n}{1} x^{n-1}y^1+\binom{n}{2} x^{n-2}y^2+\cdots+\binom{n}{j} x^{n-j}y^j+\cdots+\binom{n}{n-1} x^{1}y^{n-1}+\binom{n}{n} y^n.

A more compact way of stating the binomial theorem is: \displaystyle (x+y)^n=\sum \limits_{j=1}^n \binom{n}{j} x^{n-j}y^j.

___________________________________________________________________________

A Recursive Formula

The combination \displaystyle \binom{n}{k} can be evaluated using calculator or software. We may not need the following formula for the purpose of calculation. However, it does provide a great deal of insight.

    \displaystyle \binom{n}{k}=\binom{n-1}{k}+\binom{n-1}{k-1}

The formula can be derived by the following thought process. A k-subset of a set of n objects is a subset consisting of exactly k elements. The quantity \displaystyle \binom{n}{k} is the number of k-subsets of a set of n objects. For our purpose here, the n objects are the numbers 1,2,3,\cdots,n. Let fix one of the objects, say 1. There are two types of k-subsets – the ones containing 1 and the ones not containing 1. The number of k-subsets of the first type is \displaystyle \binom{n-1}{k-1}. The point 1 is known to be in each k-subset of the first type and the other k-1 points must be chosen from n-1 objects. The number of k-subsets of the second type is \displaystyle \binom{n-1}{k}. The point 1 is not in each k-subset of the second type. So all k are chosen from the other n-1 objects. Thus \displaystyle \binom{n}{k} is the sum of these two numbers.

___________________________________________________________________________

Pascal’s Triangle

No discussion of the binomial coefficients is complete without mentioning the Pascal’s triangle. The triangle is usually represented as an isosceles triangle (or an equilateral triangle). We use a right angle representation.

\begin{array}{rrrrrrrrr}      1 & \text{ } & \text{ } & \text{ } & \text{ } & \text{ } & \text{ } & \text{ } & \text{ }    \\ 1 & 1 & \text{ } & \text{ } & \text{ } & \text{ } & \text{ } & \text{ } & \text{ }   \\ 1 & 2 & 1 & \text{ } & \text{ } & \text{ } & \text{ } & \text{ } & \text{ }   \\ 1 & 3 & 3 & 1 & \text{ } & \text{ } & \text{ } & \text{ } & \text{ }  \\ 1 & 4 & 6 & 4 & 1 & \text{ } & \text{ } & \text{ } & \text{ }  \\ 1 & 5 & 10 & 10 & 5 & 1 & \text{ } & \text{ } & \text{ }  \\ 1 & 6 & 15 & 20 & 15 & 6 & 1 & \text{ } & \text{ }  \\ 1 & 7 & 21 & 35 & 35 & 21 & 7 & 1 & \text{ }  \\ 1 & 8 & 28 & 56 & 70 & 56 & 28 & 8 & 1        \end{array}

Each row in the triangle gives the binomial coefficients in the expansion (x+y)^n, starting at n=0. Thus the second row gives the coefficients for the expansion of (x+y)^1, the third row gives the coefficients for (x+y)^2 and so on. The most interesting part about the triangle is that it is recursive. Each row is derived from the previous row.

Each cell in a row is the sum of the two cells in the row above it – the cell directly above and the cell to the left of that. For example, the last row in the above triangle shows the coefficients for (x+y)^8. The third number from the left is 28, which is the sum of 21 and 7. The fourth element 56 is the sum of 35 (the cell directly above it) and 21 (the cell to the left of 35). As a result, any row can be derived from the previous row. The next row will be for (x+y)^9 have the following coefficients:

    1, 9, 36, 84, 126, 126, 84, 36, 9, 1

The Pascal’s triangle is really the recursive formula discussed above. A typical cell in the triangle is the binomial coefficient \binom{n}{k}. The cell directly above it is \binom{n-1}{k}. The cell to the left of that one is \binom{n-1}{k-1}.

___________________________________________________________________________

How Many Subsets Are There of a Given Set?

Given n objects, we now know that there are \binom{n}{k} many sets of k elements from these n objects. There are \binom{7}{3}=35 many sets of 3 letters from the letters C, K, M, L, T, P and O. There are several ways to come up with the answer.

One way is to use the binomial theorem. Let x=y=1 and n=7. Write out the binomial theorem. The following is the result.

    \displaystyle 128=2^7=(1+1)^7=\binom{7}{0}+\binom{7}{1}+\binom{7}{2}+\binom{7}{3}+\binom{7}{4}+\binom{7}{5}+\binom{7}{6}+\binom{7}{7}

There are 128 different subsets of the 7 letters (or any other set of 7 objects). The reason is given in the right side of the above equation, which is the sum of the number of subsets with zero elements, plus the number of subsets with exactly one element, plus the number of subsets with exactly two elements, …, plus the number of subsets with seven elements.

When buying a hamburger at a fast food restaurant, customers can always add toppings. A basic hamburger comes with a beef patty inside a bun. A customer can choose to add toppings from this list: Cheese, Ketchup, Mustard, Lettuce, Tomato, Pickles and Onion (compare the first letters with the 7 letters indicated earlier). How many different hamburgers can be created? Basically the problem is to find out the number of different subsets of the 7 letters C, K, M, L, T, P and O. The answer is of course 2^7=128. Interestingly the answer of 128 includes one choice that is the plain burger with no toppings – just the beef and the bun. So the total number of different hamburgers with at least one topping would be 127.

In general, the total number of subsets of a set with n elements is 2^n. This is confirmed by the plugging x=1 and y=1 into the binomial theorem: \displaystyle 2^n=(1+1)^n=\binom{n}{0}+\binom{n}{1}+\binom{n}{2}+\binom{n}{3}+\cdots+\binom{n}{n-1}+\binom{n}{n}

___________________________________________________________________________

Exercises

Exercise 1
Starting at Point P, a student walks 6 blocks to a diner for breakfast (represented by the red dot in the following diagram). After breakfast, the students 7 blocks to school that is located at Point Q. In how many ways can the student walk from Point P to Point Q? Assume that all the paths from any point to any point in the above diagram are available for walking.

    Figure 2

Exercise 2
In ordering a pizza, a customer can choose from a list of 10 toppings: mushroom, onion, olive, bell pepper, pineapple, spinach, extra cheese, sausage, ham, and pepperoni.

  • How many different pizza can the customer create if any topping in the list is desirable to the customer and if the customer chooses at least one topping?
  • Suppose that the customer is a meat lover. He will always choose the three meat toppings: sausage, ham and pepperoni. He will also choose up to 3 vegetable toppings. How many different pizza can the customer create?

Exercise 3
A committee of 8 people is to be chosen from a class consisting of 8 men and 9 women. If the committee must consist of at least 3 men and at least 4 women, how many different committees are possible?

Exercise 4

  • In how many ways can a father divide 8 gifts among his two children if the eldest is to receive 5 gifts and the other child 3 gifts?
  • In how many ways can a father divide 8 gifts among his two children if one child is to receive 5 gifts and the other child 3 gifts?

\text{ }

\text{ }

\text{ }

\text{ }

\text{ }

Answers

Exercise 1
315

Exercise 2

  • 1023
  • 792

Exercise 3
15876

Exercise 4

  • 56
  • 112

___________________________________________________________________________
\copyright \ 2017 \text{ by Dan Ma}

Permutations and combinations

This is the second post in a series of posts on combinatorial analysis. The first post is on the multiplication principle. This post introduces permutations and combinations.

The starting point of the discussion on counting is the idea that if one event can occur in M ways and another event can occur, independent of the first event, in N ways, then the two events together can occur in M \times N ways. This idea is often called the multiplication principle and is introduced in this previous post.

___________________________________________________________________________

Factorial

Consider the eight letters A, B, C, D, E, F, G and H. How many ways can we order the 8 letters? The following shows two ordered arrangements.

    A-B-C-D-E-F-G-H

    C-B-F-A-D-G-H-E

The multiplication principle can help us determine the number of all arrangements of 8 letters. We are trying to fill 8 objects into 8 positions. There are 8 choices for the first position. After the first letter is fixed, there are 7 letters for the second position. Continue with this approach, there will be only one letter left for the eighth position. In total, there are 8 x 7 x 6 x 5 x 4 x 3 x 2 x 1 = 40,320 ordered arrangements of 8 letters. A convenient notation for the answer is 8!, read 8 factorial. It is simply the product of the integers from 8 down to 1.

In general, if n is a positive integer, n!, read n factorial, is the product of all the integers from n down to 1.

n!=n \times (n-1) \times (n-2) \times \cdots \times 3 \times 2 \times 1

For the above definition of n! to make sense, n is a positive integer. We define 0! = 1 to make certain formulas easy to calculate.

According to the above example, there are 8! many different ordered arrangements of 8 distinct objects (objects that are distinguishable from one another). Any ordered arrangement such as C-B-F-A-D-G-H-E is called a permutation of the 8 letters. In general, a permutation is an ordered arrangement of a set of objects that are distinguishable from one another. From the above example of 8 letters, we have the following observation.

    The total number of permutations of n distinct objects is n!.

The G8 summit is an annual meeting between leaders from eight of the most powerful countries in the world. There would be 8! = 40,320 different ways to line up the 8 leaders for a group photo. A permutation can be viewed as ranking. In a soccer league of 8 teams, there would be 8! = 40,320 different possible rankings.

The factorial grows very rapidly. In ordering 9 objects, there are 9! = 362,880 permutations.

    10! = 3,628,800
    11! = 39,916,800
    12! = 479,001,600
    15! = 1,307,674,368,000
    17! = 355,687,428,096,000
    20! = 2,432,902,008,176,640,000

Thus the number of possible rankings of 12 job applicants is over 479 millions. The number of possible rankings of 20 job applicants would be a staggering number, over 2.4 quintillions! A quintillion is 10 raised to 18. In contrast, a trillion is 10 raised to 12.

Example 1
Suppose that you have 12 books with 4 of them being novels, 4 of them being math books, 3 of them being science books and one being a history book. You want to place the books on the same level in a bookshelf. If the books are put in any order, then there would be 12! = 479,001,600 different placements of books. In how many ways can the books be placed in the bookshelf if all books of the same subject are to be together?

The last question makes use of the multiplication principle in a subtle way. Consider the books arranged by subjects in this order: novel-math-science-history. By the multiplication principle, there are 4! x 4! x 3! x 1! = 24 x 24 x 6 x 1 = 3,456 many ordered arrangements. This is just for one specific way of the order of the 4 subjects – novel-math-science-history. There are 4! = 24 ways of order 4 subjects. Then the answer to the last question is 24 x 3,456 = 82,944. \square

___________________________________________________________________________

Permutation

Sometimes we do not order the entire set of n objects. We only order a subset of the n objects. Using the above example of 8 letters A, B, C, D, E, F, G and H, in how many ways can we arrange three letters from these 8 letters? The multiplication principle is at work here. We are attempting to fill 3 positions using 8 letters at our disposal. The first position has 8 choices. The second position has 7 choices and the third position has 6 choices. There are a total of 8 x 7 x 6 = 336 ordered arrangements. Three examples are A-B-C, G-A-H and E-B-F. Any one of these 336 ordered arranged is called 3-permutation of 8 letters. We can also call it a permutation of 8 letters taken 3 at a time.

Example 2
To put the 8 letters in a context, suppose that a math club has 8 students – Abby, Beth, Chloe, Diane, Edward, Frank, George and Henry. In how many ways can we choose 3 students to fill three positions of President, Vice President and Treasurer of the math club? Based on the preceding discussion, the answer is 8 x 7 x 6 = 336. The following is a presentation of the calculation that gives the hint for a formula.

    \displaystyle 8 \times 7 \times 6=\frac{8 \times 7 \times 6 \times 5 \times 4 \times 3 \times 2 \times 1}{5 \times 4 \times 3 \times 2 \times 1}=\frac{8!}{5!}=\frac{8!}{(8-3)!}

In general, any ordered arrangement of k objects from a set of n objects is called a k-permutation of n objects. We can also call any such ordered arrangement a permutation of n objects taken k at a time. Here, we attempt to fill k positions with objects selected from n objects with k \le n. How many permutations of n objects taken k at a time? There are

    n \times (n-1) \times (n-2) \times \cdots \times (n-(k-1))

many different permutations of n objects taken k at a time. Some common notations for this number are P(n,k) or P_{n,k}. According to the hint in Example 2, the following is a formula for P(n,k).

\displaystyle P(n,k)=\frac{n!}{(n-k)!} is the number of permutations of n objects taken k at a time.

Note that P(n,n) is simply n!.

Example 3
In evaluating 20 job applicants, there are 20! many possible rankings, which is over 2 quintillions as discussed above. A more sensible goal is to narrow the list to the top 5 candidates. The number of permutations of 20 candidates taken 5 at a time is

    \displaystyle P(20,5)=\frac{20!}{15!}=20 \times 19 \times 18 \times 17 \times 16=\text{1,860,480}

Example 4
Suppose that the psychology club at a university has 12 members in the lower division (consisting of freshmen and sophomores), 10 members in the upper division (consisting of juniors and seniors). Four awards (achievements in terms of overall grade, leadership potential, volunteerism, and public speaking) are given to each group. What is the total number of different outcomes if

  • a student can receive more than one award,
  • a student can receive only one award.

If a student can be awarded more than once, then there are 12^4 ways of giving out awards in the lower division and 10^4 ways in the upper division, leading to the following answer.

    \displaystyle 12^4 \times 10^4 = \text{207,360,000}

If a student can only be awarded once, then the following is the answer.

    \displaystyle P(12,4) \times P(10,4) =(12 \times 11 \times 10 \times 9) \times (10 \times 9 \times 8 \times 7)= \text{59,875,200}

___________________________________________________________________________

When Some Objects are Indistinguishable

In calculating the number of permutations, the objects are assumed to be distinct. We now look at the situation where objects are separated into groups of indistinguishable objects. The goal is still finding the number of ordered arrangements. The number of permutations as calculated above would be an over count. The answer is obtained by dividing the inflated count by the amount of over counting.

Example 5
How many different letter arrangements can be formed from the letters in the word BANANA?

There are 6 letters in BANANA. Three of the letters are indistinguishable (A) and two of them are indistinguishable (N). If all the letters are distinguishable, then there are 6! permutations. But 6! = 720 over counts. By how many?

Pretend all the letters are distinguishable. Take the arrangement A_1 \ A_2 \ A_3 \ B \ N_1 \ N_2. Now permute the A’s and the N’s. Doing so should result in 3! x 2! = 12 permutations.

    A_1 \ A_2 \ A_3 \ B \ N_1 \ N_2
    A_1 \ A_3 \ A_2 \ B \ N_1 \ N_2
    A_2 \ A_1 \ A_3 \ B \ N_1 \ N_2
    A_2 \ A_3 \ A_1 \ B \ N_1 \ N_2
    A_3 \ A_1 \ A_2 \ B \ N_1 \ N_2
    A_3 \ A_2 \ A_1 \ B \ N_1 \ N_2

    A_1 \ A_2 \ A_3 \ B \ N_2 \ N_1
    A_1 \ A_3 \ A_2 \ B \ N_2 \ N_1
    A_2 \ A_1 \ A_3 \ B \ N_2 \ N_1
    A_2 \ A_3 \ A_1 \ B \ N_2 \ N_1
    A_3 \ A_1 \ A_2 \ B \ N_2 \ N_1
    A_3 \ A_2 \ A_1 \ B \ N_2 \ N_1

But in reality, all of the 12 permutations are the same: A \ A \ A \ B \ N \ N. So each unique permutation appears 12 times in the count 6!. Thus the correct count is

    \displaystyle \frac{6!}{3! \ 2! \ 1!}=60

In general, \displaystyle \frac{n!}{m_1! \ m_2! \ \cdots \ m_k!} is the number of permutations of n objects, of which m_1 are alike, m_2 are alike, …, m_k are alike, where m_1+\cdots+m_k=n.

___________________________________________________________________________

Combination

In all the above instances of counting, the focus is to find all ordered arrangements (or permutations) of a set of objects. We now focus on the different subsets of k elements that can be chosen from a set of n objects. In this setting, the goal is not to count ordered arrangements. The goal is to count the number of subsets of a certain size. For example, let’s say you want to buy a 2-scoop ice cream cone of different flavors chosen from 3 flavors – Chocolate (C), Strawberry (S) and Vanilla (V). Suppose you do not care which flavor is on top and which one is at the bottom. The different subsets of size 2 would be: CS, CV, and SV. Here, CS is not an ordered arrangement. It just means that the two chosen flavors are chocolate and strawberry. If order matters, then there would be six outcomes – CS, SC, CV, VC, SV, and VS. So the approach is to find the number of permutations (an over count) and then divide the amount of over count.

Example 6
Suppose that a math club has 8 students – Abby, Beth, Chloe, Diane, Edward, Frank, George and Henry. In how many ways can we choose three students to form a committee of three?

The number of permutations of three students is 8 x 7 x 6 = 336. This count is inflated since there are many repeats reflected in this count. For example, if the chosen students are Abby, Beth and Chloe, then there are 3! = 6 ways to order these three students – ABC, ACB, BAC, BCA, CAB and CBA. So each set of three students is reflected 6 times in the inflated count of 336. So the answer is 336 / 6 = 56. Thus there are 56 possible groups of three students when three students are chosen from 8 students. The following calculation makes the formula clear.

    \displaystyle \frac{8 \times 7 \times 6}{3!}=\frac{8 \times 7 \times 6 \times 5!}{3! \ 5!}=\frac{8!}{3! \ 5!}

In general, \displaystyle \binom{n}{k}=\frac{n!}{k! \ (n-k)!} is the number of different groups of size k that could be chosen from a set of n distinct objects where k=0,1,2,\cdots,n.

The number \binom{n}{k} is called a binomial coefficient and is pronounced “n choose k.” Other notations for \binom{n}{k} are _nC_k, C_{n,k} and C(n,k).

Example 7
A committee of five is to be chosen from a group of 11 people consisting of 5 men and 6 women.

  • How many different committees of five people can be formed?
  • How many different committees consisting of 2 men and 3 women can be formed?
  • How many different committees of five consisting of either 2 men or 2 women?

The number of possible committees of five would be \binom{11}{5}=462. For the second problem, note that there are \binom{5}{2}=10 possible groups of 2 men and there are \binom{6}{3}=20 groups of 3 women. By the multiplication principle, there are 10 x 20 = 200 different groups of 5 people consisting of 2 men and 3 women.

For the third problem, there are two cases: a committee of 2 men and 3 women or a committee of 2 women and 3 men. The answer is the sum of these two cases. Thus the answer is

    \displaystyle \binom{5}{2} \times \binom{6}{3}+\binom{5}{3} \times \binom{6}{2}=10 \times 20+10 \times 15=350

___________________________________________________________________________

More Examples

We present a few more examples to demonstrate the ideas discussed here. Exercises are given at the end.

Example 8
A child plays with a set of 12 blocks, 4 of which are red, 4 of which are black, 3 of which are white and one of which is yellow. In how many ways can these blocks be arranged in a row? In how many ways can these blocks be arranged in a row in such a way that all blocks of the same color are together?

Note the similarity with Example 1. Both problems concern with a collection of 12 objects that can be separated into 4 sub groups. In Example 1, the objects in each sub group are distinguishable while the objects in this example are indistinguishable in each sub group. As a result, the formula discussed in the section “When Some Objects are Indistinguishable” would apply. Here’s the number of different ways to order the blocks.

    \displaystyle \frac{12!}{4! \ 4! \ 3! \ 1!}=\text{138,600}

For the second question, the answer would be 4! = 24. Unlike Example 1, we do not need to order the blocks within the same color since they are indistinguishable. \square

Example 9
The tennis team in School A has 12 players and the tennis team in School B has 14 players. If 5 players are selected from each team and then paired off for a tennis game, how many results are possible?

There are \binom{12}{5}=792 ways of choosing 5 players from Team A. There are \binom{14}{5}=2002 ways of choosing 5 players from Team B. There are 5! = 120 ways in pairing off the two teams of 5 players. Thus there are 792 x 2002 x 120 = 190,270,080 ways of choosing two teams and then pairing them off for a tennis game. \square

Example 10
A company is made up of 3 departments with the one department consisting of 8 employees, the second department consisting of 10 employees and the third department consisting of 5 employees. The owner of the company decides to promote two employees chosen from the 23 employees. Suppose that the two employees are selected at random. How many ways can the two employees be selected if

  • both promotions are from the same department?
  • both promotions are from different departments?

The problem does not mention the positions for the promotion. Thus we assume that the two positions are identical. The first question has three cases based on the departments. The following is the sum of the three cases.

    \displaystyle \binom{8}{2}+\binom{10}{2}+\binom{5}{2}=28+45+10=83

For the second question, there are \binom{3}{2}=3 cases, based on pairs of departments. The following is the sum of the 3 cases.

    \displaystyle \binom{8}{1} \times \binom{10}{1}+\binom{8}{1} \times \binom{5}{1}+\binom{10}{1} \times \binom{5}{1}=80+40+50=170

Example 11
The problem is the same as Example 10 except that the two promotions are for President and Vice President.

Now that the two promotions are different, order matters. Thus use permutations to perform the calculations. the answer to the first question is:

    \displaystyle \frac{8!}{(8-2)!}+\frac{10!}{(1-2)!}+\frac{5!}{(5-2)!}=56+90+20=166

In the second question, we still pick one person from each department, but it needs to be multiplied by two since the positions are different.

    \displaystyle 2 \times 8 \times 10+2 \times 8 \times 5+2 \times 10 \times 5=160+80+100=340

___________________________________________________________________________

Exercises

Exercise 1
How many different letter arrangements can be formed from the letters in the word MISSISSIPPI?

Exercise 2
In how many ways can a manager assigns 3 different jobs at random to 11 available workers?

Exercise 3
Suppose that the license plates for a certain state consist of three numbers and three letters. How many different license plates are possible if

  1. the letters must come before the numbers?
  2. all letters must appear together and all numbers must appear together?
  3. only the letters must appear together?
  4. there is no restriction on the positions of the letters and numbers?

Exercise 4
In ordering a pizza, cheese and tomato sauce come with the pizza. The customer can also choose to add toppings from the following lists:

    Meat items: ham, sausage, pepperoni, salami, chicken.
    Vegetable items: pineapple, olive, mushroom, onion, bell peppers.

  1. How many distinct pizzas are possible?
  2. Suppose that a customer does not like salami and sausage in the meat list and onion and olive in the vegetable list and thus will not add these toppings. How many different pizzas are possible?
  3. Suppose that a customer would like to add two meat items and three vegetable items as toppings. How many different pizzas are possible?
  4. Suppose that a customer would like to add exactly 5 toppings. How many different pizzas are possible?
  5. Suppose that a customer would like to add at most 5 toppings. How many different pizzas are possible?

Exercise 5

  1. Twelve workers are randomly assigned to work in four factories with 3 workers assigned to each factory as assembly line workers. How many ways can the workers be assigned to the factories?
  2. Twelve workers are randomly assigned to work in four factories with 3 workers assigned to each factory as manager, assistant manager and assembly line worker. How many ways can the workers be assigned to the factories?

\text{ }

\text{ }

\text{ }

\text{ }

\text{ }

\text{ }

\text{ }

\text{ }

Answers

Exercise 1

    34,650

Exercise 2

    990

Exercise 3

  1. 17,576,000
  2. 35,152,000
  3. 70,304,000
  4. 351,520,000

Exercise 4

  1. 1024
  2. 64
  3. 100
  4. 252
  5. 638

Exercise 5

  1. 369,600
  2. 479,001,600

___________________________________________________________________________
\copyright \ 2017 \text{ by Dan Ma}

The multiplication principle

This is the first post in a series of posts on combinatorial analysis, which is a study on counting, e.g. finding effective methods for counting the number of ways to arrange objects and for counting the number of ways events can occur. Many problems in mathematics, e.g. in probability theory, require the counting of the number of ways an event can occur. Permutations and combinations are basic ideas in counting. Both ideas are based on an idea called the multiplication principle. This post focuses on the multiplication principle. The next post is on permutations and combinations.

___________________________________________________________________________

The Multiplication Principle

Let’s start with some examples.

Example 1
Jack is buying ice cream – two scoops of ice cream of the same flavor on either a cone or a cup. The available flavors are vanilla, strawberry, chocolate, butter pecan, mint chocolate chips and rocky road. How many possible arrangements of ice cream are available for Jack?

The answer is 2 x 6 = 12 arrangements. The following lists out all the possibilities.

    Cone-Vanilla
    Cone-Strawberry
    Cone-Chocolate
    Cone-Butter Pecan
    Cone-Mint Chocolate Chips
    Cone-Rocky Road

    Cup-Vanilla
    Cup-Strawberry
    Cup-Chocolate
    Cup-Butter Pecan
    Cup-Mint Chocolate Chips
    Cup-Rocky Road

With the cone option, there are 6 choices for flavors. With the cup option, there are also 6 choices for flavors. So the total number of possible choices would be 6 + 6 = 2 x 6 = 12.

Of course, if Jack only limits his purchase to his most favorite ice cream (say mint chocolate chips), then there would be only two choices, cone or cup. But if he is indifferent to the flavors, there would be 12 arrangements as listed above. \square

The listing in Example 1 suggests a general principle.

    The Multiplication Principle
    If one event can occur in M ways and another event can occur independent of the first event in N ways, then the two events successively can occur in M \times N ways.

Of course, the principle just described has an obvious generalization. Instead of just considering two events, we can count how many ways a series of events can occur. As long as the events are independent, the total number of ways all events occurring in a row is obtained by simply multiplying the numbers of possible occurrences of the individual events.

Example 2
The following is a menu for a banquet. Assuming that each guest can only pick one item for each course, how many different possible dinner decisions are possible?

    menu-for-counting

This problem is mentioned in previous post. Using the multiplication principle, the answer is 4 x 2 x 3 x 2 = 48 choices. \square

Remark. For the multiplication principle to apply, the events must be independent. This means that the number of occurrences of the one event is not restricted by the previous event. If a guest in the banquet is indifferent to all items in the menu (if taken individually) but does not like to mix certain items (e.g. he does not like to have French onion soup to go with goat cheese salad), then he or she might not have 48 choices. The following example illustrates that for a certain kind of dependency between events, the multiplication principle can still apply.

Example 3
A PIN code for a bank ATM machine is a 4-digit number. Determine the following:

  • How many PIN codes are possible?
  • How many PIN codes are possible if no repetition of digits is allowed?
  • How many PIN codes are possible if the first two digits form a number less than or equal to 33 and the last two digits form a number that is at least 3 times the first two digits?

There are 10 \times 10 \times 10 \times 10= 10^4=10,000 many possible codes. If no digits can repeat, then there are

    10 \times 9 \times 8 \times 7=5040

different codes. For the no repetition case, there are 10 choices for the first digit. Once the first digit is fixed, there are only 9 choices for the second digit. Once the first two digits are fixed, there are 8 choices for the third digit. Then the fourth digit has 7 choices once the first three digits are fixed. Though the number of choices for each digit is limited by the previous digit, it is the same regardless the value of the previous digits. Thus the principle still applies.

The third problem illustrates a dependency that makes the multiplication principle not applicable. For example, if the first two digits are 01, the last two digits would be 03, 04, 05, 06, …, 99 for a total of 97 possibilities. If the first two digits are 02, then there are 94 possibilities for the last two digits (06, 07, …, 99). As the number for the first two digits increases to 33, the number of choices for the last two digits would decrease. The largest possibility for the first two digits would be 33 and there is only one choice for the last two digits (99). Thus the multiplication principle would not work here. The answer is not obtained by multiplication. It is obtained by adding up all the possibilities as the first two digits go from 01 to 33. We perform the calculation in Excel and the answer is 1617. Of course, the third problem is not a realistic way to obtain pass codes. It is just an illustration of a case where the multiplication principle does not apply. \square

We look at more examples.

Example 4
In how many ways can 9 students (4 males and 5 females) stand in a row for a group photo

  • if there is no restriction on the standing positions?
  • if the females must stand together and the males must stand together?
  • if the females must stand together?

If the students can stand anywhere they want, there are 9 x 8 x 7 x 6 x 5 x 4 x 3 x 2 x 1 = 362,880 ways of arranging these 9 students in a row. For the first spot, there are 9 students to choose from and there are 8 students to choose from for the second spot and so on. A convenient notation for the answer is 9! (read 9 factorial). It simply means that it is the product of all the integers from 9 down to 1.

By the same reasoning, there are 5! = 120 ways of arranging 5 persons in a row. There are 4! = 24 ways of arranging 4 people standing in a row. If the females stand on the left and the males stand on the right, then there are 120 x 24 = 2,880 arrangements in a row for the 9 students. The answer is 2 x 120 x 24 = 5,760 since there is also the case of males standing on the left and females on the right.

The third problem requires that only the females must stand together. The use of the multiplication principle is subtle. If the females must stand together, think of all the females as one unit. There are 5 units – 4 males and one combined unit of females. There are 5! ways of arranging the 5 units. Within the combined unit of females, there are 5! ways of arranging the 5 females. Thus the answer is 5! x 5! = 120 x 120 = 14,400. \square

The next example shows that the multiplication principle is invaluable in calculating large numbers.

Example 5
In general, longer passwords are stronger passwords. The principle of counting discussed here can be used to quantify the strength. Consider an 8-character password consisting of letters from English. To start, let’s say the letters in the password are all upper case or all lower case, i.e. the password is case insensitive. Then the number of possible passwords would be

    \displaystyle 26 \times 26 \times 26 \times 26 \times 26 \times 26 \times 26 \times 26=26^8=\text{208,827,064,576}

That’s over 208.8 billions possibilities. What if the password is case sensitive? If each character in the password can be in upper case or in lower case, then each character has 52 choices (26 upper case letters and 26 lower case letters). Then the number of possible passwords would be

    \displaystyle 52 \times 52 \times 52 \times 52 \times 52 \times 52 \times 52 \times 52=52^8=\text{53,459,728,531,456}

That’s over 53 trillions possible passwords! Of course, the length of the password by itself may not mean strength. For example, people may use familiar words to form the password. One of the famous examples would have to be “Password”. In addition to being long enough, the characters in the passwords should be randomly chosen if possible.

To make the 8-character password even stronger, we can include numbers in addition to letters. Then the total number of possibilities is

    \displaystyle 62^8=\text{218,340,105,584,896}

which is over 218 trillions. \square

___________________________________________________________________________

Factorial

The factorial concept introduced in Example 4 is a useful tool in counting. If n is a positive integer, the number n! is defined to be the product of all the positive integers less than or equal to n.

    n!=n \times (n-1) \times (n-2) \times \cdots \times 3 \times 2 \times 1

For convenience, we define 0! = 1. In Example 4, 9! is the number of ways of ordering 9 people in a row. In general, n! is the number of ordered arrangements of n distinct objects. There will be more discussion about this interpretation of n factorial in the next post.

___________________________________________________________________________

Exercises

Exercise 1
Example 3 and Example 5 concern with finding the number of possible ordered arrangements of objects for use in setting passwords or pass codes. Here’s is an example involving license plates. When a motor vehicle authority (or some relevant governmental body) designs license plates for automobiles, one important goal is to have a sufficiently large number of possibilities to accommodate all the vehicles in that jurisdiction. Suppose that the license plate has 7 characters. The first character is a number and the next three characters are letters. The last three characters are numbers.

  1. What is the total number of possibilities in this license plate design?
  2. If the motor vehicle authority is beginning to issue license plates with the first digit being 3, how many license plates will the authority have to issue before switching to using 4 as the first digit?

Exercise 2

  1. Twelve tasks are to be assigned to twelve workers. What is the total different number of job assignments?
  2. In how many ways can four students be randomly selected to receive four cash prizes ($10, $20, $30, $40)?

Exercise 3

  1. In how many ways can 4 boys and 4 girls stand together in a row for a group photo?
  2. In how many ways can 4 boys and 4 girls stand together in a row for a group photo if people of the same gender must be together?
  3. In how many ways can 4 boys and 4 girls stand together in a row for a group photo if only the girls must be together?
  4. In how many ways can 4 boys and 4 girls stand together in a row for a group photo if no people of the same gender can stand together?

\text{ }

\text{ }

\text{ }

\text{ }

\text{ }

\text{ }

Answers

Exercise 1

  • 175,760,000
  • 17,576,000

Exercise 2

  • 479,001,600
  • 11,880

Exercise 3

  • 40,320
  • 1,152
  • 2,880
  • 1,152

___________________________________________________________________________
\copyright \ 2017 \text{ by Dan Ma}

A banquet menu can be a lesson in counting

    menu-for-counting

The above is a menu found in the Internet. Though we do not know how the dishes taste, the menu seems enticing. Whatever the eventual quality of the food, it is a well crafted menu. There are choices for each course. Assuming you can only pick one choice for each course, how many possible choices does a guest have for a complete meal in this banquet?

The above question is a counting problem, which in this case is easy to answer. The answer is derived by multiplying a few numbers. However, the significance of the problem is broader than the procedure to get the answer. The fundamental ideas behind this and other “easy” examples form a foundation of the mathematical subject called combinatorics. In order to help build a firm foundation, we devote several posts introducing the subject of counting, starting from the fundamental notions.

___________________________________________________________________________
\copyright \ 2017 \text{ by Dan Ma}

Celebrate the year of the rooster

year-of-rooster

According to the Chinese zodiac, the year 2017 is the year of the rooster. In fact, today (January 28, 2017) is the start lunar New Year, which is the first day of the year of the rooster. What better way to celebrate the year of the rooster than working a related math puzzle, or to perform a related random experiment!

At the office yesterday, conversation at the beginning of a meeting, before discussing the main topics, centered on the Chinese zodiac animal signs (the Chinese zodiac system is a 12-year cycle with a different animal representing each year). One coworker mentioned he is a tiger. Another coworker did not know his sign and Googled to find out that he is a rat! Another coworker is a rooster. It turns out that a pig is also represented. Imagine that you keep picking people at random and ascertain his/her animal sign. How many people do you have to ask in order to have met all 12 animal signs?

The number of people you have to ask is a random quantity. If you are very lucky, you may encounter all 12 signs by asking just 12 people (the minimum number of people you have to ask). The outcome of twelve people is not likely (it has only about 0.005% chance of happening). It is almost certain that you have to ask many more people. It turns out that the first few animals are easy to obtain (just like in the meeting with 4 different animal signs for 4 people). As you ask more people, the animal signs tend to repeat. So you have to keep asking more people. As you encounter more animal signs, it takes longer to get a new animal sign. For example, if you have encountered 11 animal signs already, you will have to ask on average 12 more people to find the last animal sign. So the overall average is not a number near 12. On average, you will have to ask about 37 people to get the entire set of 12 animals.

The random experiment that has been described is this. Put 12 slips of papers numbered 1 through 12 in a hat. Randomly draw a piece of paper and note the number and then put it back into the hat. Keep drawing until all 12 numbers have been chosen. Let X be the number of selections that are required to perform this random experiment. Of course, you can expand the sample space to include more slips of papers (i.e. with more numbers). But the context will not be picking animal signs.

There are two ways to get a handle on the random variable X as described above. One is through simulation and the other is through math.

Before discussing the simulation or the math, let’s point out that the problem discussed here is a classic problem in probability that goes by the name “the coupon collector problem”. The numbers 1 to 12 in a hat are like coupon (prizes) that are randomly given out when purchasing a product (e.g. a box of cereal). The problem discussed here is that the coupon collector desires to collect the entire set of coupons.
___________________________________________________________________________

Simulation

To get a sense of how long it will take, simulate random numbers from 1 through 12 until all numbers have appeared. The following is 5 iterations of the experiment.

    9, 2, 6, 8, 9, 10, 2, 9, 8, 1, 5, 11, 1, 1, 2, 10, 9, 8, 8, 9, 5, 11, 9, 3, 7, 9, 8, 8, 4, 3, 1, 4, 3, 12 (34 draws)

    7, 7, 1, 10, 11, 11, 10, 4, 5, 8, 8, 2, 6, 4, 6, 2, 12, 6, 6, 12, 9, 5, 8, 10, 1, 5, 10, 4, 9, 4, 1, 11, 11, 6, 2, 1, 6, 6, 3 (39 draws)

    9, 5, 2, 2, 1, 5, 6, 11, 7, 11, 4, 6, 1, 12, 3, 7, 8, 3, 3, 2, 2, 3, 5, 6, 2, 5, 1, 6, 8, 5, 4, 10 (32 draws)

    1, 5, 5, 4, 5, 12, 10, 1, 8, 1, 3, 9, 1, 3, 11, 9, 10, 3, 9, 11, 4, 4, 4, 7, 7, 3, 1, 11, 11, 4, 10, 6, 3, 2 (34 draws)

    6, 7, 6, 1, 12, 6, 1, 1, 7, 1, 11, 10, 3, 3, 9, 6, 9, 4, 2, 6, 11, 7, 7, 11, 2, 6, 2, 1, 7, 2, 5, 9, 6, 12, 6, 11, 1, 11, 11, 2, 5, 6, 7, 5, 2, 11, 2, 2, 6, 2, 12, 5, 5, 5, 12, 10, 3, 11, 1, 10, 10, 6, 9, 11, 10, 7, 11, 5, 1, 9, 11, 9, 8 (73 draws)

Each of the number is generated by using the =RANDBETWEEN(1, 12) in Excel. In each iteration, the numbers are generated until all 12 numbers have been generated.

There is considerable fluctuation in this 5 iterations of the experiment. With the 5th one being exceptionally long, it is possible that it takes a long time to find all 12 animal signs. The average of the first iteration is obviously 34. The average of the first two iteration is 36.5. The averages of the first 3, 4, and 5 iterations are 35, 34.75, and 42.4, respectively.The last average of 42.4 is quite different from the average of 37 indicated earlier.

What if we continue to run the experiment, say, for 10,000 times? What would the long run averages look like? The following graph shows the averages from first 100 runs of the experiment. It plots the average of the n iterations from n=1 to n=100.

    Figure 1 – Long run averages from 100 runs
    animal-signs-simulations-100b

Figure 1 shows that there is quite a bit of fluctuation in the averages in the first 25 runs or so. Eventually, the averages settle around 37 but still retain noticeable fluctuation. The following graph shows the averages from first 1000 runs of the experiment.

    Figure 2 – Long run averages from 1000 runs
    animal-signs-simulations-1000

The graph is Figure is smoother as it moves toward 1000, but still has noticeable fluctuation from 37 (in fact the graph is mostly below 37). The following graph shows the averages from first 10000 runs of the experiment.

    Figure 3 – Long run averages from 10000 runs
    animal-signs-simulations-10000

The graph in Figure 3 shows the average of the first n iterations with n goes from 1 to 10,000. The graph is for the most parts a horizontal line slightly above 37, especially after n=3000. In fact the average of all 10,000 iterations is 37.3381, which is close to the theoretical average of 37.2385.

The simulation is an illustration of the law of larger numbers. The essence of the law of large numbers is that the short run results of a random experiment are unpredictable while the long run results are stable and predictable and eventually settle around the theoretical average.

The first 5 runs of the experiment (as shown above) are certainly unpredictable. It may take 34 draws or may take 73 draws. The first 100 simulations also have plenty of ups and downs, even though graph in Figure 1 shows a movement toward 37. The first 1000 simulations display more stable results but are below average as the graph move toward 1000 (Figure 2). In simulating the experiment 10,000 times (Figure 3), the long run averages settle around the theoretical average of 37.2385.

So if you survey people their animal signs, the time it takes has a great deal of random fluctuations. It may take 34 asks or 73 asks (as shown in the first 5 simulations). If the experiment is done repeatedly, the results are predictable, i.e. the average is around 37.

The long run results of a gambling game are predictable too and will settle around the theoretical average. The theoretical average of a gambling game is usually referred to as the house edge. For example, for the game of roulette, the house edge is 5.26%. For each bet of $1, the gambler is expected to lose 5.26 cents. In playing a few games, the gambler may win big. In the long run, the house is expected to gain 5.26 cents per one dollar bet. Thus the law of large numbers can mean financial ruin for the gambler (or profits for the casino). For an illustration of the law of large numbers in the context of the game of Chuck-a-Luck, see here. For an illustration in the context of the roulette wheel, see here.

Another piece of useful information from the 10,000 simulated runs of the experiment is the frequency distribution.

Table 1
Frequency Distribution of the 10,000 Simulated Runs
\begin{array}{rrrrr}      \text{Interval} & \text{ } & \text{Frequency} & \text{ } & \text{Relative Frequency} \\      \text{ } & \text{ } \\      \text{10 to 19} & \text{ } & 375 & \text{ } & 3.75 \% \\      \text{20 to 29}  & \text{ } & 2817 & \text{ } & 28.17 \% \\      \text{30 to 39}  & \text{ } & 3267 & \text{ } &  \text{ } 32.67 \% \\      \text{40 to 49}  & \text{ } & 1931 & \text{ } & \text{ } 19.31 \% \\  \text{50 to 59}  & \text{ } & 901 & \text{ } & \text{ } 9.01 \% \\  \text{60 to 69}  & \text{ } & 379 & \text{ } & \text{ } 3.79 \% \\  \text{70 to 79}  & \text{ } & 190 & \text{ } & \text{ } 1.90 \% \\  \text{80 to 89}  & \text{ } & 88 & \text{ } & \text{ } 0.88 \% \\  \text{90 to 99}  & \text{ } & 30 & \text{ } & \text{ } 0.30 \% \\  \text{100 to 109}  & \text{ } & 13 & \text{ } & \text{ } 0.13 \% \\  \text{110 to 119}  & \text{ } & 3 & \text{ } & \text{ } 0.03 \% \\  \text{120 to 129}  & \text{ } & 3 & \text{ } & \text{ } 0.03 \% \\  \text{130 to 139}  & \text{ } & 1 & \text{ } & \text{ } 0.01 \% \\  \text{140 to 149}  & \text{ } & 2 & \text{ } & \text{ } 0.01 \% \\      \text{ }  & \text{ } & \text{ } & \text{ } & \text{ } \\      \text{Total }  & \text{ } & 10000 & \text{ } & 100.00 \%    \end{array}

Figures 1 to 3 tell us the long run behavior of the simulations (e.g. the long run average is 37). Table 1 gives the counts of the simulations that fall into each interval and the corresponding relative frequency (the percentage). Table 1 tells us how often or how likely a given possibility occurs. The total number of simulations that fall within the range 20 to 49 is 8015. So about 80% of the time, the experiment ends in 20 to 49 draws. Furthermore, 92.95% of the simulations fall into the interval 20 to 69. This really tells us what the likely results would be if we perform the experiment. The frequency distribution also tells us what is unlikely. There is only 3.75% chance that the experiment can be completed with less than 20 draws. In the simulations, there are two that are above 140 (they are 141 and 142). These extreme results can happen but are extremely rare. They only happened about 2 times per 10,000 simulations.

___________________________________________________________________________

The math angle

There is also a mathematical description of the random experiment of surveying people until all 12 animal signs are obtained. For example, there is a formula for calculating mean, and there is also a formula for calculating the variance. There is also a probability function, i.e. a formula for calculating probabilities (akin to Table 1). The formula for the mean is actually simple to describe.

Let X_n be the number of draws from the set \left\{1,2,3,\cdots,n \right\} with replacement such that each number in the set is picked at least once. The expectation of X_n is the following.

    \displaystyle E[X_n]=n \biggl[ \frac{1}{n}+\frac{1}{n-1}+ \cdots+ \frac{1}{3}+\frac{1}{2}+1    \biggr]

The 37.2385 theoretical average discussed above comes from this formula. The the case of n=12, the mean would be

    \displaystyle E[X_{12}]=12 \biggl[ \frac{1}{12}+\frac{1}{11}+ \cdots+ \frac{1}{3}+\frac{1}{2}+1    \biggr]=37.23852814

For more details of the math discussion, see this previous post. For a more in-depth discussion including the probability function, see this post in a companion blog on probability.

___________________________________________________________________________
\copyright \ 2017 \text{ by Dan Ma}

The coupon collector problem

The coupon collector problem is a classic problem in probability. It is usually described in this way: a person (called the coupon collector) is trying to collect all coupons (promotional gift items) when purchasing a certain type of products (e.g. boxes of breakfast cereal). Suppose that there are n different types of coupons. Suppose that when someone buys a unit of the product, he or she does not know what type of coupons that is inside the box. How many units of the product does the coupon collector have to buy in order to collect the entire set of coupons?

Note that the number of units to buy is a random quantity. To see it for yourself, roll a fair die until each of the six faces appears at least once. You can try this with your own die and record the numbers as you go. We rolled our own die and the following shows three such experiments.

    3, 4, 6, 2, 6, 1, 3, 2, 1, 4, 6, 2, 6, 1, 6, 1, 2, 3, 5

    6, 6, 4, 1, 3, 6, 6, 4, 1, 5, 4, 2

    6, 2, 1, 1, 5, 5, 1, 1, 1, 3, 6, 1, 3, 5, 4

The first experiment has 19 trials, the second one has 12 trials and the third one has 15 trials. If we run the random experiment another time, we would get another sequence. Also note that the first 5 of the faces (coupons) show up fairly quickly. It takes a few more trials just to get the last coupon.

Rolling a dice is simply a way to generate random numbers. Thus we can describe the coupon collector problem as a random sampling problem. Draw numbers repeatedly from the sample space S=\left\{1,2,3,\cdots n \right\} with replacement until each of the n numbers has been chosen at least once. Let X_n be the number of trials that are required to perform this experiment. As demonstrated above (or as confirmed by your rolling of a die), X_n is a random variable. The smallest value it can take on is n. Theoretically, the random variable X_n could be n or some other integer greater than n. The goal is to describe the probability distribution of the random variable X_n.

Some interesting things to know about this probability distribution are the expected value E[X_n] (the mean or the average number of trials that are needed to complete the experiment). Another one is the probability function (or probability mass function) so that we can determine how likely it is to complete the experiment in k trials. For a more detailed discussion, go to this previous post. In the remainder of the post, we discuss the expected value.

The expected value E[X_n] has a clear formula and is easily computed.

    \displaystyle E[X_n]=n \biggl[ 1 + \frac{1}{2}+\frac{1}{3}+ \cdots+\frac{1}{n-1}+\frac{1}{n} \biggr]

For the 6-coupon case (or the rolling of a die), it would take on average 15 trials to get all 6 faces.

    \displaystyle E[X_6]=6 \biggl[ 1 + \frac{1}{2}+\frac{1}{3}+\frac{1}{4}+\frac{1}{5}+\frac{1}{6} \biggr]=14.7 \approx 15

Of course, 15 is simply the average. The actual results may be more or less than 15 as the above illustrations show (or your own rolling of a die). To find out how likely it is to have a result around 15, it is necessary to have the probability function of X_6. See the previous post for more information on that.

Consider this example. Suppose that you want to throw candies at random to a group of 10 kids. Each candy is assigned to each child at random. You want to throw candies until each of the children has at least one candy. Of course some kids would receive a few candies. But the goal is that no one would come away empty handed. On average, you would need to throw about 30 candies.

    \displaystyle E[X_{10}]=10 \biggl[ 1 + \frac{1}{2}+\frac{1}{3}+ \cdots+\frac{1}{9}+\frac{1}{10} \biggr]=29.29 \approx 30

If there are 50 coupons, it would take on average 225 trials to get all 50 coupons. One interesting thing is that it is relatively easy to get the first few coupons. To get the last coupon, it would take a quite a few trials. In fact, it would take on average n trials to get the last coupon (if n is the number of coupon types).

Another point we would like to make is that the coupon collector problem requires an understanding of the geometric distribution. A random experiment that can modeled by a geometric distribution is the performance of a series of independent trials until some criterion is satisfied. The random variable X_n described here has some similarity with a geometric model (but not totally similar). The similarity: we keep drawing numbers with replacement until all numbers are drawn.

Let’s look at the third experiment 6, 2, 1, 1, 5, 5, 1, 1, 1, 3, 6, 1, 3, 5, 4 listed above. We draw the first number (happens to be 6). Next we wish to draw another number different from 6. Luckily it takes only one trail to get 2. Now we wish to draw a number different from 6 and 2. It also takes one trial (to get 1). Next, continue to draw until we get a number different from 6, 2 and 1. This time it takes two trials to get a 5. Then continue to draw until we get a number different from 6, 2, 1, and 5. It takes 5 trials to get a 3. Lastly it takes 5 more trials to get the last number 4. What this shows is that the random variable X_6 is a sum of 6 geometric random variables.

    X_6=C_1+C_2+C_3+C_4+C_5+C_6

Each of the C_i is a geometric random variable. The first one C_1 is kind of special. The value of C_1 is always 1 (it takes one trial to get the first number). The second variable C_2 is a geometric random variable (the number of trials to get a number different from the first chosen number. Then C_3 is the number of trials to get a number different from the first two chosen numbers and so on. If there are n coupons, then X_n is the sum of n geometric random variables.

    X_n=C_1+C_2+C_3+\cdots +C_{n-1}+C_n

The formula for the expected value E[X_n] is derived through this connection with the geometric distribution. See the previous post for a more in-depth discussion.

___________________________________________________________________________
\copyright \ 2017 \text{ by Dan Ma}