Statistics · AP Statistics
Chi-square tests
counts against expectations
A chi-square test asks whether counts in categories are too far from what a hypothesis predicts to blame on chance.
A candy maker says its bags are 24% blue, 20% orange, 16% green, 14% yellow, 13% red, 13% brown. A bag of 200 is counted (black bars). If the claim is true, the bag should hold about 200 × 0.24 = 48 blue, and so on (blue ghosts).
No bag matches exactly. The red rods are the gaps, observed − expected. Each gap is squared and divided by its expected count, and the pieces stack into one number, the chi-square statistic, χ² = 12.11.
Small χ² means the counts sit close to the claim; large χ² means they don’t. This chapter asks how large is too large: for one variable against a claimed distribution (goodness of fit), and for two-way tables (homogeneity and independence).
what chance alone would produce
Random samples vary. To judge a result, compare it with the results chance alone produces when the claim is true.
The figure fills 300 bags of 200 at random from exactly the claimed mix. Each simulated bag (left) misses the expected counts a little, and its χ² lands as one dot in the dot plot.
Even with the claim true, χ² is never 0, and it is usually a few units. These 300 values average 4.99. Only 10 of the 300 (red) reach the real bag’s 12.11: a proportion of 0.033.
That is the logic of every test: if the claim were true, how often would we see a result at least this extreme? If rarely, the result is statistically significant evidence against the claim. If often, the data give no reason to doubt it, which is not proof that it is true.
The grey curve is the χ² distribution the dots pile up toward. It lets us find the probability without simulating (8.3).
hypotheses, expected counts, conditions
A chi-square test for goodness of fit tests whether the distribution of one categorical variable matches a claimed distribution.
Hypotheses are about the population proportions, never the sample counts:
H0: pblue = 0.24, porange = 0.2, … , pbrown = 0.13 (the claimed distribution is correct).
Ha: at least one of the proportions differs from the claim.
Ha is not “all of them differ”, and it names no direction.
Expected counts are n × pi, worked out from H0:
| colour | blue | orange | green | yellow | red | brown |
|---|
| claimed p | 0.24 | 0.2 | 0.16 | 0.14 | 0.13 | 0.13 |
|---|
| expected | 48 | 40 | 32 | 28 | 26 | 26 |
|---|
They add up to n = 200, and they need not be whole numbers: don’t round them.
Conditions. Random: a random sample (or randomized experiment). 10%: sampling without replacement, n ≤ 10% of the population (200 candies is far less than 10% of all the maker’s candies). Large counts: every expected count is at least 5; here the smallest is 26. Check the expected counts, not the observed ones.
squared gaps, scaled by what was expected
The χ² statistic adds up, over every category, the squared gap between observed and expected, divided by the expected count.
| colour | O | E | O − E | (O − E)²/E |
|---|
| blue | 34 | 48 | −14 | 4.08 |
| orange | 48 | 40 | +8 | 1.60 |
| green | 41 | 32 | +9 | 2.53 |
| yellow | 19 | 28 | −9 | 2.89 |
| red | 27 | 26 | +1 | 0.04 |
| brown | 31 | 26 | +5 | 0.96 |
| total | 200 | 200 | 0 | 12.11 |
|---|
The gaps always add up to 0, so they can’t be summed as they are: squaring makes every one count. Dividing by E makes a gap count for more when little was expected: missing by 10 out of an expected 20 matters more than missing by 10 out of 200.
The degrees of freedom are the number of categories minus one: df = 6 − 1 = 5. Once five counts are known, the sixth is fixed by the total.
The largest contribution says where the data and the claim disagree most: blue, 4.08 of the 12.11, with 34 observed against 48 expected: fewer blue candies than claimed.
how far into the tail
The p-value is the area under the χ² curve with the test’s df, at and beyond the observed χ²: large values are the evidence against H0, so the test always looks at the right tail.
The simulation in 8.1 gave 0.033, close to the same answer.
Interpretation. Assuming the maker’s claimed distribution is correct, there is a 0.033 probability of getting a χ² of 12.11 or larger by chance alone in a random sample of 200 candies.
Conclusion. Since p = 0.033 < α = 0.05, reject H0. There is convincing evidence that the colour distribution of this maker’s candies differs from the claim.
With a table instead of technology: the critical value for df = 5 and a tail of 0.05 is χ²* = 11.07 (blue). 12.11 > 11.07, so p < 0.05.
Had p been above α, the conclusion would be “not convincing evidence that the distribution differs”: never “the claim is correct”, and never “accept H0”. A χ² test also can’t say which way the distribution is off; for that, look at the largest contributions.
a family, one for each df
The χ² distributions are a family of density curves, one for each number of degrees of freedom.
Each takes only values of 0 or more, since χ² is a sum of squares. Each is skewed right. Its mean is df, and for df ≥ 2 its peak is at df − 2.
As df grows the curve moves right, spreads out and becomes less skewed (the red curve steps through df = 1 to 12). More categories mean more pieces in the sum, so a larger χ² is expected even when H0 is true.
So the same χ² can be strong evidence at one df and none at another. The upper critical values:
| df | 1 | 2 | 3 | 4 | 5 | 10 |
|---|
| tail 0.05 | 3.84 | 5.99 | 7.81 | 9.49 | 11.07 | 18.31 |
|---|
| tail 0.01 | 6.63 | 9.21 | 11.34 | 13.28 | 15.09 | 23.21 |
|---|
A rough check: a χ² near its df is ordinary. The candies’ 12.11 is more than twice its df of 5.
The χ² distribution is only an approximation to the true sampling distribution of the statistic. It is good when every expected count is at least 5: that is why the condition matters.
from the margins
For a two-way table, each expected count is the row total times the column total, divided by the table total.
A random sample of 400 adults, each classified by age group and main news source. Observed counts, with expected counts in brackets:
| online | TV | print | total |
|---|
| 18–34 | 64 (48.2) | 46 (49.6) | 24 (36.2) | 134 |
|---|
| 35–54 | 43 (47.5) | 48 (48.8) | 41 (35.6) | 132 |
|---|
| 55+ | 37 (48.2) | 54 (49.6) | 43 (36.2) | 134 |
|---|
| total | 144 | 148 | 108 | 400 |
|---|
Why it works. Overall, 144 of 400 adults (36%) get their news online. If age group made no difference, the same share of the 134 adults aged 18–34 would: 134 × 144/400 = 48.24.
The expected counts have the same margins as the observed ones. The figure works out one cell at a time: the red lid is the expected count, the blue tiles the two totals it comes from.
Keep the decimals. Expected counts are averages over many samples, not counts that could be observed.
the design decides
The same two-way table can come from two designs, and the design decides which test it is.
Homogeneity (left): separate random samples from several populations (or groups in an experiment), with one categorical variable recorded. Here, a sample from each age group, each person asked their news source. The row totals were chosen by the design.
H0: the distribution of news source is the same for all three age groups. Ha: the distribution of news source is not the same for all three.
Independence (right): one random sample from one population, with two categorical variables recorded for each individual. Only the table total was fixed; how many fall in each age group is up to chance.
H0: there is no association between age group and news source among adults (they are independent). Ha: there is an association (they are not independent).
| homogeneity | independence |
|---|
| samples | one per population or group | one |
| variables | one | two |
| question | same distribution in each? | an association? |
The arithmetic is the same: expected counts, χ², df and p-value. Conditions, for both: random samples (or random assignment); each sample at most 10% of its population; every expected count at least 5.
χ² and df = (r − 1)(c − 1)
For a two-way table, χ² adds (O − E)²/E over every cell, and df = (rows − 1)(columns − 1).
The news-source sample is one sample with two variables: a test for independence, at α = 0.05.
H0: age group and main news source are independent among adults. Ha: they are associated.
Conditions. A random sample; 400 is less than 10% of all adults; the smallest expected count is 35.6, at least 5.
Each cell’s contribution (red columns):
| online | TV | print |
|---|
| 18–34 | 5.15 | 0.26 | 4.10 |
|---|
| 35–54 | 0.43 | 0.01 | 0.81 |
|---|
| 55+ | 2.62 | 0.39 | 1.29 |
|---|
Why (r − 1)(c − 1): with the margins fixed, once a 2 × 2 corner of the table is filled in, every other cell follows from the totals.
The largest contributions: 18–34, online (5.15: 64 observed, 48.2 expected) and 18–34, print (4.10: 24 observed, 36.2 expected). Younger adults lean online and away from print.
evidence of an association, not a cause
The p-value is the area beyond χ² under the curve with df = (r − 1)(c − 1), and the conclusion is about the populations, in context.
Conclusion. Since p = 0.0046 < α = 0.05, reject H0. There is convincing evidence of an association between age group and main news source among adults.
By the critical value: χ²* = 9.49 for df = 4, and 15.06 > 9.49.
Not a cause. This is an observational study: the test finds an association, and says nothing about age causing a choice of news source. Only a randomized experiment, tested for homogeneity across its treatment groups, can show cause.
Had the same counts come from three separate samples, the conclusion would read: convincing evidence that the distribution of news source differs among the three age groups.
Common mistakes. Writing hypotheses about the sample; using percentages in place of counts; checking the observed rather than the expected counts; df = rc − 1; concluding H0 is true when p is large.
z for proportions, or χ²
For categorical data, the choice turns on how many samples there are, how many variables, and how many categories.
| the data | procedure |
|---|
| one sample, a two-category variable, a claim about p | one-sample z test (or interval) for p |
| two samples or groups, a two-category variable | two-sample z for p1 − p2 |
| one sample, one variable with several categories, against a claim | χ² goodness of fit, df = k − 1 |
| several samples or groups, one variable | χ² homogeneity, df = (r − 1)(c − 1) |
| one sample, two variables | χ² independence, df = (r − 1)(c − 1) |
The z procedures estimate: they give a confidence interval and allow a one-sided Ha (“greater than”). χ² tests only test, and their Ha has no direction.
With two groups and two categories, the two-sample z test with a two-sided Ha and the χ² test for homogeneity give the same p-value (χ² = z2).
Read the design first: how was the sample taken, and what was recorded about each individual? The figure shows the shape each design leaves in the data, and picks one for each question.
← Inference for means · Inference for slopes →