Inference for proportions · AP Statistics

Chapter 6 · Inference for proportions

intervals and tests for p

A sample proportion estimates a population proportion. Inference says how far to trust it.

In the figure, 50 random samples of 500 adults are taken from one city, where a proportion p = 0.60 support a new bike lane. The figure knows p; a real survey never does.

Each sample gives its own p̂ (a dot) and a 95% confidence interval around it (a rod). The vertical line is p. 47 of the 50 intervals reach over it; 3, in red, miss.

This chapter builds those intervals and says what “95% confident” does and does not mean. Then it turns the question round: a significance test asks whether a sample would be surprising if a claimed value of p were true, and what errors that decision can make. Last, the same two tools for the difference between two proportions.

6.1 · Why be normal?

the sampling distribution of p̂

Sample proportions vary from sample to sample. When the sample is large enough, their distribution is approximately normal, so a normal model says which results are unusual.

In the figure, random samples are drawn from a population with p = 0.1, and each sample’s p̂ drops into a histogram: samples of 20 on the left, of 200 on the right. The red curve is the normal model.

\mu_{\hat p} = p \qquad \sigma_{\hat p} = \sqrt{\frac{p(1-p)}{n}}

p̂ is centred on p: it is an unbiased estimator. Its spread shrinks as n grows: σ is 0.067 for n = 20 and 0.021 for n = 200. 10 times the sample, √10 times less spread.

Large counts. The shape is approximately normal when np ≥ 10 and n(1 − p) ≥ 10. With n = 20, np = 2: the histogram is skewed right and piled against 0, and the normal curve puts 6.8% of its area below 0, where p̂ cannot go. With n = 200, np = 20 and n(1 − p) = 180: the curve fits.

The 10% condition. The formula for σ assumes independent observations. Sampling without replacement, that holds closely enough when the sample is at most 10% of the population.

Example. With n = 200, how likely is p̂ ≥ 0.15? z = (0.15 − 0.1)/0.0212 = 2.36, and P(Z ≥ 2.36) = 0.0092. A sample that far above 0.1 would be very unusual by chance alone.

6.2 · Constructing a confidence interval

statistic ± (critical value)(standard error)

A confidence interval gives a range of plausible values for p: the sample’s p̂, plus and minus a margin of error.

A random sample of 500 adults in a city: 290 support a new bike lane, so p̂ = 290/500 = 0.580. A one-sample z interval for p:

\begin{aligned}\hat p &\pm z^*\sqrt{\frac{\hat p(1-\hat p)}{n}} \\ &= 0.580 \pm 1.960\sqrt{\frac{(0.58)(0.42)}{500}} \\ &= 0.580 \pm 0.043 = (0.537,\ 0.623)\end{aligned}

z* = 1.960 for 95%: the middle 95% of a standard normal lies between −1.960 and 1.960. The standard error uses p̂ in place of the unknown p. The margin of error is z* × SE = 0.043.

condition	check
Random	a random sample of adults
10%	500 ≤ 10% of the city’s 820,000 adults
Large counts	np̂ = 290 and n(1 − p̂) = 210, both ≥ 10

Why it works. In 95% of samples p̂ lands within 1.96σ of p (the blue band). Whenever it does, the interval reaching 1.96 SE either side of p̂ reaches back over p. In the figure, sample 6 lands outside the band, and its interval misses.

On the exam, name the procedure, check the conditions, do the arithmetic, and state the interval in context.

6.2 · Margin of error and sample size

choosing n before sampling

The margin of error shrinks as n grows, but slowly: like 1/√n. To plan a survey, choose the margin first, then solve for n.

\begin{aligned}&z^*\sqrt{\frac{p^*(1-p^*)}{n}} \le \text{ME} \\ &n \ge \Big(\frac{z^*}{\text{ME}}\Big)^2 p^*(1-p^*)\end{aligned}

p* is a guess at p. With none, use p* = 0.5: it makes p*(1 − p*) = 0.25, its largest value, so the n found is enough whatever p turns out to be.

Example. A margin of at most 0.03 at 95% confidence:

n \ge \Big(\frac{1.960}{0.03}\Big)^2(0.5)(0.5) = 1067.1

so n = 1068. Always round up: 1067 would give a margin a little over 0.03. With a pilot study’s estimate p* = 0.2, 683 are enough (the blue curve).

Halving the margin takes four times the sample: n is under a square root.

The margin of error covers only the chance variation of random sampling. It says nothing about bias from undercoverage, nonresponse or a leading question.

6.3 · Interpreting a confidence interval

what 95% confident means, and what it doesn’t

The interval is a claim about p. The confidence level is a claim about the method that made it.

The interval. “We are 95% confident that the interval from 0.537 to 0.623 captures the true proportion of all adults in the city who support the bike lane.” Name the parameter, in context.

The confidence level. “If we took many random samples of 500 and built a 95% interval from each, about 95% of the intervals would capture the true proportion.” In the figure, 47 of 50 do; the 3 red ones miss, and nothing about them warned us.

Not: “there is a 95% probability that p is between 0.537 and 0.623.” p is fixed; this interval either captures it or it doesn’t. Nor is it about 95% of adults, or 95% of sample proportions.

Justifying a claim. Every value inside the interval is a plausible value of p.

claim	from (0.537, 0.623)
a majority support it, p > 0.5	convincing: every plausible value is above 0.5
p = 0.60	plausible: it is inside the interval
p ≥ 0.65	not plausible: the interval is below it

Plausible is not proven: the interval does not show that p = 0.60, only that 0.60 can’t be ruled out.

6.3 · Confidence level and width

more confidence costs width; more data buys it back

A higher confidence level needs a larger z*, and so a wider interval. A larger sample shrinks the standard error, and so the interval.

level	z*	margin of error	interval
90%	1.645	0.036	(0.544, 0.616)
95%	1.960	0.043	(0.537, 0.623)
99%	2.576	0.057	(0.523, 0.637)
95%, n = 2,000	1.960	0.022	(0.558, 0.602)

All from the same p̂ = 0.58. In the figure the red interval sweeps through levels from 80.0% to 99.5%; the blue one has four times the sample, and half the margin of error.

A 99% interval catches p more often, but says less. Wanting both high confidence and a narrow interval means a larger sample.

The confidence level is chosen before the data; it is not the probability that one particular interval is right. Neither level nor n changes where the interval is centred: p̂.

6.4 · Setting up a test

hypotheses and conditions, with p₀

A significance test asks whether a sample gives convincing evidence against a claim about p.

A phone maker says 25% of its phones need a repair within two years. A consumer group suspects more. In a random sample of 400 of the maker’s phones, 118 needed a repair: p̂ = 0.295.

H_0\!: p = 0.25 \qquad H_a\!: p > 0.25

where p is the proportion of all the maker’s phones that need a repair within two years. The null hypothesis H₀ is the claim, “no difference from 0.25”; the alternative H_a is what the group hopes to find evidence for. Hypotheses are about the parameter p, never about p̂, and are set before the data are seen.

H_a is one-sided (> or <) when the question has a direction, two-sided (≠) when any difference matters.

condition	check
Random	a random sample of the maker’s phones
10%	400 ≤ 10% of all its phones
Large counts	np₀ = 100 and n(1 − p₀) = 300, both ≥ 10

Large counts uses p₀, not p̂: the test works out what would happen if H₀ were true. The figure draws 200 samples of 400 from p = 0.25; they pile up around the normal curve with centre 0.25 and standard deviation 0.0217.

z = \frac{\hat p - p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}} = \frac{0.295 - 0.25}{0.0217} = 2.08

6.5 · Interpreting p-values

how surprising, if H₀ were true

The p-value is the probability, assuming H₀ is true, of a statistic at least as extreme as the one observed, in the direction of H_a.

\text{p-value} = P(Z \ge 2.08) = 0.019

In the figure, 5 of the 200 samples drawn with p = 0.25 reached 0.295 or more: a fraction 0.025, close to the normal model’s 0.019.

In context: “Assuming 25% of all the maker’s phones need a repair, there is a 0.019 probability of getting a sample proportion of 0.295 or higher in a random sample of 400 phones, by chance alone.”

A small p-value means the data would be rare if H₀ were true: evidence against H₀. A large one means the data are the kind H₀ often produces: no evidence against it.

Not: the probability that H₀ is true, nor the probability that the result happened by chance. The p-value is computed assuming H₀.

For a two-sided H_a, count both tails: the p-value is twice the area beyond |z|.

6.6 · Concluding a test

compare the p-value with α

Before the data, choose a significance level α. If the p-value is less than α, reject H₀; if not, fail to reject H₀.

At α = 0.05: 0.019 < 0.05, so reject H₀. “Because the p-value of 0.019 is less than α = 0.05, we reject H₀. We have convincing evidence that more than 25% of all the maker’s phones need a repair within two years.”

At α = 0.01: 0.019 > 0.01, so fail to reject H₀. “We do not have convincing evidence that more than 25% need a repair.”

Never “accept H₀”: failing to find evidence against a claim does not show it is true.

The same decision with the test statistic: reject when z is beyond the critical value, z* = 1.645 for α = 0.05 one-sided, 2.326 for 0.01. In the figure α sweeps from 0.005 to 0.1; the blue region is beyond z*, and the decision flips as it passes z = 2.08.

A result with p-value below α is called statistically significant at that level. Significant is not the same as large or important.

6.7 · Errors and power

Type I, Type II, and the power to detect

A test can be wrong two ways. Its power is the chance it gets a false H₀ right.

	H₀ true	H₀ false
reject H₀	Type I error, probability α	correct: power
fail to reject	correct	Type II error, probability β

In context: a Type I error would be concluding that more than 25% of phones need repair when really 25% do; the maker is wrongly accused. A Type II error would be missing a real excess; owners go on paying for repairs.

Suppose the truth is p = 0.3. At α = 0.05 the test rejects when p̂ > 0.2856. With n = 400:

\begin{aligned}\text{power} &= P(\hat p > 0.2856 \mid p = 0.30) \\ &= 0.735 \\ \beta &= 1 - \text{power} = 0.265\end{aligned}

n	100	200	400	800
power	0.322	0.496	0.735	0.937

Power rises with a larger sample (the curves narrow, as in the figure), a larger α (at α = 0.1 and n = 400, 0.834), and a true p farther from p₀. Lowering α cuts Type I errors but raises β: the two trade off at a fixed n.

6.8 · An interval for a difference

p̂₁ − p̂₂, from two independent samples

To compare two proportions, estimate their difference: (p̂₁ − p̂₂) ± z* × SE.

Independent random samples of 400 teens and 500 adults: 248 teens and 255 adults say they get their news mainly from social media. p̂₁ = 0.62, p̂₂ = 0.51, a difference of 0.11.

\begin{aligned}\text{SE} &= \sqrt{\frac{\hat p_1(1-\hat p_1)}{n_1} + \frac{\hat p_2(1-\hat p_2)}{n_2}} \\ &= \sqrt{0.000589 + 0.000500} \\ &= 0.0330\end{aligned}

\begin{aligned}(\hat p_1 - \hat p_2) &\pm z^*\,\text{SE} \\ &= 0.11 \pm 1.960(0.0330) \\ &= 0.11 \pm 0.065 \\ &= (0.045,\ 0.175)\end{aligned}

The variances add, not the standard errors: the difference of two independent estimates varies more than either.

condition	check
Random	two independent random samples (or two groups from a randomized experiment)
10%	each sample ≤ 10% of its population
Large counts	248, 152, 255, 245: all four ≥ 10

Say which is group 1. The order sets the sign: adults minus teens would give (-0.175, -0.045).

6.9 · Justifying a claim about a difference

is 0 a plausible value of p₁ − p₂?

If 0 is outside the interval, the data give convincing evidence that the two proportions differ. If 0 is inside, they do not.

In context: “We are 95% confident that the interval from 0.045 to 0.175 captures the true difference (teens minus adults) in the proportions who get their news mainly from social media.”

Every value is above 0, so there is convincing evidence that a larger proportion of teens than of adults do: by somewhere between 4.5 and 17.5 percentage points.

A smaller study. The same p̂s from 100 teens and 100 adults give (-0.027, 0.247) (blue). 0 is inside: a difference of 0 is plausible, so these data do not give convincing evidence of a difference.

That is not evidence that there is no difference: the interval also allows a difference as large as 0.25. A larger sample narrows it.

An interval that is entirely negative would be evidence that p₁ < p₂. An interval that contains 0 is the same verdict as a two-sided test at α = 1 − C failing to reject.

6.10 · Setting up a two-sample test

H₀: no difference, and the pooled proportion

A two-sample z test asks whether a difference between two sample proportions is larger than chance would give.

In an experiment, 300 volunteers were randomly assigned: 150 got a text reminder, 150 did not. 87 of the reminded group got a flu shot (0.58) and 66 of the others (0.44).

\begin{aligned}H_0\!&: p_1 - p_2 = 0 \\ H_a\!&: p_1 - p_2 > 0\end{aligned}

where p₁ and p₂ are the true proportions who would get a flu shot with and without a reminder.

Pooling. H₀ says the two proportions are the same, so the test estimates that one common value from both groups together, the pooled (combined) proportion. In the figure the two groups merge and the shots level out across them:

\hat p_c = \frac{X_1 + X_2}{n_1 + n_2} = \frac{87 + 66}{150 + 150} = 0.510

condition	check
Random	random assignment to the two treatments
10%	not needed: no sampling from a population
Large counts	with p̂_c: 76.5, 73.5, 76.5, 73.5, all ≥ 10

6.11 · Carrying out a two-sample test

z, the p-value, and a conclusion in context

The test statistic measures the observed difference in standard errors, with the standard error computed from p̂_c.

\begin{aligned}\text{SE} &= \sqrt{\hat p_c(1-\hat p_c)\Big(\frac{1}{n_1} + \frac{1}{n_2}\Big)} \\ &= \sqrt{(0.51)(0.49)\Big(\frac{1}{150} + \frac{1}{150}\Big)} \\ &= 0.0577\end{aligned}

\begin{aligned}z &= \frac{(\hat p_1 - \hat p_2) - 0}{\text{SE}} \\ &= \frac{0.58 - 0.44}{0.0577} = 2.43\end{aligned}

p-value = P(Z ≥ 2.43) = 0.0076, the red tail in the figure.

Conclusion. “Because the p-value of 0.0076 is less than α = 0.05, we reject H₀. We have convincing evidence that a text reminder increases the proportion of people like these volunteers who get a flu shot.”

Random assignment lets the conclusion be about cause. The volunteers were not a random sample, so it reaches only people like them.

Pool for the test, because H₀ says p₁ = p₂. Don’t pool for the interval, which makes no such assumption.