Statistics · AP Statistics
Exploring two-variable data
scatterplots and the least-squares line
Two things measured on each individual: does one tell you about the other, and how much?
Forensic scientists estimate a person’s height from a single bone. The figure plots femur (thigh bone) length against height for 14 adults. Taller people have longer femurs, and the points lie close to a straight line.
Which line? A line tried through the points misses each one by a residual, red. Square each residual (the red squares) and add up their areas. As the trial line turns, the total area changes; the least-squares line is the one that makes it smallest: here ŷ = 49.8 + 2.64x, with total area 126 cm².
The chapter starts with categorical variables (two-way tables, bar graphs, conditional percentages), then describes scatterplots, measures linear association with the correlation r, fits and checks the least-squares line, and asks what a single unusual point, or a curve, does to it.
a pattern, or chance?
Two variables are associated when knowing the value of one tells you something about the value of the other.
Statistics starts with a question about variation. Do adults with longer femurs tend to be taller? Does how a student gets to school depend on where they live? Each question pairs two variables measured on the same individuals.
Data always vary, so some pattern can appear by chance alone. The figure tests that. On the left are the real pairs, r = 0.95. On the right the same 14 heights are shuffled among the same 14 femurs, at random, again and again. Shuffling breaks any real link, so whatever pattern is left is chance.
Below, the correlation of each of 40 shuffles. They scatter around 0, and the farthest is −0.81. The real 0.95 is beyond all of them: chance alone is not a believable explanation for a pattern this strong.
That is the reasoning of the whole course in miniature: ask what chance alone would produce, and see whether the data look like it. Later units make it exact.
two-way tables and bar graphs
A two-way table counts the individuals in each combination of two categorical variables.
300 students were asked where they live and how they usually get to school:
| walk or bike | bus | car | total |
|---|
| city | 48 | 36 | 16 | 100 |
|---|
| suburb | 22 | 50 | 48 | 120 |
|---|
| rural | 6 | 30 | 44 | 80 |
|---|
| total | 76 | 116 | 108 | 300 |
|---|
The totals at the edges are the marginal counts: each variable on its own. 108 of the 300 students come by car, whatever the place.
Side-by-side bars (left) draw the counts. They are easy to read but hard to compare when the groups differ in size: there are 120 suburban students but only 80 rural ones, so counts mislead.
Segmented bars (right) fix that: each group is one bar of height 100%, cut into the shares of its own students. Rural students come by car 55% of the time, city students 16%. A mosaic plot (2.3) also makes each bar’s width match its group’s size.
Choose which variable splits the bars by the question. To compare how places travel, make a bar per place.
three kinds of relative frequency
A relative frequency divides a count by a total. Which total you divide by decides what it means.
- Joint: one cell over the table total. 44 of 300 students are rural and come by car: 44/300 = 14.7%.
- Marginal: a row or column total over the table total. 108/300 = 36% of all students come by car.
- Conditional: a cell over its own row or column total. Of rural students, 44/80 = 55% come by car. Of the car riders, 44/108 = 40.7% are rural.
The last two differ: ‘of rural students, the share by car’ and ‘of car riders, the share rural’ divide by different totals. Read the wording for the group named after of, or after given: that total is the denominator.
The mosaic plot shows all three at once. Each column’s width is its place’s marginal share; each block’s height is a conditional share within the place; so each block’s area is a joint share. The rural car block is 14.7% of the whole mosaic.
compare the conditional distributions
Two categorical variables are associated when the conditional distribution of one differs across the categories of the other.
The conditional distributions of travel, for each place, and for all students together:
| walk or bike | bus | car |
|---|
| city | 48% | 36% | 16% |
|---|
| suburb | 18% | 42% | 40% |
|---|
| rural | 8% | 38% | 55% |
|---|
| all | 25% | 39% | 36% |
|---|
If travel did not depend on place, every row would match the ‘all’ row, and the segmented bars would be cut at the same heights. They are not: the car share climbs from 16% in the city to 40% in the suburbs and 55% in the country, while walking or biking falls from 48% to 8%. There is an association between where a student lives and how they get to school.
Compare percentages, not counts. 48 suburban car riders outnumber 44 rural ones, yet a smaller share of suburban students come by car.
An association in data is not proof of cause. Living in the country need not itself cause the driving: distance to school, a variable not in the table, may explain much of it. Chapter 8 tests whether such a difference could be chance (the χ² test).
direction, form, strength and unusual features
A scatterplot shows two quantitative variables measured on the same individuals, one point per individual.
The explanatory variable goes on the x-axis: the one that may explain or predict. The response variable goes on the y-axis: the outcome. Here femur length explains height.
| femur (cm) | 39.0 | 39.8 | 40.7 | 40.9 | 40.9 | 43.1 | 43.5 |
|---|
| height (cm) | 154.3 | 152.9 | 160.5 | 162.1 | 156.1 | 159.8 | 162.2 |
|---|
| femur (cm) | 44.1 | 45.2 | 46.5 | 46.9 | 47.2 | 48.7 | 49.8 |
|---|
| height (cm) | 163.3 | 168.2 | 169.7 | 180.1 | 172.3 | 178.2 | 183.1 |
|---|
Describe a scatterplot by four features, in context:
- Direction: positive (larger x goes with larger y), negative, or neither.
- Form: linear, or curved, or clusters.
- Strength: how closely the points follow the form: strong, moderate or weak.
- Unusual features: outliers, gaps, clusters.
Here: there is a strong, positive, linear association between femur length and height for these adults, with no clear outliers. The small plots show other patterns: a weaker negative one, a strong curve, and none.
the strength of a linear association, as a number
The correlation r measures the direction and strength of a linear association between two quantitative variables.
Standardise each variable: zx and zy say how many standard deviations a value lies from its mean. Lines at the means cut the plot into four quadrants. A point up and right (or down and left) has a positive product zxzy, a red rectangle; up and left or down and right, a negative one, blue.
Add the signed areas and divide by n − 1. For the femurs, 14 products sum to 12.305, and 12.305/13 = 0.947. Nearly all the area is red: a strong positive association.
r lies between −1 and 1. Its sign is the direction; its size is the strength: near ±1 the points lie close to a line, near 0 there is no linear pattern. Interpret it in context: r = 0.95 says there is a strong, positive, linear association between femur length and height.
On the exam, find r with a calculator; know what it means.
only linear, sensitive to outliers, not causation
r is a summary of a straight-line pattern. Always look at the scatterplot first.
- Linear only. The red plot has a strong curved relationship, yet r = −0.02. r near 0 means no linear association, not no association. A high r doesn’t prove a line is the right form either.
- No units. r stays the same if either variable changes units: with femurs in inches, r is still 0.947. Swapping x and y doesn’t change it.
- Not resistant. Like the mean, r can be moved a lot by one outlying point (2.9).
- Not causation. A strong correlation doesn’t mean that changing x changes y. A lurking variable may drive both (ice-cream sales and drownings both rise in summer heat), or the link may run the other way. Only a well-designed experiment (Chapter 3) shows cause.
The four small plots: strong, positive, linear, r = 0.97; moderate, negative, linear, r = −0.71; strong, curved, r = −0.02; no association, r = 0.01.
a line for predicting y from x
A regression line models how a response y changes with an explanatory x, and predicts y for a given x.
ŷ (y-hat) is a predicted value, not an observed one: write the hat. b is the slope and a the y-intercept. Write the model in context: predicted height = 49.8 + 2.64 (femur length).
Predicting. For a femur of 45.0 cm, ŷ = 49.8 + 2.64(45.0) = 168.5 cm. That is the average height the model gives for adults with that femur length; any one of them may be taller or shorter.
Extrapolation is predicting outside the range of x in the data, here 39.0 to 49.8 cm. The line has no evidence there: the form may change. A child’s 30 cm femur gives ŷ = 128.9 cm, from a model built on adults only. In the figure the dot turns grey whenever it leaves the data’s range.
what the line misses, and the residual plot
A residual is an observed value minus its predicted value: how far the point lies above or below the line.
The adult with femur 46.9 cm is 180.1 cm tall. The line predicts 173.5 cm, so the residual is 180.1 − 173.5 = 6.6 cm: 6.6 cm taller than predicted. A negative residual means the line overestimated. For the least-squares line, the residuals add to 0.
A residual plot graphs each residual against x (or against ŷ). In the figure each residual drops from the scatterplot to the plot below: the line laid flat at 0.
- No pattern, an even scatter about 0: a linear model is appropriate. That is what the femurs show.
- A curve in the residuals: the relationship is not linear; another form would fit better (2.9).
- A fan, spread growing with x: predictions are less precise for larger x.
A residual plot shows the leftover pattern more clearly than the scatterplot, because the line’s trend has been taken away.
the line that makes the squared residuals smallest
The least-squares regression line is the line that makes the sum of the squared residuals as small as possible.
On the right, the sum of squared residuals for each slope (each line through the mean point): a parabola, lowest at b = 2.64. Any other slope gives a larger total.
So the line always passes through (x̄, ȳ) = (44.02, 165.91), and its slope has the sign of r. An increase of one standard deviation in x predicts an increase of r standard deviations in y: less than one, the ‘regression to the mean’.
Interpreting. Slope: for each additional 1 cm of femur length, the predicted height increases by about 2.64 cm. Intercept: an adult with a femur of 0 cm would have predicted height 49.8 cm; that is extrapolation and has no meaning here, as often for an intercept.
Regressing x on y gives a different line. Predict the response from the explanatory variable, not the other way.
how well the line fits, and how far off it typically is
Two numbers say how good the line’s predictions are: the coefficient of determination r2 and the standard deviation of the residuals s.
Software prints the line like this:
| Predictor | Coef | SE Coef |
|---|
| Constant | 49.807 | 11.454 |
| Femur | 2.6375 | 0.2594 |
S = 3.2393 R-sq = 89.6%
Read a beside Constant and b beside the explanatory variable: ŷ = 49.8 + 2.64x. (SE Coef is for Chapter 9.)
r2 = 89.6%: without the line, the squared deviations of the heights from ȳ total 1210 cm²; about the line, 126. The line removes 1 − 126/1210 = 89.6% of it. In context: about 89.6% of the variation in height is accounted for by the linear model with femur length.
s = 3.24 cm, from the residuals (divide by n − 2):
In context: actual heights typically differ from the heights the line predicts by about 3.2 cm. The blue band is ŷ ± s.
what one point can do to the line
One unusual point can change a regression a great deal, or hardly at all. Where it lies decides which.
- An outlier in regression is a point with a large residual: far above or below the line.
- A high-leverage point has an x-value far from x̄. It has the power to tilt the line.
- An influential point is one whose removal changes the slope, intercept or correlation markedly. High-leverage points often are.
The red point is added to the 14 adults and moved. The grey line is without it:
| the 15th point | b | a | r |
|---|
| none | 2.64 | 49.8 | 0.947 |
| (66, 223.9) | 2.64 | 49.8 | 0.985 |
| (66, 197.9) | 1.76 | 88.0 | 0.931 |
| (44, 183.9) | 2.64 | 51.1 | 0.846 |
Far right and on the line: high leverage but not influential on the slope; it even raises r. Far right and below the line: it drags the slope from 2.64 to 1.76, influential. Near x̄ and high above: an outlier that lowers r to 0.846 but barely moves the slope.
Never just delete an unusual point. Check it for an error; if it is real, report the fit with and without it.
logarithms straighten exponential growth
When a scatterplot curves, transforming a variable can make it linear; then the least-squares line can model it.
A culture of bacteria was counted every hour for 10 hours. The counts curve upward: each hour multiplies the count rather than adding to it. A line fits badly even though r = 0.900: its residual plot is a clear curve.
Take logarithms of the counts. Exponential growth becomes linear: the figure moves between the two. Against t, log10(count) is straight, r = 0.998, and its residuals show no pattern.
Predicting needs the transformation undone. At t = 8.5 hours: log10(count) = 1.594 + 0.1626(8.5) = 2.976, so count ≈ 102.976 = 946. The slope says the predicted count is multiplied by 100.1626 = 1.454 each hour.
Other curves need other transformations: log y against log x straightens a power model, y = axb; a square root or a reciprocal may straighten others. Choose the transformation whose residual plot shows no pattern.
← Exploring one-variable data · Collecting data →