Simple Linear Regression and Inference on the Slope

A fitted equation describes a line estimated from observations. The population model adds a random error to a mean response: y = a + bx + e. At a given x, a + bx is the mean response and e is the departure from it. The fitted line estimates that mean. A residual uses the fitted line, so it is observed y minus fitted y rather than the unknown population error.

Least squares chooses the intercept and slope that minimise the sum of squared residuals. Provided the predictor values vary, the slope estimate is Sxy/Sxx, where Sxy sums centred cross-products and Sxx sums squared predictor deviations. The intercept then makes the line pass through the pair of sample means.

For the usual exact t inference, errors are independent across observations and normally distributed with mean zero and constant variance at each predictor value. Normality concerns departures from the mean line at each x. It does not require all response values pooled across different x values to have a normal distribution.

Inserting a predictor value into a fitted line gives a point prediction. That number can estimate the mean response at the chosen x or predict the response of one new observation. These targets have different uncertainty.

A confidence interval for the mean response accounts for uncertainty in estimating the line. A usual prediction interval for a new observation also accounts for the new observation's random departure from its mean. With the same x and confidence level, and under the usual regression assumptions with positive error variance, the prediction interval is wider.

Check whether the proposed x is inside the observed range. Inside is interpolation; outside is extrapolation. Range membership alone does not establish accuracy. The model might miss a curve or important variation even within the data. Outside the range, the relationship may change in ways the observations cannot reveal. An interval calculated under the fitted model does not account for every possible model failure.

For a two-sided test of a zero slope, divide the estimated slope by its standard error. Under the null hypothesis and the model assumptions, this statistic follows a t distribution with n - 2 degrees of freedom. Its two-sided p value is the probability of an absolute test statistic at least as large as the one observed.

Compare the p value with a significance level selected before examining the result. A value below that level leads to rejection of a zero slope. A larger value leads to nonrejection, not a finding that the slope is exactly zero. Statistical evidence for a nonzero slope does not establish causation, a large practical effect or accurate prediction.

To construct a slope interval, multiply the slope's standard error by the critical t value and subtract and add that margin to the estimated slope. A valid two-sided 95% t interval excludes zero when the matching two-sided test rejects at 5%. Both procedures depend on the model assumptions. Residual plots can warn of curvature or changing spread, but sampling design and observation order also matter when judging independence.