Regression Diagnostics

Check the model assumptions and data problems before relying on a regression equation. Residuals and influence measures for every observation are shown in tables, and normality, equal variance and independence are checked with plots.

1Data Table

Paste a table copied from Excel into a cell (Ctrl+V). Click a column header (A, B…) before pasting to use the first line as column names. Double-click a header to rename it, right-click to insert/delete rows and columns, and formulas such as =A1*2 work too.

2Regression Diagnostics Options

Numeric predictors (multiple selection):
Categorical predictors (multiple selection, optional):

How to diagnose a regression model

  1. Copy the response Y and the predictor columns with their headers.
  2. Choose the response Y and the numeric/categorical predictors, and set the VIF threshold (default 5) and the outlier threshold (default |studentized residual| > 3).
  3. Click Analyze and review the diagnostic tables, plots and automatic interpretation.

Worked example with sample data

The example data are the temperature (°C), pressure (bar), additive (%) and strength (MPa) of 25 production runs. Strength is the response and the other three columns are numeric predictors. An outlier was deliberately placed in the 8th observation.

Influence plot (leverage vs studentized residual) - vertical dashed line at the leverage threshold, horizontal dotted lines at ±3
Influence plot (leverage vs studentized residual) - vertical dashed line at the leverage threshold, horizontal dotted lines at ±3

The diagnostics find 1 outlier (|studentized residual| > 3) and 2 high-leverage points (above 2p/n = 0.32). Observation 8, with residual -19.92, studentized residual -4.087, leverage 0.5981 and Cook's D 6.215, is both an outlier and an influential point that pulls the model strongly.

Observation 22 has a leverage of 0.3448, so its predictor combination is unusual, but its residual is small (0.3138) and it causes no problem. In the influence plot, observation 8 sits alone outside the -3 dotted line at the lower right.

The VIFs of 1.128 ~ 1.518 show no multicollinearity problem, and residual normality (p=0.3788) is fine. Check the measurement record of observation 8; if it is an error, remove it, refit and compare how much the coefficients change.

How it is calculated

statsmodels OLS and OLSInfluence give the leverage (hat diagonal), internally studentized residuals, Cook's D, and DFFITS/DFBETAS based on the leave-one-out variance; VIF comes from variance_inflation_factor.

The high-leverage threshold is 2p/n (p = number of coefficients) and the DFBETAS threshold is 2/√n.

What the results show

The residual and influence table, DFBETAS, VIF, residuals vs fitted, a residual QQ plot, scale-location, residuals vs order and an influence plot (leverage vs studentized residual).

Interpreting the regression diagnostics

  • Observations with |studentized residual| above the outlier threshold disagree strongly with the model, and those with leverage above 2p/n have unusual predictor combinations.
  • A funnel shape in scale-location suggests unequal variance, and a trend or cycle in residuals vs order suggests a violation of independence.
  • If the points of the QQ plot depart from the line, residual normality is doubtful; check the Anderson-Darling p-value in the automatic interpretation as well.

Frequently asked questions

Can I just delete outliers?

Delete them only when a measurement or entry error is confirmed. If they are real values, investigate the cause and compare with the model without them.

What is DFBETAS?

It shows how many standard errors each coefficient changes when that observation is removed.

How is this different from multiple linear regression?

It fits the same model but focuses on the per-observation influence tables and assumption-checking plots.

Other chart tools