Psych Safety, by Iterum

Kernel Regularized Least Squares: Reducing Misspecification Bias with a Flexible and Interpretable Machine Learning Approach

Hainmueller & Hazlett (2014) · Functional form learned, not assumed

Hainmueller & Hazlett · Methodological · Measurement & Method, Critique & Boundary · Political Analysis · 2014 · Paywalled

Open this neighbourhood on the live map →

Hainmueller and Hazlett propose Kernel Regularized Least Squares (KRLS) to let social scientists fit regressions without first committing to a functional form. Their target is the generalised linear model, the workhorse of the social sciences, which imposes strict functional form assumptions; misspecified models, they note, can lead to bias, incorrect inferences and fragile, model-dependent results. Adding higher-order terms only allows interactions of a prescribed type and, they argue, can under common conditions make matters worse while appearing to work.

KRLS models the outcome not as a weighted sum of predictors but through each case's similarity to every other case, measured with a Gaussian kernel, on the premise that more similar observations are expected to have more similar outcomes. A penalty favours smoother, less 'wiggly' functions over ones that chase noise; it is set by leave-one-out cross-validation and the kernel width by a default rule, so by default the analyst makes neither choice. The fit gives a marginal effect at every observation: averaged, these read like a regression coefficient; plotted, they show whether an effect varies across cases. They derive closed-form variances, establish unbiasedness, consistency and asymptotic normality, and supply software.

With simulated linear data, KRLS standard errors were on average 14% larger than those of ordinary least squares. Where the outcome depended only on a mildly curved x1, a regression with an interaction term fitted well (R squared 0.89), its main and interaction terms all highly significant, yet led to entirely false conclusions; KRLS put the average effect of the irrelevant x2 at zero.

Unbiasedness holds only for the best approximation given the penalty, not the true function, the same sense, they note, in which OLS is unbiased. In simulation KRLS fails to approximate a sharp discontinuity, though it still beats OLS on a continuous regressor, variances assume homoscedastic errors, memory limits most users to samples in the tens of thousands, and unobserved confounders remain. They do not claim it is generally superior to other approaches.

The bridge is ours: the paper does not mention psychological safety. Eldor, Hodor and Cappelli (2023) cite it, and Tom Geraghty's critique of their study names their use of KRLS alongside fixed-effects regressions, which, though sophisticated, he says raises concerns about interpretability, replicability and potential overfitting. We read the paper as bearing on curvilinear claims: the squared terms Kostopoulos and Bozionelos (2011) fitted, and the inverted U Cangiano (2026) proposes, for which Eldor, Hodor and Cappelli is the key reference. The question it raises is whether a curve comes from the data or from the terms an analyst chose. On their own account, KRLS smooths over a sharp threshold the analyst has not modelled, though they judge thresholds that sharp to be rare.

Hainmueller, J. and Hazlett, C. (2014) 'Kernel regularized least squares: Reducing misspecification bias with a flexible and interpretable machine learning approach', Political Analysis, 22(2), pp. 143–168.

Save this paper to a bibliography, keep notes on it, and export citations. Create a free account →

Related psychsafety.com articles (2)

Related papers (4)

View the source paper →