Psych Safety, by Iterum

Kernel-Based Regularized Least Squares in R (KRLS) and Stata (krls)

Ferwerda, Hainmueller & Hazlett (2017) · A marginal effect per point

Ferwerda, Hainmueller & Hazlett · Methodological · Measurement & Method · Journal of Statistical Software · 2017 · Open access

Open this neighbourhood on the live map →

Ferwerda, Hainmueller and Hazlett put kernel-based regularized least squares (KRLS), the machine learning method set out in Hainmueller and Hazlett (2014), into a Stata command, krls, with a matching R package, KRLS. Their case is practical: generalised linear models fix the functional form in advance, guessing which squared or interaction terms to add is difficult and can make matters worse, and KRLS learns the shape from the data while reporting output that reads like a regression table.

Each prediction is a weighted sum of how similar an observation is to every other, with a penalty favouring smoother functions over erratic ones. The command standardises the variables, picks the penalty by leave-one-out cross-validation and by default sets the kernel bandwidth to the number of covariates, with options to override both. The table reports the average pointwise marginal effect with its standard error and quartiles, and the effect at every observation can be saved.

The worked example regresses GDP growth over 1960 to 1995 for 65 countries on initial GDP, trade share, schooling and political assassinations. KRLS gave an in-sample R2 of .52 against .32 for OLS. In a smaller model of schooling and assassinations, plotting the pointwise effects showed the effect of schooling positive at low levels, shrinking towards zero in the middle and slightly negative at high levels; in the full model, the effect of trade share turned negative as assassinations rose. OLS rebuilt by hand with cubic terms and interactions reached .50, against .52. On leave-one-out forecasts KRLS and the original OLS model performed similarly (MSE 2.97 against 2.75), while the rebuilt OLS model had an MSE of 17.6.

They name the limits. The average effect is only a summary: a covariate can be positive in one region and negative in another and average near zero. The implementation begins to slow with several thousand observations, and weights and heteroskedasticity-robust and cluster-robust standard errors are listed as hoped-for improvements. A footnote adds that at the most extreme values the estimated effect of trade share returns to zero, in part because the fit tends towards zero for points far from the rest of the data, to protect against extrapolation.

The paper does not mention psychological safety; the bridge is ours. Eldor, Hodor and Cappelli (2023), whose turning points Tom Geraghty has argued appear arbitrary, possibly statistical artefacts, cite Hainmueller and Hazlett (2014) for the method, though not this software paper. We read the extrapolation footnote and the lack of cluster-robust errors in this release as two things to check before reading a turning point or a ceiling off KRLS output from survey data on people nested in teams.

Ferwerda, J., Hainmueller, J. and Hazlett, C.J. (2017) 'Kernel-based regularized least squares in R (KRLS) and Stata (krls)', Journal of Statistical Software, 79(3), pp. 1–26.

Save this paper to a bibliography, keep notes on it, and export citations. Create a free account →

Related psychsafety.com articles (3)

Related papers (2)

View the source paper →