Two Lines: A Valid Alternative to the Invalid Testing of U-Shaped Relationships With Quadratic Regressions
Simonsohn (2018) · Sign change, not parabola
Simonsohn argues that the standard way of testing U-shaped relationships, the quadratic regression, is invalid, and proposes a replacement. A U-shaped hypothesis claims only a sign change: the effect of x on y is positive at low values of x and negative at high values, or the reverse. A quadratic adds an arbitrary assumption that the relationship is exactly a parabola. Where the true curve rises and then flattens, as with a ceiling effect, the quadratic cannot produce a long plateau and generates a spurious downturn; in some scenarios the false-positive rate reaches 100%, concluding that y = log(x) is U-shaped. It can also miss a real U-shape. Using the Johnson-Neyman procedure to test whether the slope changes sign within the observed range does not rescue it, since those calculations rest on the same assumption.
The alternative is two regression lines, one for low and one for high values of x, which can be estimated together, for efficiency, as an interrupted regression. A regression slope is the average slope in the data whatever the true shape, so the test makes no functional form assumption: a U-shape is supported when the slopes have opposite signs and both are significant. To place the break he introduces the Robin Hood algorithm, which moves it within the flat region around the most extreme point of a fitted spline so the statistically weaker line gets more observations, taken from the stronger. In simulations where the true curve was a strong early effect followed by a long flat stretch, the case expected to produce most false positives, or a monotonic logistic curve, the quadratic often gave 100% false positives, while the two-lines test was typically below the nominal 5%, and Robin Hood generally outperformed the four other breakpoint rules on power. Replicating a published too-much-talent effect in international football with independently obtained data, he found the second line positive, though far from significant, and concluded the data do not support too much talent.
The algorithm was assessed by simulation, without asymptotic properties derived. N, W or X shapes can be misread, shapes he says also undermine the interpretation of quadratics. The false-positive rate is not exactly known, though expected almost never to exceed 6% at a nominal 5%. The break is not an estimate of where the effect turns.
Psychological safety is not discussed, though one of the eight articles cited as examples of U-shape testing, Koopmann and colleagues (2016), is on team tenure and team psychological safety climate. The bridge is ours. Claims that psychological safety can be too high are claims of a sign change: Eldor, Hodor and Cappelli (2023) report turning points at high levels of psychological safety climate, and Cangiano (2026) proposes an inverted U. We read Simonsohn's argument as a test such claims are open to: a significant curve is not a downturn, which needs a significantly negative slope above the break, at higher levels of safety.
Save this paper to a bibliography, keep notes on it, and export citations. Create a free account →
Related psychsafety.com articles (3)
Related papers (4)
- Eldor, Hodor & Cappelli (2023): The Limits of Psychological Safety: Nonlinear Relationships with Performance
- Cangiano (2026): How safe is too safe? A dual-pathway framework for understanding the dark side of psychological safety
- Kostopoulos & Bozionelos (2011): Team Exploratory and Exploitative Learning: Psychological Safety, Task Conflict, and Team Performance
- Katahira et al (2020): Commentary: A Robust Data-Driven Approach Identifies Four Personality Types Across Four Large Data Sets
