Elliot Tucker-Drob and Tim Bates have published a meta-analysis [fire-walled] of the Scarr-Rowe effect, the observation that the heritability of intelligence is lower in impoverished families.
The reaction to the analysis has been a little strange. I read the meta-analysis as broadly supportive of the existence of the effect, more details on that below. To the extent I have some personal attachment to the finding, which I admit, I felt good about it.
But to my surprise, the reaction on Twitter sounded as though they had disconfirmed the finding. I think the first tweet I saw was this one….
@StuartJRitchie Turkheimer’s GxSES study is always cited even though it’s an extreme outlier in literature, as shown in recent meta-analysis
— Timofey Pnin (@pnin1957) December 7, 2015
Boy did he get it wrong, I thought, and (as I end up regretting every time) shot off a reply:
— Eric Turkheimer (@ent3c) December 14, 2015
To my surprise, Tim Bates said in fact, they were right….
— Timothy Bates (@timothycbates)
Some comments were even stronger:
@StuartJRitchie Yes of course, answer all the comments. Nail Turkheimer (2003).
— James Thompson (@JamesPsychol) December 7, 2015
What was going on? Had I completely misread the paper?
It turns out that of this had started up in response to a more general blog post by Stuart Ritchie (I hope to find the time to comment on that whole argument), in which he described the effect size as an “outlier”.
Upon further twittering, Stuart agreed that “extreme outlier” (by the other poster) was an overstatement, but still… what exactly did Tucker-Drob and Bates show? The effect size we reported in 2003 was indeed the largest in the meta-analysis, however:
1) It certainly wasn’t an “extreme outlier” and other than being the largest wasn’t an outlier at all. It was reasonably close to the high-side of the other estimates. Plus, TD&B performed a quantitative test of the null hypothesis that all US effect sizes were drawn from the same population, and failed to reject it.
2) Not only was the average effect size non-zero in the US, it remained non-zero if Turkheimer (2003), or even every study I have had anything to do with, was removed from the analysis.
3) Although the mean effect size was smaller than what we reported, it was by no means trivial, in a p
