I like the way "Is Bloom's "Two Sigma" phenomenon real? If so, what do we do about it?" is loaded. Touches on several "questions" in the vein of this article.
One load is "why don't we know this with more certainty?" It's a useful thing to know. What is achievable? What's the bar? How good could education be, regardless of resources and other limitations. If true, that can be our 100% marker for instructional effectiveness.
From there, we cold say that average school instruction is 10% effective or 60% effective.
Realistically, this needs to be tackled with a lot of force to make progress. Change is likely inconvenient, in a real world environment with limited resources. Maybe school should be 3 hrs per day, not 8.
In any case, I think it's notable that high-end private education, which has more resources available, have not pursued these kind of goals any more than public education.
There's an assumption in this line of thinking which I think is unlikely to be true: that this stuff is measurable, at least in principle.
I think 'instructional effectiveness' can't ever be clearly and unambiguously defined, much less measured. As with anything that involves people. In my experience, most attempts to do so only cause more harm than good.
There are different kinds of measurable, and the purpose of the measure is important. Measuring for the purpose of management (eg creating incentives for teachers and schools), measuring for the purpose of most academic publishing, etc. Each have their own pitfalls, and I agree with you on these.
That said... there is clearly such a thing as better and worse instruction.
Where the 2-sigma claim gets interesting is scale. We're not concerned with marginal differences (management measures) and we're not concerned with legibly identifiable causal relationships (academic publishing). We're just concerned with establishing a high watermark.
The simple measures (eg testing) we have, work fine for that... in the context of math, reading/writing, foreign language, etc. Stuff that's easy to test for, and assuming we're only interested in big differences.
Two sigma implies that within 1 year of instruction, the two sigma group will have progressed by several years. 12th grade math level by grade 10. This doesn't have to be achievable by every teacher, it just has to be achievable at the high end.. assuming a random sample of students.
Agreed though.. we have a history of insisting that the unmeasurable be measured... and its a very common, long term failure mode. more harm than good. I'm not suggesting implementing anything, let alone implementing anything using measures. Just setting that watermark, so we know what awesome looks like.
If one is trying to improve education given class sizes of 1 or 2 dozen students, it may not be helpful to pursue better quantitative studies for class sizes of 1 or 2 students.
The goal, in my view, of such research is to make discoveries. I'm dubious of academic research that targets scalable educational methods too closely.
In essence, I agree with Patrick. I want to know if "2-sigma" is true, replicable. Establish that watermark. At that point we'll have a high watermark for education, what a median student can achieve in a near ideal environment. Is there really a 2 sigma difference between normal school results and ideal?
If so, this difference represents potential.
Figuring out how to apply that IRL with resource constraints, scaling issues, etc... That may not be a job for academics. Either way, it comes later.
One load is "why don't we know this with more certainty?" It's a useful thing to know. What is achievable? What's the bar? How good could education be, regardless of resources and other limitations. If true, that can be our 100% marker for instructional effectiveness.
From there, we cold say that average school instruction is 10% effective or 60% effective.
Realistically, this needs to be tackled with a lot of force to make progress. Change is likely inconvenient, in a real world environment with limited resources. Maybe school should be 3 hrs per day, not 8.
In any case, I think it's notable that high-end private education, which has more resources available, have not pursued these kind of goals any more than public education.