Is a longer essay really a better one? No. But, at the level the students were at, it just so happened that the students who were able to write better, also were able to write longer essays.
It could also be the case that length is one of the features your human graders are using to grade essays. I.e., it might really be causal, rather than merely correlated.
In my (anecdotal) experience, teachers certainly do this. While in college I developed the skill of utilizing excessively long and verbose language while elucidating simple points simply to incrementally increase the length of essays [1].
Luckily a great prof in grad school (thanks Joel) beat this bad habit out of me.
[1] In college I learned to pad my essays with verbose language.
Its possible.
More generally, its also possible the human graders were doing a bad job; the ML system can only learn 'essay quality' to the extent that the training data reflects it.
However, the kaggle supplied 'straw-man' benchmark, which worked solely based on the count of characters and words in the essay, had an score of .647 with the training data. (The score metric used isnt trivial to interpret - it was 'Weighted Mean Quadratic Weighted Kappa' - but for reference the best entries had a score of ~.8 at the end)
The score of .647, just using length, is quite high.
For length to have this powerful a causal predictive effect, the human graders would have to be weighting for length, as a feature, very heavily.
I can't rule that out; but I think its highly likely a major component of the predictive effect of length was correlative, rather than causal.
It could also be the case that length is one of the features your human graders are using to grade essays. I.e., it might really be causal, rather than merely correlated.
In my (anecdotal) experience, teachers certainly do this. While in college I developed the skill of utilizing excessively long and verbose language while elucidating simple points simply to incrementally increase the length of essays [1].
Luckily a great prof in grad school (thanks Joel) beat this bad habit out of me.
[1] In college I learned to pad my essays with verbose language.