Exactly what my sibling comment said, it's the publisher (presumably a human) who does the cleaning. It's not as easy to get a bot to clean things up and then measure weight, and the various "Readability" views aren't always correct (they're using heuristics).
Sounds like a publisher-assisted "readability view" would solve both problems, though. It would allow the user to view a "lighter" version of the page and search engines could just measure how large the "non-light" version is and penalize accordingly. I'm afraid that would incentivize the publisher to put ads and other crap in the "light" view, though.
Sounds like a publisher-assisted "readability view" would solve both problems, though. It would allow the user to view a "lighter" version of the page and search engines could just measure how large the "non-light" version is and penalize accordingly. I'm afraid that would incentivize the publisher to put ads and other crap in the "light" view, though.