It took me a long time to recognize and be able to articulate the seriously detrimental effect of reading pop sci on my thinking.
“Knowing” incomplete, false, distorted, or misleading facts is often much worse than having no information, or having never even considered a topic.
If I read fiction, it may imply some fundamental lens on the world, but I think my brain is better able to recognize it as an opinion, or subjective.
But if I internalize the “truth” of dietary principles or basic psychology, I will make real choices and have my perceptions shaped to believe I am seeing the results of these choices ... and it is often completely wrong.
This can be subtly or devastatingly negative for my success and quality of life.
I think this problem is under considered. In recent years I have started realizing that most of what I read literally makes me dumber.
It’s a thought that has more implications than I think most people want to consider. Especially people whose identity is connected to being someone who “knows” things.
I prefer having a few wrong concepts in my toolbox than none at all. As the Gates quotes in the article suggests, it's good practice to be vigilant and run a basic reality check before absorbing an idea as part of your worldview, but that you might encounter wrong ideas is not a reason to stop reading accessible science books. I don't need to read papers on economic theory if I can pick up a basic book by a respected author and at least end up with some additional concepts, ideas and a good understanding what at least some experts believe is the truth. That doesn't mean I need to believe everything they say and if it matters you should neither rely on one journal's output nor on one author's book as the ultimate truth.
I can read a book about dieting or psychology or evolutionary theory and I should be aware that I won't end up as an expert, but at least I'll very likely be better informed.
"Why we sleep" has lots of good content, generally coherent information, etc. As the article here shows you shouldn't believe everything at face value but you will learn something. I'd love a reaction by the author to the claims here, but even if he says "I was wrong to do this" it doesn't invalidate his whole book or all his knowledge on the topic. He might have taken it out to avoid confusing laymen that all like to imagine themselves in the "under 5 hours" category. That doesn't make it sound in a scientific way but it does if his main aim is to get a message across that sleep is grossly undervalued.
There's lots of science on the need for sleep and what the article here says that there's no proof that sleep under 5 hours is harmful is simply not true.
Anyone else tired of learning and internalizing cool new concepts and then those ideas you had come to adopt and believe in gets debunked/disproven soon after? Might be better just to not believe anything so you don’t get betrayed anymore lol
I often think about that. It is why I wrote many thoughts about "Maturity Models": the best ideas for any subject (the system being far from ... uh, mature, at this point). But the idea is to take best practices and collate and order them with dependencies, instead of reading thousands of books to know what is "next" for one's own growth in an given subject. The idea is that multiple optionally-integrated-if-and-when-they-want communities of practice could develop. More info buried at at http://lukecall.net under Lessons Learned etc etc.
Edit: url & minor fixes. Also I hope to eventually progress with this, using my http://onemodel.org project.
Edit: ideally anything (or any source of info) could be assigned transitive trust levels. Dietary science would get a low one in my system at this time. ;) Physics, math, and epidemiology would probably get much higher ones.
For me, it feels like I've fallen face flat into a puddle of doubt. Yes, believing in some particular idea and putting it to practice felt great at the time, but now the science has said it's bullshit and I'm wrong for believing in it. If I choose to keep believing in it because it makes me feel good, they're going to call me stupid and a science denier!
Half joking. I'd rather be stupid, get my sleep, and feel great. ;)
The book full of excellent, albeit commonsense advice and probably mostly claims made from studies which would hardly even need to verified by science because it’s pretty obvious from anecdotal experience. Your brain works better after you’ve had a good nights sleep. You’re body can repair while resting.
I think readers mostly enjoy the book because it’s a reminder of sleeps importance in a world where spending time on the internet and working long hours can seem even more important.
It’s good you decided to write off a whole genre, I’ll enjoy taking a light hearted approach an a open mind to reading similar publications in the future.
I once wrote a paper related to genetics. It's known that cells contain various parts (organelles), some of which used to be independent cells. Those are rather well researched but finding scientific references for some points were surprisingly difficult and I had to quote some 1940s papers to actually have a solid citation for a well-known but not trivially verifiable fact.
I can imagine much common knowledge is difficult to proof and the various alternate Nobel prizes are great at rewarding some of this important but hardly recognised work of "proving the obvious".
I'm tired of the endless posts over Why We Sleep. The book attempts to cover hundreds of studies in an accessible way, and it largely succeeds. It's certainly fluffy at times (as are all popular books, by necessity), but if you didn't like its message, nitpicking at minor points doesn't refute the central thesis, and pretending it does is below the standards of even internet flame wars.
Like, this kind of exchange is exactly why academics try to avoid randos from the internet. They tend to seize on one point, declare victory, and refuse to change their minds. And when the academic doesn't grovel in compliance, they declare academia to be a failure. As far as I'm concerned, UC Berkeley responded perfectly.
Falsifying data is not being 'fluffy', it's academic misconduct. You might not personally care about this but academic norms exist for a reason. Universities are trusted institutions precisely because they follow these norms.
But" Why We Sleep" is not intended for an academic context, right?
I mean, obviously it wouldn't pass a "peer-review" analysis, but that is exactly what happens every time in science pop readings. Either we eliminate every science reading that is not strictly papers, or we accept some misinformation here and there.
No, but the post is largely a quote of a blog post (the guy who declared UC Berkeley a failure), which is in turn a followup on a different blog post.
I respect Andrew Gelman, but in this case I think he jumped to fit this into a pattern where it doesn't belong. I've looked into all the criticisms of Why We Sleep (which, again, covers hundreds of studies), and there's no p-hacking, there's no fraudulent data, there's no piles of studies retracted, and there's no crisis of replication. Literally the worst criticism is that one bar of a bar graph is removed, in a way that doesn't even change the point of the single sentence referring to it. There are simply better targets out there than sleep.
Yes, I copy-pasted that comment because this is my tenth time commenting on guzey's critiques, and I'm tired of writing new ones.
I still don't think that trimming a graph in a way that doesn't even change the conclusion of the sentence that refers to it (having less than 8 hours of sleep increases injury risk, and it's still true upon included the omitted bar, which barely even changes the average) is worth trashing an entire 300 page book. Academia isn't about following a set of rigid rules, as if it were some endless high school project. It's about finding the truth, and I've seen no evidence that this was harmed at all here.
I agree that academia isn't only about following a set of rigid rules, but following rigid rules is certainly partly what academia is about. For example, you must include all data. Omitting data is research misconduct.
> I still don't think that trimming a graph in a way that doesn't even change the conclusion of the sentence that refers to it … is worth trashing an entire 300 page book.
You're strawmanning. The trimming is simply the most obvious of many errors found in a single chapter. There are other known errors as well as (probably) unknown errors.
That said, I think we should be forgiving when people make mistakes as long as they own up to them. Neither the author, the publisher nor the university have demonstrated much of a commitment to research integrity in this case. That's my main complaint.
The general message and key takeaway I got from the book was that sleep affects your health in various ways and getting a sufficient amount of higher quality sleep is better than not doing so.
Do any of these errors reverse that information? Is sleep actually harmful? Should we in fact be loading up on caffeine and alcohol and sleeping 3 hours a night? If not, then what's the point?
Bullshit arguments can lead to true conclusions. They are still bullshit and it bears pointing out, lest other conclusions are drawn from the same pool.
You can even use the actual data in the article. Leave only a chart with 2 bars:
- 5h sleep - 60% injury likelihood
- 6h sleep - 75% injury likelihood
Now maintain the conclusion that the graph supports the idea of the likelihood of injury being higher the fewer the hours slept per night. Of course such manipulation will not make your book a bestseller.
Studies [0] show that the likelihood of your science being bunk increases with your desire to turn them into bestsellers and make more money from them.
[0] Too lazy to manipulate a bar chart to show whatever I want it to show because I don't actually aim to turn this comment into a bestseller.
These were exactly my thoughts when I first saw the critics of this book some time ago (btw I don't see much added value in this publication relative to what Alexey Guzey wrote initially).
Like I agree that removing the bar is bad, but Walker's takeaway is pretty much the same as that of the original paper from which the graph is taken - the paper states that predictor for injury is sleeping less than 8 hours, which Walker points out as well.
> First, an enormous literature dedicated to the treatment of depression with sleep deprivation has found that people with depression frequently benefit by not getting a good night’s sleep.
> Second, Walker directly contradicts himself in Chapter 7 by acknowledging that there are cases when a good night’s sleep is not helpful after all...
> Finally, although Walker states that “sleep deprivation is not a realistic or comprehensive therapy option”, a review chapter of sleep deprivation in the book Sleep, Neuronal Plasticity and Brain Function published in 2014 reads: "[C]onsidering its safety, this technique [sleep deprivation] can now be considered among the first-line antidepressant treatment strategies for patients affected by mood disorders."
There is no one-size-fits-all advice for sleep, it would seem. For some folks, yes, too much sleep is harmful, and yes, sleep deprivation therapy benefits them.
Moreover, the point is that scientific literature should be peer-reviewed.
Absolutely not. Not only do Guzey's nitpicks not refute the main point, they fail to refute even the subpoints and subsubpoints.
The example cited here, where people on 5 hours of sleep actually have slightly less chance of injury than 6, is by far the most egregious one, but even it doesn't change the conclusion of the sentence where it appears. At most, it would require a footnote: "also, something weird is going on in the tiny 5 hour group".
Nobody is claiming the data doesn't matter. The data they showed supported the conclusion. The author seems mad that there is additional data that also supports the conclusion that they didn't show. My question is, "so what?".
If the additional data falsified the conclusion, that would be one thing. But it doesn't, so why keep harping about how it wasn't shown?
Did you need to spend money on a book to tell you that?
Is it okay for me to make money by distorting science? At what point does my behaviour cross from "finding the best presentation for the data" into "misleading the public"?
This isn't just one book either. It's a whole industry. People know that some magazines have comprehensive fact-checking, and they used to expect a level of fact-checking in pop sci print books. They didn't know that pop-sci books have almost no fact-checking.
Did you need a full book to tell you that though? The fundamental problem with pop sci books is that very often the most important facts (and probably the only generally communicable facts) about the topic should not need more than a few dense pages in a long-form article. But you can't sell an article for twenty bucks so the professors who would love a million or two (do they make that much?) Just expand it into drivel and cut corners to "simplify" the prose and take creative license.
Even the greatest popsci book of all time, a brief history of time, is famously a tome almost no one ever reads fully[1]. Every popsci book I've taken up I can never finish because half of it would be repetitive BS and I'll come out of it not taking anymore takeaway points than what was in the article in Forbes promoting the book. The motivations for publishing popular science books by professors are probably to blame here, on top of the fundamental research ethics problems that plague academia nowadays.
> Did you need a full book to tell you that though?
Yes, actually. The point of writing an entire book is to present enough evidence for the reader to come to the author’s conclusion on their own.
The important part of reading a book, the part that has a chance of altering your behavior in the future, is spending time considering the subject matter from various different angles. That’s what gives you the ability to recognize similarities between situations described in the book and situations that appear in your life. This ability to see the things that happen to you in a new light is the real value these books provide.
This is no different that standard text books. There is a lot of filler. You can skim if the primary concept already makes sense, or you can read the extra details if you are confused.
If that's the only factual information worth taking away from the book, then what is the point of the book? I have plenty of anecdotal experience to suggest that sleep impacts my health already. A book that uses questionable data to back up something I can observe directly seems to have absolutely no utility.
If you change your habits so that you sleep for eight hours a day instead of seven, after reading Why We Sleep, but in fact you didn't have to and you function just as well on seven, then you are literally wasting 4.1% of your life. That is a lot of life.
I’m pretty sure if you’ve read the book he says that you’re better off to “plan” for 8 hours sleep, even though you might not need it? The author clearly takes this into account.
He doesn’t say If you wake up early stay in bed, actually the advice given was once you’re up, get up, don’t snooze.
I clearly phrased this badly. I will be as precise as possible. Note that I've read about half of Why We Sleep, a year ago or so, before giving up because parts of it were such cult-like "I am going to write about how wonderful sleep is without actually paying any attention to the content of what I am writing"; I simply couldn't stand the nonsense evo-psych "explanations" every couple of pages. I'm aware that you don't get to be one of the top sleep scientists in the world without being capable of competent thought, but that book did not provide evidence of this fact. I also strongly believe that playing around with the amount of sleep you get each night is one of the highest-value experiments it's possible to do, because the cost of sleep is so enormously high and because its benefits are so high up to a point (and the graph of benefit against time varies from person to person).
The comment to which I replied, in part: "Do any of these errors reverse that information? Is sleep actually harmful?… If not, then what's the point [of arguing about whether Why We Sleep contains errors]?"
I interpret this comment as: "Why We Sleep may contain errors, but I don't care; sleep is obviously good for you. I know how much sleep is necessary for me, and am capable of ignoring whatever Why We Sleep says on the matter of how much is required."
My response is that we should care that Why We Sleep contains errors. People are going to read that book and use it to determine how much sleep they need, whether or not the parent commenter has done. The fact that Why We Sleep contains errors obviously doesn't mean we should get less sleep - the greatest idiot on Earth may say that it's raining, but that doesn't mean it's sunny outside - but it should make you at least a little sceptical of its arguments that you should get more sleep, because that sleep comes at a massive cost.
Suppose for the sake of argument it is the case that I require seven hours of sleep a night, and the additional eighth hour confers little additional benefit (or even confers harm - it doesn't matter to my argument). Suppose I habitually sleep seven hours, but then I read Why We Sleep and am terrified into sleeping eight hours a night. Now I am spending a full 1/24th of my life - about 4.1% of it - on extra sleep that is doing very little for me except robbing me of consciousness.
Perhaps it is the case that for you in particular, there is enough benefit from that eighth hour of sleep to outweigh the cost (and I assert that it's definitely worth everyone individually doing this little experiment on themselves, because people differ)! Perhaps it is the case that for most people, there is enough benefit - but we don't know, and we can't know simply from Why We Sleep, because it contains errors.
(The first part of the comment to which I replied was "The general message and key takeaway I got from the book was that sleep affects your health in various ways and getting a sufficient amount of higher quality sleep is better than not doing so." This is undoubtedly true: so obviously true, in fact, that there is no need for anyone to read a book to learn it.)
I explicitly predicated my answer on the assumption that the reader needs seven hours of sleep and not eight. Unless you mean "all else being equal, would you not rather spend another hour of your life per day unconscious?" - in which case no, I would not.
Wait, why is this trimming controversial? The original paper is a little mediocre (would be down for generic criticism of relying on it), but this trimming seems to match the main drive of it.
Just read the paper, and the association is pretty clear there: fewer hours of sleep means more injuries. P = ~0.006 rr=.8 per hour of additional sleep. That’s not a huge effect size, but I find that pretty believable from my lived experience. Lol, yes, p values, but does anyone think less sleep makes them more coordinated? I don’t allow myself to drive long distances on less than 6h sleep based on my prior experience having tried.
For a lay audience explaining the data for the left most point would be hard. Why is 5h of sleep less injurious than 6h? Could be random chance (only 160 students in the study). Could be that those players are worse (coach pulls sleepy kid off the field).
I doubt the most uncharitable interpretation of Walker’s trimming is correct (people who sleep 5h/night are better at sports and less likely to be injured).
As a data scientist, I feel like I have to deal with this shit at work all the time. The data matters and "we're data driven" unless the data disagrees with whomever you're talking to. If someone slices data in a shitty way, then you're being too pedantic when you point it out that they're implicitly making bad assumptions.
I wrote the comment below over a year ago, about how these popsci books are very unreliable sources of knowledge because of the perverse incentive structure (eg would this book have been written if the conclusion was 'we don't know how important sleep is, but it seems generally pretty important although this is difficult to prove and largely obvious'). It was downvoted pretty heavily.
I'm intrigued by the subject due to life long sleep issues. Mostly caused by bad habits. I enjoyed about 50% of the book and overall I appreciated that it got me some food for thought and back to working on my sleep quality. But other than that I had to say the book is really thin on material relative to its size. And the way the guy delved into subtopics like drugs and lucid dreaming left me quite skeptical about how deep he is into the field at all. That book could have been written by a student of medicine. There was nothing conveying any deeper understanding, interest or insight into sleep - no intuition.
That does not address one of the main points of UBI: eliminating the welfare cliff
Like someone is on welfare and the welfare includes free health care insurance. Then they get a job, they do not get welfare anymore, but then they need to pay for insurance, so with the job they have less money than before.
If this is the strongest criticism someone who I know is knowledgeable and intelligent can muster, it gives me more confidence in the conclusions of the book.
“Knowing” incomplete, false, distorted, or misleading facts is often much worse than having no information, or having never even considered a topic.
If I read fiction, it may imply some fundamental lens on the world, but I think my brain is better able to recognize it as an opinion, or subjective.
But if I internalize the “truth” of dietary principles or basic psychology, I will make real choices and have my perceptions shaped to believe I am seeing the results of these choices ... and it is often completely wrong.
This can be subtly or devastatingly negative for my success and quality of life.
I think this problem is under considered. In recent years I have started realizing that most of what I read literally makes me dumber.
It’s a thought that has more implications than I think most people want to consider. Especially people whose identity is connected to being someone who “knows” things.