“Harvard study says” isn’t good enough
Good Morning! Let’s continue talking about critical thinking! This is part of my Critical Thinking series of essays – you can find the link to 1 – 15 in the comments! We just finished talking about 30 logical fallacies – and it took what seemed like forever! So, what’s next?
Essay 16: Who Should You Actually Trust?
I’ve seen people cite “a Harvard study” many times without providing any other information. Just “Harvard study says.” As if the word Harvard settles it.
That’s not how credibility works.
The study might exist. It might be from Harvard. It might even say what they claim. But who funded it? What was the sample size? Was it peer-reviewed? Has it been replicated? What’s the methodology? Is the person citing it accurately representing what it actually found?
“Harvard” doesn’t answer any of those questions. It’s just a name being used to shut down scrutiny.
There are better tools for evaluating sources than “sounds prestigious” or “confirms what I already think.” Because both of those will lead you wrong constantly.
A good place to start is with conflicts of interest. Not to automatically dismiss sources who have them, but to know they exist. A study about sugar funded by Coca-Cola might be methodologically sound. It also might be asking questions designed to get specific answers, or emphasizing findings that make sugar look better, or downplaying risks.
Knowing who paid for research doesn’t mean the research is automatically wrong. It means you should look harder at the methodology and check whether independent research reaches similar conclusions.
Think tanks do this constantly. Someone cites “economic research shows tax cuts boost growth” from a conservative think tank funded by billionaires who benefit from tax cuts. That doesn’t make the research false. It means you should be aware of the incentive structure and look for research from sources with different funding.
Same thing works in reverse. Progressive think tanks funded by labor unions produce research supporting union-friendly policies. Not automatically wrong. But you should know the incentive structure exists.
Expertise matters, but it’s not simple. Someone with a PhD in virology is a better source on viruses than someone without one. But expertise in one field doesn’t transfer to others. A Nobel Prize in physics doesn’t make you an expert on education policy. A successful business career doesn’t make you an expert on climate science.
Watch for people who are experts in something being treated as experts in everything. Elon Musk built electric cars and rockets. That doesn’t make him an expert on public transit, AI safety, or whether remote work is effective. But he gets cited as an authority on all of them because he’s successful in his actual field.
Credentials matter, but they’re not the only thing that matters. Someone can have impressive credentials and still be wrong, or lying, or paid to say something specific, or operating outside their area of expertise.
The question isn’t “do they have credentials?” The question is “do they have relevant expertise, what are their incentives, and what does the broader evidence show?”
I try to look for consensus among experts in the relevant field. Not because experts are always right – they’re not – but because when the overwhelming majority of experts agree after years of research and evidence gathering, they’re probably on to something.
Climate change. Evolution. Vaccine safety. The expert consensus is overwhelming. That doesn’t mean every single claim is correct or that science won’t evolve. It means the basic conclusions are well-supported.
When someone says “I found one scientist who disagrees,” ask yourself: what does the broader scientific community say? Because you can always find one doctor who says vaccines cause autism. The question is what the weight of evidence shows.
Also, original sources matter. Someone citing a news article citing a study is less reliable than someone citing the actual study. News articles simplify, sometimes distort, sometimes get it wrong. Go to the source when you can.
I often get bombarded with requests to cite my sources – but I will NEVER put a URL link in a caption – it’s terrible for your reach. Facebook hates sending people to other websites. I used to include sources in the comments, but people ask for sources without checking first, and I don’t have time to respond to every comment. I DO always put sources on my website – and all of my essays can also be found there. But the truth is – most people won’t take the time to find them there.
The truth is – you shouldn’t trust me anyway – who the hell am I? Certainly no expert on anything. If I post something you want more info on – you should go look it up for yourself.
So, when someone makes a factual claim, ask yourself: can I find this in a primary source? Can I verify it independently? Do multiple credible sources confirm it?
I’m not saying become a research expert on every topic. You don’t have time for that. I’m saying develop a bullshit detector. Know the red flags. Know when to dig deeper and when you can reasonably trust something.
Trust isn’t binary. You can tentatively accept something while being open to better evidence. You can trust experts in their field while recognizing they’re human and can be wrong. You can acknowledge uncertainty while still making reasoned judgments based on best available evidence.
The goal isn’t perfect certainty. The goal is to evaluate sources well enough to make informed decisions.