(Still) Not a PubMed Warrior

More than a decade ago, I posted Not a PubMed Warrior. I outlined several reasons why I didn’t use PubMed as a tool for this blog.

To start, I am not qualified to read a nutritional study. I’ve never received training, nor do I care enough to pursue that knowledge.

I also referenced an article stating that 90% of landmark cancer studies could not be replicated.

I still feel the same as I did back then. Only now it should be more obvious to others. The last few years have shown us that very few people are qualified to not only read a study but know the quality of the study. They would also need to know the landscape of similar studies in that domain. Is the study confirming a body of knowledge or contradicting it? Who financed the study? This is hard stuff. In order to become qualified or decent at this skill requires both a high intellect and lots of practice.

We recently had a study on time-restricted feeding that received a lot of press. The paid science writers for different news outlets, whose job it is to read and report on health studies, each read the study differently and came to different conclusions.

Keep Researching Until…

On a podcast recently, the host said something so wise. People keep researching until they find a study that agrees with their opinion. You see it everywhere. Are you vegan? Carnivore? Love or hate masks? You can find a study that supports your position. Once you find what you went looking for, you can stop researching and feel great that you were right.

Will you know if it is a good study? Will you know the landscape of other studies on the topic? Almost certainly not.

Remember the Kale Incident?

This happened a decade ago. Health blogger Melissa McEwen of the now offline blog HuntGatherLove did an epic nutritional prank. Months prior to the prank, she mentioned she would use nutritional studies in a misleading way to demonize a food commonly considered to be healthy.

I still remember when Just Kale Me: How your Kale habit is slowly destroying your health and the world was published. Everyone that didn’t like kale was quick to share the link – including some smart people with massive numbers of followers in the Paleo community.

Later the “Just kidding” section was added after it blew up in popularity.

…people took it very very seriously, I guess they forgot I had put up a poll some time ago asking what food should be my victim to demonstrate you can demonize anything with Pubmed.

Kale by kiona

Directionally Accurate

Years ago, I began thinking about nutritional truths as being more or less directionally accurate. From my 2017 post Directionally Accurate and Metabolic Unicorns:

One of the guiding principles I have used for many years is paying attention to what is becoming more or less popular as a way to measure how true it might be. Health and fitness trends come and go. The best ideas will stick around longer. The worst ones will be forgotten.

An optimal strategy would be to avoid the new fads until they have been around for a while. Give the critics time to respond. Let more and more people go before you. If more people are getting great results then they will tell more people. Keep watching. Solid ideas will stand up to criticism. And the marketplace of individuals will, in the long run, figure out what is directionally accurate. This won’t always be the truth, but close enough in many cases.

In a recent post, The Seed Oil Debate Revisited, I outlined how 2 of the 4 reasons to avoid seed oils have become less directionally accurate. They could still be true, but the direction of evidence is not mounting.

Collect Smart People and Discard Team Players

If I can’t trust myself or the health writers, who can I trust? Doctors know medicine but might be ignorant of the statistical skills needed to know the quality of a study. They might have other biases as well.

I collect smart people that can explain studies that are not on a team. This means that although I could learn from smart vegans or paleo people, I know they get their income from representing their customers, who want to hear the team’s message.

I hesitate to name names, as my list will likely change over time.

It can be a challenge to hear a persuasive voice that says the things we want to hear and then seek out the counterargument. But I’ve been fooled by so many hucksters in the nutritional space, I need to have my guard up.

I can still be fooled, but I like to think that the period of time that I am fooled has been greatly reduced.