Welcome to #BeyondTheMean! Check out this post to see what this blog is all about.
This weekend, I binge watched Hulu’s new docudrama “Dopesick.” This eight part series portrays the events surrounding America’s opioid crisis and takes you into the lives of a handful of individuals who become addicted to OxyContin. The show deals with some heavy stuff and forces you to think a lot about medical ethics, the role pharmaceutical representatives play in the medical diagnosis process, and the impact of poverty and racism on society. Since I am the super nerd that I am, I couldn’t help but also notice a subtle, yet serious warning to about the importance of practitioner research fluency.
If you haven’t seen Dopesick yet, there are some spoilers in the coming paragraphs – feel free to come back later and finish this post.
Early in the series, viewers are introduced to the Porter-Jick study. This study eventually informs the essential sales narrative for OxyContin – the idea that this particular opioid was only addictive to roughly one percent of the population due to its time delay formula. The problem is, this study wasn’t actually a study at all. It was a five sentence letter to the editor who’s use to defend opioids would later be described as “mortifying” by its author.
The Porter-Jick “study” became one of the most infamous studies of the 1980s. At the time of this writing, the New England Journal of Medicine says that it has been cited 495 times! It was taught in medical schools, used by Purdue Pharma to successfully lobby the FDA for a special drug label, and repeated by well-intentioned doctors across the nation.
While it is easy to point fingers at Purdue Pharma and the Sackler family for questionable and predatory business practices, I cannot help but to stop and wonder – What would the opioid situation in the United States look like today if even just a quarter of the doctors prescribing OxyContin for long-term pain had actually looked up and read the Porter-Jick letter on their own?
The claims made by the Porter-Jick letter were manipulated into a mountain of marketing materials and talking points that were used to convince doctors to prescribe higher, longer term doses of the drug to their patients. Purdue Pharma would later manipulate their own data visualizations to further market their drugs. I am convinced that the lack of intellectual curiosity and access to meaningful research and accurate data interpretation skills significantly contributed to the opioid epidemic.
But Matthew – this is an education blog. What does the Porter-Jick letter have to do with educational decision making?
I’m glad you asked!
Education decision makers are in the same precarious situation today that pain doctors were facing in the 1980s. We are facing a seemingly insurmountable number of educational challenges posed by the COVID-19 pandemic and an unprecedented influx of cash to help address it. This influx of cash obviously comes with an influx of vendors sharing their wares and seeking that payday. I don’t blame them; it makes good business sense and schools across the world genuinely need the resources that vendors have to offer. Without careful consideration and a thoughtful review of the research, education decision makers are positioned to make similar mistakes as their counterparts in the medical field.
This wouldn’t be the first time that the education field has been swayed by shiny ideas that weren’t supported by thoughtful research. In the 1980’s, the Los Angeles Police Department and the Los Angeles Unified School District came together to develop a program to help address substance abuse concerns in their communities. This program, called the DARE program, brought local police and drug enforcement officers into the schools in a collaborative arrangement to help students learn to strategies to resist drug use and abuse.
This well intentioned program took off like a wildfire across the United States. I would comfortably bet that you are currently shaking your head with familiarization as you read this. Communities spent millions of dollars on signage, t-shirts, curriculum, field trips, and in a community near my house, a DARE branded Corvette.
The problem is, the DARE program really didn’t work. DARE’s own website discusses how researchers in the 1990’s found that there were minimal long-term effects of the program on student drug use. One study found a small but significant increase in drug use for suburban students who participated in the program. To DARE’s credit, they re-worked their curriculum in response to this research and today have a more reliable product on the market.
As an educator and a researcher, this reality hurts my heart. It is another instance where a lack of research literacy led to questionable decisions that cost a lot of money and may have had a long-term negative impact on students. To ensure that we can avoid these types of events in the future, education decision makers must commit to intentional research use and building their own research fluency.
So where to begin? Here are five things that education practitioners can do to make better informed decisions.
1. Ask vendors to show their receipts.
When you sit with vendors to talk through their new products, ask them to provide you the actual studies being cited in their marketing material. A reputable vendor should be able to provide you with hard copies of the work they are using to promote their program. Read those studies with a careful, critical eye and think about how they might reflect your unique situation.
2. Make reading research a part of your professional practice.
Research is written in its own – sometimes confusing – language. For good commentary on this, check out “Academia Obscura: The Hidden Silly Side of Higher Education.” It’s one of my favorites. Reading research is a skill just like anything else. My own professional practice requires me to apply research to various policy making situations. I like to periodically visit the ERIC database and search for recent literature on whatever topic I am studying that week (I also post interesting articles to my Twitter using the hashtag #whatimreading). This practice helps me stay up-to-date on new thinking and also keeps my research reading skills sharp.
3. Engage in action research.
Action research is a style of research that is performed by practitioners for the purpose of driving continuous improvement efforts. Its focus is on helping you understand the nuances of your unique situation so that you can make better decisions. It is also a great way to engage your team and boost research adoption across your school or district.
4. Overcome your statistical shyness.
When I work with education leaders on data analysis, they are always quick to tell me that they struggled with statistics in high school, college, or grad school. Guess what? So did I! I had to take my doctoral statistics course twice (and I only got a B the second time despite having completed all of the assignments once already!). Statistical training isn’t a huge part of educator training – so we have to overcome that with self-study, online courses, and mentorships. Embrace the productive disequilibrium you may feel and dig into some statistical study. You will be glad you did.
5. Create a culture of research use.
Research use isn’t super effective in isolation. If you want boost your research fluency, bring some friends along for the ride! Make research use a clear expectation of your team. One way that I do this is with the “Who’s research?” rule. Whenever somebody knowingly says “research says…” in a meeting, stop and ask, “Who’s research says that?”. Then, pause and find the receipts. You will be shocked at how often you think research says something that it actually doesn’t. By holding yourself and your teammates accountable to research use, your collective efficacy will rise and you will make better decisions.
To wrap up what has become a rather long-winded post – practitioner research fluency in education is of vital importance. We must all make a more concerted effort to learn from the mistakes of the past to ensure that our children have a brighter future. If you’re ready to dig deeper into the world of educational research use and data analysis for decision making, give me a ring! Also, check out The Repository. This is a living pool of resources that I regularly update to try to help my fellow educators boost their skills.
Good luck on your journey friends!