Welcome to #BeyondTheMean! Check out this post to see what this blog is all about.
Every year, educational leaders are faced with new challenges that require direct solutions. Effective continuous improvement systems begin with an annual needs assessment. When done correctly, this process exposes new problems that schools must grapple with. So – how does a school begin to address new issues effectively? The key to successfully implementing new strategies is to begin with research.
I have good news for you! There are over 100,000 schools in the United States. You are unlikely to be the first school to face your current issue. The education field has a rich research literature that examines problems of practice, tests solutions, and applies existing theories in new ways. You can, and should, tap into that existing literature when seeking to solve new problems in you school. If you have followed me for a while, you know that I am all about a system. Continuous improvement requires replicable systems for sustainability. Therefore, in this post, I want to present a replicable five step process for using research to make informed decisions.
Step One: Create a Team
No continuous improvement effort becomes sustainable when it performed by one guy in his office. Thoughtful and sustainable school improvement requires a team. Building a system for research use is no different. The school improvement leader should build a team dedicated solely to reviewing research and applying it to the problem at hand. This should not be your usual school improvement team. This is a good opportunity to practice distributed leadership and build a sub-committee of individuals dedicated to the task.
When building your research review team, ensure that you have a varied representation of skills. Your team should include individuals with one of five skill profiles.
The Organized One: This person has an eye for detail and an innate ability to bring order to chaos. As a member of your team, the skills of this individual will help you keep the research literature organized, see themes in the literature that other team members may have missed, and ensure that benchmark goals and deadlines are met.
The Nerdy One: Educational research is a broad bucket with a variety of methodologies and statistical analysis. Your research review team needs someone who loves data and can interpret the results of a variety of statistical tests. Their job is to ensure that everyone on your team is able to understand the data reported in each study and how that outcome relates to your problem of practice.
The Curious One: This is your friend who is always asking questions. Their job is to keep the group on their toes. This is a process that can feel arduous at times and it can be easy to simply accept the results of a few studies for the sake of brevity. The Curious One is there to ensure that the group truly gets to an answer.
The Snobby One: Our profession publishes thousands of studies every year each one varying in scope and quality. The job of The Snobby One is to make sure that the group is focused on the results of the highest quality research available and is not clinging to lower quality research simply because it aligns to their pre-existing beliefs.
The Articulate One: Last, but certainly not least, is The Articulate One. This is a person who is highly skilled at taking complicated information and synthesizing it into a form that can be easily understood by a large variety of stakeholder groups. Without a clearly articulated outcome, your research review journey will be fruitless.
Once you have your team together, you must provide them with dedicated time to do their work. No team can do effective work in the hallway between classes. Treat your research review team as the vital members of your school that they are. Their work will be fruitful, and your improvement will be expedited.
Step Two: Ask the Right Questions
When using research to answer difficult continuous improvement questions, it is important to take time to hone your questions so that you get meaningful answers. You should pose a question that is specific, measurable, and aligned to your needs assessment. Let’s consider an example. Imagine that you are working in a Title I school, and you want to boost your average daily attendance. So you ask the question: How do we boost attendance?
An initial search of the word “attendance” in the ERIC Database (which I will discuss next) returns more than 20,000 research articles. I don’t care how good your team is, they will never be able to digest 20,000 research articles and make a meaningful decision in the time allotted to them. They need to ask a better question.
Let’s consider: How do low-income schools boost attendance? A search of the term “attendance low-income” returns only around 1,200 results. A much more manageable number, but I think we can narrow this even more. How about: What interventions boost attendance in low-income schools? The phrase “attendance intervention low-income” returns only 180 results.
Asking the right question is about more than simply lowering the number of search results. It also helps your research team get to the right answer. To stay with our attendance example; let’s imagine that your school has received a grant that would allow you to invest in an attendance intervention. This would give you different options than a school that did not have funds for an intervention. If this variable is not in your question, your team may return with a solution that you cannot afford.
Step Three: Search the Literature
Once you have created your team and identified your research question, its time to get to work finding some actual research to read! This is where the rubber meets the road. There are three primary areas that your research team may search for relevant literature; databases, clearinghouse, and organizations.
Let’s start with databases. I always recommend that teams begin with the ERIC database. This is a FREE database supported by the US Department of Education. It is remarkably inclusive and gives you wide access to a variety of studies from around the world. To find research in a database, you must first identify a series of keywords related to your question. Databases work like a search engine, but they are not as intuitive as Google or Bing. You will need to try out several keyword searches to get the information you really need.
For the research question, what interventions boost attendance in low-income schools, I would run a number of searches including phrases like, attendance interventions, attendance interventions low income, attendance title i, increasing attendance, and increasing attendance low income. These key words will return some of the same articles (those are likely the articles that are most relevant to your search) as well as many different articles that may also help you. If your search returns too many articles, apply filters to narrow the scope by location, school type, or journal category.
As you find new articles, download them and save them to your computer. Don’t read as you go. This will slow you down and make you agitated. Get them all together, then send them out to your team. Tips on that to come…
If databases feel too grand for you, check out one of the many clearinghouses. Clearinghouses are like curated databases. The clearinghouse staff spends time reviewing, sorting, scoring, and summarizing research for you. This can save you a ton of time! One of the most popular clearinghouses is the What Works Clearinghouse (WWC). This is maintained by the US Department of Education and includes curated research lists for a variety of issues (including attendance).
While clearinghouse are more user friendly than databases, they do have their drawbacks. First, no clearinghouse is complete or universal. You will only have access to the interventions and strategies that the clearinghouse staff have had time to curate. This means that you may miss good opportunities simply because you didn’t know they were there. Another downside to clearinghouses is that you are rarely able to access the specific pieces of research. Clearinghouses generally provide summaries of research. These summaries are convenient, but they may be biased towards a certain world view or they may miss important details that would have helped you to make an informed decision.
Finally, specialized organizations can be a great source of research literature. Think tanks, nonprofits, and professional associations generally curate and disseminate literature on topics that are important to them. These types of groups are sometimes called “research brokers”. This will certainly be your fastest route to meaningful literature, but you should review what they send you with caution. These types of groups have an agenda – that is literally why they exist. The teacher’s association is likely to be biased towards research that is favorable to teachers. The nonprofit focused on reducing recidivism is likely to be biased towards research that is favorable to their own methods. The think tank that advocates for a certain political viewpoint is likely to be biased towards research that is favorable to their political view. This isn’t necessarily a bad thing – it helps these organizations accomplish their stated goals. However, reputable organizations will have measures in place to prevent or counter their own implicit bias and as a savvy consumer of research, you should ask hard questions and consider the research they provide you with a critical eye.
Step Four: Meaningfully Digest and Synthesize the Literature
Now that you have a body of literature ready to review, it is time to read, digest, and synthesize that literature. Have your team come together and assign readings to each team member. Depending on how many studies you wish to review, I always think it is best to have a couple of reviewers on each paper. This section is going to be short because there simply is no trick here. Steep some tea, get a cozy spot, and start reading.
While you read, I recommend that you have a handy place to take notes. Your notes should include a citation for the study you are reading, the data you read it, and any key takeaways from the study. I also like to include a list of other interesting studies discussed in the study I am reading so that I can go find those later. Finally, write down the key-point or finding and highlight it in a bold color so that you can find it quickly later.
If you are working with a large group, you may want to consider a shared document where everyone’s notes can be compiled in one spot. I take a more thoughtful look at this process in my eBook “Writing a Great Literature Review”. Since I gave it more through attention there, I will move on to the final step and save your eyes some trouble.
Step Five: Make a Decision and Communicate it Effectively
The time has come. You have built a team. They have scoured the internet for articles, spent countless hours reading and taking notes. They have highlighted key findings. Now it is time to make a decision. So how do you take the variety of findings of multiple studies and use them to inform your leadership decision?
First, you must go back to your initial question. As your team worked and reviewed meaningful literature, it is likely that they found new and interesting things. These shiny new ideas may have caused some drift from your question. Regroup your ideas around your initial question.
I am a very visual person, so I like to perform this task very literally. With my team gathered, I write the question down on a piece of chart paper. Then, I have the team write key findings or solutions on sticky notes. As a team, we review each sticky note and group it somewhere around our question. As you complete this task, the leading solutions will start to appear. The areas with many clustered sticky notes are where you want to focus your attention next.
Having narrowed down to two or three sticky note clusters, the team can now have a meaningful discussion about each of the three solutions. Review what the research said about each of them. What are the pros and cons of each solution? Then think beyond the research. Ask yourself some of these questions:
Do we have the staff capacity to implement this solution with fidelity?
Can we afford to implement this solution with fidelity?
How does this solution interact with other improvement efforts currently underway in our school?
How do we think our students will react to this strategy?
Does this strategy align to the mission, vision, and values of our organization?
This step is where the leadership part come in. You have to apply your localized, practitioner knowledge to the research body to decide which solution will be the best fit for your organization. Once you have answered these questions you should be able to narrow in on the right solution for you.
The final step in this process is to communicate your decision with your stakeholders. Since you have already done the research, you should share it widely and proudly. Consider how your stakeholders like to receive information and prepare a communication that meets their needs. Maybe it’s a sleek infographic that highlights the key findings from your research, your capacity restrictions, and the opportunities that a particular solution may provide. Perhaps your team prefers formal presentations or videos with slides that review the research in more detail. Maybe you are writing a grant application and need to craft a thoughtful narrative that includes research citations.
Whichever step you choose, be sure to include your research review in your communication processes. Your stakeholders will be grateful for the extra information and you will get major leadership street-cred when you can show them that you put the work in to create a thoughtful decision.
As you continue with your research driven decision making process, make sure you jump over to The Repository and check out some of the other resources I have on this topic. You will find video tutorials, ebooks, and auto-analysis tools that were all designed with educators in mind. I hope they will help you to make better continuous improvement decisions. Good luck on your journey friends, and let me know how I can help.