The Seven Most Popular Posts from 2021

Updated: Jan 23

Welcome to #BeyondTheMean! 2021 was a challenging year for sure, but it also marked the first year of the #BeyondTheMean blog. This blog has given me the opportunity to connect with hundreds of educators across the country as we work together to improve teaching and learning conditions through the thoughtful application of data analysis and research. Here are the seven most popular posts from our first year together!


Welcome to #BeyondTheMean

I’m Matthew Courtney. I help schools help kids, and I want to show you how to harness the power of educational research and data analysis to drive continuous improvement in your school!

#BeyondTheMean is a weekly blog where I share tutorials, case studies, and research challenges to help get your mind thinking about continuous school improvement in a new way.

Continuous school improvement isn’t for “low performing” schools. It isn’t a theoretical policy framework cooked up by folks in Washington, D.C. It isn’t the enemy of good teachers and leaders.

Research and data driven continuous school improvement is for everyone. It is a focused mindset that ensures that everybody is bringing their best to every situation – every day. It is about intentionality; doing what’s best for kids not because we think it is what’s best, but because we know it is what’s best.

If you’re struggling with a big problem in your classroom, school, or district, I have GREAT news for you! You are not alone! Thousands of other educators across the nation and around the globe have experienced the same problem. By learning to leverage existing research tools and deeply understanding your local data, you can solve any problem that your school is facing. Read more.


Leading Effective Data Conversations

The phrase data-driven decision making has become ubiquitous in education. You find it on job descriptions, embedded in strategic plans, and copied into faculty meeting minutes. But do we, as an education profession, really have a good grip on what that phrase even means? I don’t think we do. In this post, I want to provide some guidance for the next time you gather your continuous school improvement team around the conference room table.

Develop a Data Analysis Approach

The first step to having a meaningful data discussion with your team is to develop a consistent data analysis approach. Consider the myriad of ways that you can manipulate data – there are dozens of methods. Your team should select an approach to deploy consistently throughout the year. This ensures that your meetings are efficient and everyone knows what they’re looking at and for when they open the data reports.

I am a big fan of open ended exploratory data analysis processes. I like that they are replicable, focused on simple mathematical principles, and are cyclical and iterative. Your school or district may have access to additional tools, such as standardized dashboards or reports that come out of your student information system, that may expedite your data review. Your team could also opt to develop their own data analysis protocols. The key is to agree on a process, train everyone on that process, and stick to it. Read more.



Five Tools to Boost your Improvement Planning

It can feel overwhelming when you have a big problem to tackle and no idea where to start. Believe me, I’ve been there. There are any number of gurus who will sell you “quick fixes” or condescendingly say that you simply need to improve core instruction and all of your problems will go away –as if its that easy. Sustainable school improvement requires careful planning and consistent evaluation. Without the tools to complete these two tasks, you will be working blind. You wouldn’t wander into a dark forest without a map, you shouldn’t wander into a continuous improvement meeting without one either. In this post, I want to share five of my favorite continuous improvement planning tools.

30-60-90 Day Planning

30-60-90 day planning is my go-to tool when I need to launch a new project. You know the phrase you eat an elephant one bite at a time? Well, 30-60-90 day planning is how you eat an elephant! The planning tasks are simple. Take a big project and decide what you can do in the next thirty days to work towards that goal. Then project out, what will you do in the next 60 days? 90 days? 120 days? Get the picture?

PDSA Cycles

PDSA stands for plan-do-study-act and is a cyclical planning process that is great when you are operating in maintenance mode. I know what you’re thinking: But Matthew, how can we have maintenance mode if we are focused on continuous improvement? With PDSA Cycles of course! Here’s how it goes. First, you think through your immediate improvement plans for whatever process you’re maintaining. Next, you implement your plan – documenting those implementation steps for future reference. During the study phase, review the effectiveness of your previous implementation. Finally, you act upon the outcomes of your review to determine which incremental changes need to be made next. Then, you do it all over again! Read more.


Data Disaggregation: The First Step towards Educational Equity

Educational equity is the idea that every student gets what they need in order to be successful in school and life. It forms the core of our educational system and is how educators ensure that each student shows appropriate growth throughout the school year regardless of their background, zip code, or personal circumstances. But how do educators know if they are meeting the needs of all students? With the average total school enrollment in the United States hovering around 550 students, it becomes necessary for educators to summarize student data to make informed decisions. Data disaggregation is one tool that educators can use to understand the performance of various groups of students.

Data disaggregation is a method of analyzing and presenting data that is divided into segments to show a greater level of detail. In education, it is common to disaggregate data by gender, race/ethnicity, ability, economic status, English fluency, course section, or schools or districts. This process helps educators better understand discrepancies in student performance and build plans to address inequities.

Data disaggregation became the norm in educational data analysis during the No Child Left Behind (NCLB) era. NCLB required states to report data disaggregated for six groups: race/ethnicity, economic disadvantage, disability, limited English proficiency, migrant status, and gender. Over time, educators and data analysts alike discovered new understandings and new benefits of disaggregating data more deeply. Read more.


Educational Data and the Law

When working with educational data, it is important that you are up-to-speed on the various legal requirements related to student’s privacy. Educational data is extremely sensitive information and its protection and proper use should be the first priority of any data analyst. In this post, I will provide an overview of some of the key student privacy laws. I am not a lawyer, this post should not be interpreted as legal advice.

Family Educational Rights and Privacy Act (FERPA)

The Family Educational Rights and Privacy Act (FERPA) is the primary law that governs the privacy of student educational records. Generally, an educational record is any part of the students record generated by their school, district, or state. The primary foundation of FERPA is that only the parent (which is defined as a natural parent, guardian, or an individual acting as a parent in the absence of a parent or a guardian) can access students private records.

There are a few exceptions to this rule as the records exist to support the functioning of the school itself. For example, school officials, such as teachers or administrators, have broad access to the records of students under their care but do not have access to student records for students in whom they do not have a “legitimate educational interest”. FERPA also allows a school to share student records with another school upon the enrollment of the student in their new school. Read more.



The Dangers of Over the Fence Decision Making

I want to talk about a problem that has plagued the education profession for generations – a trend that I like to call “Over-the-Fence Decision-Making” (shoo that’s a lot of hyphens).

Let’s start with a story. Assistant Principal Martin has just been named the new Principal at George Washington Middle School on the other side of the school district. She is excited for this new opportunity and can’t wait to meet her new kiddos. George Washington Middle School, or WashMid as the locals call it, is generally considered to be a good school. They aren’t the highest performing school in the region academically, but their proficiency rates are good, they have lots of extracurricular activities, and outstanding parent and community engagement. Shortly after arriving at WashMid, newly appointed Principal Martin is faced with a predicament – the newest TikTok challenge has swept through the student body leading to constant interruptions and long lasting distractions from learning. What is a new principal to do?!?

Having never been a middle school principal before, Principal Martin begins to search for help. She calls on her principal friends and mentors to ask their advice. She asks the district behavioral interventionist to review the school’s behavior policies. She attends a training hosted by a regional educational organization. In the end, she is still stuck; having received too much conflicting and out-of-context advice. One day while checking her Twitter feed she sees a blog post written by another principal discussing how they addressed this internet challenge in their school. Principal Martin, desperate for a solution, reads the post and implements the solution in her school without a second thought. Read more.


Creating a Healthy Culture for Research Use

Continuous improvement decisions should not be made in isolation. Education decision makers should engage a variety of stakeholders as they seek to make informed decisions to remedy long-lasting problems of practice. Unfortunately, the task of reviewing research for decision making is often left to a single leader who performs their review on the couch on a Saturday morning or in the bleachers at their kids soccer game. We can do better! Research use in continuous improvement must become intentional and systematic in our schools. It must become the way we make decisions. It must become part of the school culture.

Here are five steps you can take to start creating a healthy culture that promotes research use in your school.

Create an Expectation

The first step in building a culture that prioritizes research use for decision making is to create a clear expectation. Just like you would for any other new initiative in your school, the leaders in a building or system must clearly state a consistent expectation for research use in decision making. Once that expectation has been set, the leadership team must establish the expectation through their own practice and modeling. Remember, we should never ask anyone to do something that we ourselves are not willing to do. Read more.


Research Bias and Continuous Improvement

When working with research and data to drive continuous improvement in a school, education decision makers must be aware of the various types of bias that may be introduced into research. I’m not talking here about bias that may impact instruction or access, such as implicit bias, racial bias, or gender bias. I am talking about research biases that may impact the way a study was performed, the group of participants included in the study, or the analysis procedures applied to a body of data. While some degree of bias is unavoidable – no researcher or study is ever perfect – education decision makers must be on the lookout for cases of extreme bias and be prepared to consider the impact of that bias on their decision making processes. Let’s examine three categories of research bias.

Sampling and Selection Bias

Let’s begin our discussion by looking at sampling and selection bias. These two biases are related to how we select the data or groups of people that we include in our study. Sampling bias occurs when a researcher selects participants for a study and does not apply appropriate randomized controls. It also occurs when a study sample includes only a specific type of participant (inclusion bias) or doesn’t include an impacted population (omission bias); when it only inc