Welcome to #BeyondTheMean! Check out this post to see what this blog is all about.
The phrase data-driven decision making has become ubiquitous in education. You find it on job descriptions, embedded in strategic plans, and copied into faculty meeting minutes. But do we, as an education profession, really have a good grip on what that phrase even means? I don’t think we do. In this post, I want to provide some guidance for the next time you gather your continuous school improvement team around the conference room table.
Develop a Data Analysis Approach
The first step to having a meaningful data discussion with your team is to develop a consistent data analysis approach. Consider the myriad of ways that you can manipulate data – there are dozens of methods. Your team should select an approach to deploy consistently throughout the year. This ensures that your meetings are efficient and everyone knows what they’re looking at and for when they open the data reports.
I am a big fan of open ended exploratory data analysis processes. I like that they are replicable, focused on simple mathematical principles, and are cyclical and iterative. Your school or district may have access to additional tools, such as standardized dashboards or reports that come out of your student information system, that may expedite your data review. Your team could also opt to develop their own data analysis protocols. The key is to agree on a process, train everyone on that process, and stick to it.
Review, Ask Questions, Review Again
Once you have agreed upon a data analysis protocol, you need to develop a routine or rhythm for your data meetings. I recommend that you start with a simple review of the data. Throw your dashboard, report, or other analysis tool up on a big screen and take some time to walk through it from top to bottom. As you do so, allow team members to document their questions along the way. When you have completed your initial review, open the floor for team members to pose their questions and facilitate some discussion.
It is important to make sure that you complete your full review of your analysis before opening the floor for discussion. This ensures that every data point in your agreed upon approach is highlighted and your limited time together doesn’t get eaten up by the first two findings while leaving the next eight on the screen for later (later will never come).
As your team discusses the data, allow the data to provide answers to questions. Turn back to the data and apply your data skills to dig deeper and better understand nuances or abnormalities.
If you get stuck…
It is likely that your team will get stuck early on in the process as you build both routines and confidence. You should have a back-up protocol in place for those days when thoughtful questions from the data simply aren’t flowing. I like the to start with the six data questions by Edie Halcomb:
1. What question/s are we trying to answer with the data?
2. What does the data tell us?
3. What does the data not tell us?
4. What are the causes to celebrate?
5. What is the need for improvement?
6. What are our next steps?
Have these questions printed up on a poster and hung in your conference room for easy reference. They will get you unstuck every time.
Look for Different Angles
One common issue in data team meetings is that we tend to look for the things we are interested in. We frequently ask questions like:
Are our minority student populations performing at the same rate as our majority student populations?
Are our different grade levels reporting roughly the same attendance rates?
Are boys and girls receiving roughly the same number of behavior referrals?
These equity questions are important, but sometimes they make us to miss other important anomalies in our data set. When we only seek answers to questions that interest us, we set ourselves up to miss important things that we may not have thought to ask about. Consider opening your lens up and broadening the questions that you ask. Instead of asking about achievement between two populations, look at the achievement among all populations and across various measures of intersectionality. Identify anomalies and focus your discussion around them. This will lead to richer conversation and better decisions.
Keep Good Records
It is important that your data team keeps good records of your meetings. Detailed meeting minutes and archived analytic reports will be useful to you later on when you want to see how a variable has changed over time or for those times when you can’t quite remember why you made a particular decision.
I recommend that you develop both a standardized meeting minute agenda and a standardized analysis archival instrument. Just as you sought to create a standardized analysis protocol, you should standardize these record keeping instruments as well. This will ensure that everyone on your team knows how to keep the records and refer to them later. These types of system level protocols also ensure that the good work you’re doing in your school continue even when leadership or teaching staff changes over time.
Make Good Decisions as a Team
Finally, commit to never leaving a meeting without making a decision. Ironically, when I work with schools on data-driven decision making it’s the decision making part that they often leave out. They get so deep into the data conversations that they either confuse themselves and give up in frustration or get so passionately involved that the run out of time. By committing to always making a decision – any decision – at the end of your data meetings, you ensure that your meetings maintain a focus on continuous improvement and don’t devolve into a show and tell session. Maybe your decision is “we need to come back together next week.” That is a-okay. Eventually, as you become more comfortable and your conversations become more routine, your decisions will become richer and more powerful.
As you seek to expand data use for decision making among your school improvement team, I hope you will consider some of these tips. A focus on standardization and replicability is key for long term success. More importantly, a commitment to doing the work and doing it well will serve you and your students in the long term. Good luck on your journey friends and let me know how I can help.