Welcome to #BeyondTheMean! Check out this post to see what this blog is all about.
In the field of education, we are obsessed with the analysis and reporting of outcome data. On its surface, this makes sense. We want to ensure that every student in every classroom is reaching their fullest potential. Unfortunately, conversations centering on educational data almost always focus on a quantitative analysis of standardized test scores. While these scores are important, classrooms are extremely data rich environments and both classroom teachers and education leaders would benefit from adopting a broader definition of the word “data” that includes qualitative metrics.
Qualitative data are observational data that are used to describe an event, activity, or outcome – usually in words. In this post, I want to explore five types of qualitative data that is readily available to educators and discuss how they might be used to inform both instruction and policy.
Observations
Whenever you have a large group of people together in a single space, you have an environment ripe for observation. Observations occur when you watch an event or activity and document what is happening through your own eyes and in your own words. Observation is a powerful tool for classroom educators seeking to understand the experience of their students. Let’s say that an elementary school teacher is challenged by classroom management during rotational center activities. This teacher may benefit from stepping out of the center activity to observe and take notes as the students engage and rotate. By thoughtfully observing the process over a number of classes or groups, and taking thoughtful notes, the teacher may be able to design better procedures.
Non-evaluative teacher observations are also a great tool for education leaders seeking to improve school systems. Principals and instructional coaches should spend a large portion of their day in classrooms. As they do so, they should be taking notes of the things they see that are great and the things they see that may need improving. Once a month, these leaders can sit down with their notebook and reflect on their observations to make a data-informed decision about systematic changes, professional learning opportunities, or awards and recognition.
Journal Entries
Journal entries are a great way to deeply understand the experience of an individual over time. Teachers can have students journal their thoughts and feelings as they reflect at the end of the school day. Principals can have teachers journal after a faculty meeting or professional learning session, or as they implement a new technique in their classrooms. These journal entries can be collected and turned into incredibly valuable data.
Analyzing journal entries takes time and requires the deployment of qualitative coding techniques. Coding is when a researcher systematically applies themes to a body of text to identify themes. Once you have identified themes, you have turned your narrative entries into meaningful information that can be acted upon.
Document Analysis
In education, we love a paper trail. From student assignments to lesson plans, improvement plans to grant applications, educators are always producing documents. These documents provide a rich source of qualitative data that can help you understand your organization in a myriad of ways.
Teachers can benefit from exploring their student work through a rigorous document analysis process. This makes a great professional learning community (PLC) activity! The process is simple: First, the teachers involved must agree on a protocol for reviewing the documents. Perhaps they want to see if students have demonstrated a specific skill or achieved an identified standard, or maybe they want to scan for the use of key vocabulary. Regardless of the goal, a shared protocol is vital for ensuring that document review leads to the creation of actionable data. Next, each teacher should implement their protocol before finally coming together to review the results of your document analysis.
Building and system level leaders can also benefit from implementing a document analysis protocol. In the same manner I described above, district leaders can systematically analyze lesson plans to understand the interpretation of a district level initiative or review school improvement plans to see what supports schools need to achieve greatness.
Personal Interviews
Interviews are a great way to get feedback about a process or experience. Interviews consist of short conversations with individuals. They are relatively quick and easy to pull together, but they require a great deal of trust which makes them sometimes difficult to perform within a managerial structure.
There are three general categories of interview:
Structured Interview – during a structured interview, the interviewer works from a pre-set list of questions and does not wander off of them. Think of this like a survey that you say out loud.
Semi-structured Interview – during a semi-structured interview, the interviewer has a list of pre-set questions to guide the conversation, but also has the leeway to take questions out of order or ask follow up questions to better understand the topic.
Unstructured Interview – during an unstructured interview, the interviewer established a topic before engaging in a free flowing conversation.
Regardless of which method you choose, note-taking is very important. I recommend making a voice recording that can be later transcribed word-for-word to ensure accuracy. Once you have your transcripts in hand, you can begin to identify themes from the rich data source you have created.
Focus Groups
Focus groups are like big group interviews. During a focus group, the interviewer sits with a group of individuals and seeks their thoughts, opinions, and reactions to a scenario. Focus groups are powerful tools during the continuous improvement planning process as they allow leaders to hear about the user experience in their school or system. Focus groups are also awesome tools to deploy when you are considering a change in policy. Bringing together a group of teachers to think through the implications of a procedural change will never disappoint.
When performing focus groups, it is important that you don’t introduce acquiescence bias into the mix. Acquiescence bias is when a leader signals a desired outcome from a conversation, after which the participant acquiesces and proves the answer they think the leader is looking for. This is especially common in situations with a strong power dynamic or where a new leader is put in place and the participants may wish to seek their favor. It is important to stay neutral during focus groups. Your job is to listen and collect data.
I hope that this post has perked your interest in expanding your definition of the word “data”. Remember, qualitative data is data. It is processed differently and requires a little more effort to interpret, but it can provide rich insights into your classrooms, schools, and systems. Before you head out, make sure you check out the new resources in The Repository; my free, members only webpage that houses data analysis tools, tutorial videos, and downloadable eBooks. Good luck on your journey friends!