Welcome to #BeyondTheMean! Check out this post to see what this blog is all about.
When it comes to applying research to continuous improvement tasks, education decision makers should take time to understand the strengths and weaknesses of the study designs deployed by the researchers. In this post, I will hit the highlights of the five most common categories of study design.
Randomized Controlled Trial
Randomized controlled trials, or RCTs, are experiments in which the research breaks the study participants into two distinct groups. The first group, the control group, goes about life as normal. The second group, the intervention group, receives some kind of intervention. At the end of the study, the researcher deploys inferential statistical methods to compare the performance of the two groups and measures the impact of the intervention.
RCTs are often considered to be the gold standard in research design. The randomization of participants helps to filter out confounding factors – or elements that may influence the outcome of the study. This type of study design lends itself to the examination of educational interventions, teaching strategies, or the deployment of programs because it allows you to clearly and accurately compare the performance of two groups. It can be challenging to perform a rigorous RCT in education because they are very expensive, often take a long time, and are difficult to monitor as the population of a school or classroom is ever changing.
Quasi-Experimental Design
Quasi-experimental designs are similar to RCTs in that they compare the performance of two groups – a control group and an intervention group. The difference here is that the groups are previously assigned groups, such as schools or classrooms. For example, a researcher may deploy an intervention in Mr. Campbell and Ms. Stein’s fifth grade classes but not in Ms. Foster or Mr. Napier’s classes. At the end of the study, inferential statistics are deployed to measure the impact of the intervention.
Quasi-experimental studies are common in education because researchers have quick and ready access to previously assigned groups of students. However, this type of study is less rigorous than an RCT due to the prevalence of confounding factors. A lot goes into why Johnny is assigned to Mr. Napier’s or Ms. Stein’s classroom. Students are rarely randomly assigned to their teacher. So, by using these previously assigned groups to study an intervention, the researcher is unable to account for the fact that Mr. Napier’s class has all of the students with individual education plans because he is dual certified to teach both fifth grade and special education.
Correlational Study Design
Sometimes of activities cannot be studied effectively using RCT or quasi-experimental groups. In those instances, correlational study designs may be appropriate. A correlational study design is when we look to determine if a relationship exists between an activity and an outcome. This relationship, called a correlation, can tell us the strength of a relationship but cannot tell us if one thing caused another thing. This makes correlational studies less rigorous when trying to determine the effectiveness of an intervention.
Correlational studies are handy for studies that include surveys or archival data. Any time the researcher needs to make an inference about a population, a correlational study can provide a useful framework. Correlational studies are also useful in the planning stages of larger, longitudinal studies that use an RCT or quasi-experimental design. If a correlation does not exist, it is very likely that an RCT or quasi-experimental study will find a similar result. By completing a cheaper, faster correlational study, a researcher can save time and money for more rigorous studies later.
Single Case Design
Single-case designs are commonly deployed in schools in the special-education setting. In a single-case study design, the researcher documents a phenomenon as it relates to a single observation (a student in most cases). Single-case designs usually deploy an ABA methodology, in which a baseline set of data is collected (A), then an intervention is deployed and more data is collected (B), then the intervention is removed and a third set of data is collected (A). These phases are sometimes called baseline, intervention, and reversal phases.
This type of study helps the researcher to understand how an intervention impacts the outcomes on a single pupil. The analysis can help measure the impact of an intervention and determine if the intervention has a long term or short term impact. When multiple single-case studies are performed using the same intervention, they begin to form a body of research that can help future educators choose effective interventions for their individual students.
Case Study Design
A case study is designed to share a story or document an experience. They are used when a researcher wants to share a phenomenon with the world so that it can be studied in more detail later on. Case studies focus on the experience of one unit – a person, small group, or school – and how a given phenomenon shaped their views of an issue. They provide valuable insight and are a good starting point for more rigorous research review.
Case studies are also frequently used by businesses as marketing materials designed to share the success of their product. Education decision makers should examine case studies to determine if they were conducted using sound methodology or were created solely as marketing materials. Either way, it is important for decision makers to remember that case studies only document the experience of one person or group of people and that other people may have had a different experience.
Wrapping Up
Next time you dive into the research literature to try to solve a continuous improvement problem in your school, take some time to consider the study design deployed by the researcher. What are the pros and cons of that study design? How will you allow that study design to influence your decision? When you’re ready to start analyzing some data of your own, check out the free auto-analysis tools available to you in The Repository. They have been built with these various study designs in mind in hopes that they will help you jumpstart your action research and continuous improvement monitoring processes. Good luck on your journey friends and let me know how I can help.