Intervention Analysis Tool
Teachers design interventions for students every day. Whether it is an instructional intervention designed to increase academic performance or a behavioral intervention designed to lower unpleasant incidents in the classroom, accurate data analysis is a vital element for measuring the impact of the intervention. I designed the Intervention Analysis Tool (IAT) to be an incredibly easy and efficient way for teachers to gain better insight into the impact of single-student interventions. With just a few clicks, the IAT will prepare a report that you can use to jump-start your data conversations. This article will provide detailed technical directions for using the tool.
About the Tool
The IAT is a Shiny Web Application developed by Matthew Courtney in 2020. It uses the R statistical programming language to read data off of a spreadsheet and create a summary of any column. The IAT is hosted on the shinyapps.io server.
Preparing your Data
Attention should be paid to the proper preparation of your data before uploading it to the IAT. The IAT will return accurate calculations and visualizations for whatever data you upload, but it cannot account for mistakes in your original data worksheet. If you are pulling your data from a standardized gradebook or testing system, it is likely ready to go with very little preparation.
To ensure that all of the auto-generated graphs and statistics are accurate, you should upload a spreadsheet that contains two columns of data where each row represents a unique student observation.
The first column must be titled “Phase” and should include a code for each observation indicating which phase of your study in which the observation was made. If you are using an AB study design, you should title your baseline phase “A” and your intervention phase “B”. If you are using an ABA or ABAB model, your baseline phase should be titled “A1”, your first intervention phase should be titled “B1”, your reversal phase should be titled “A2”, and your second intervention period (if using an ABAB design), should be called “B2”.
The second column must be titled “Score” and should include the observed score for each observation. It doesn’t matter what activity you are observing, whether it is an assessment score or a behavior activity, as long as you use a numerical variable to record it. You should ensure that each score in your column is formatted the same. For example, if your dependent variable is a test score, you want to make sure that each test score is either a whole number, like 89, or a decimal point, like 0.89. The IAT cannot tell the difference between these variable formats and this will cause you to have incorrect outcomes.
Remember that you should NEVER upload personally identifiable information for yourself or your students to the internet. While the information uploaded into the IAT is not permanently stored, personally identifiable data is always vulnerable to cyber-attack. Do not upload personally identifiable data for either yourself or your students to the IAT.
When your data is clean and ready, save the file as a .CSV file. CSV stands for comma separated values. This is a common file format for transferring large amounts of data quickly and efficiently. The IAT will only read a .CSV file.
Using the IAT
Using the IAT to analyze your data is simple. First, upload your .CSV file by selecting the “Browse” button in the grey box. This will open a window that will allow you to find the file. Select the file and click “Open”.
The IAT will automatically upload your spreadsheet. This process is normally pretty quick, but the time it takes to upload will vary greatly depending on the size of your file. An upload progress bar will light up under the browse box.
When the IAT has completed its upload, it will automatically display the summary information of your spreadsheet in the white space. You should select the appropriate study design (AB, ABA, or ABAB) from the tabs at the top to ensure that you are viewing the appropriate outputs.
All of the information presented by the IAT is static – meaning that you cannot change or customize it. You can, however, copy and paste the information into a document or slideshow presentation to easily share the result with your colleagues. You can also save the graphs by right clicking on the graph and selecting “Save Image As” from the menu.
Interpreting the Results
The IAT will return two summary statistics and three visualizations for your data set. While the IAT will quickly and accurately summarize your student data, it will not tell you what that data means. It is up to you to apply local context and your own background information about your students to derive meaning from the data. The IAT will present the following outputs:
Mean – The mean is the average of your distribution. It is a measure of central tendency that allows you to summarize a distribution.
Effect Size – The IAT uses the Cohen’s d formula to calculate effect size. Cohen’s d is a measure of effect size that tells you how far apart the scores from your two study phases are.
Line Plot – The IAT will generate a line plot to show how your data changes from phase to phase. It uses the standard plotting for single case design visualizations.
Plot with Mean Line – The second plot is the line plot with an overlaid line showing the mean, or average, for that phase. This is the same value as is reported above.
Line Plot with LSR Trend Line – The IAT will generate a third plot that includes a scatter plot of your student data overlaid with an LSR trend line. LSR stands for least squares regression. The LSR trend line visualizes a regression analysis and helps you see your scores with limited variance.