Technical Directions

Outcome Forecasting Tool

Every year, educators across the nation set goals for their students. By looking at trends in historical data, educators can set realistic goals for the future. I designed the Outcome Forecasting Tool (OFT) to help educators apply statistical forecasting methods to their data and set more accurate goals. With just a few clicks, the OFT will provide predictions and visualizations using four forecasting methods; simple exponential smoothing, Holt linear exponential smoothing, naïve random walk, and cubic spline smoothing. This article will provide detailed technical directions for using the tool.

About the Tool

The OFT is a Shiny Web Application developed by Matthew Courtney in 2020. It uses the R statistical programming language to read data off of a spreadsheet and create a summary of any column. The OFT is hosted on the shinyapps.io server.

Preparing your Data

Attention should be paid to the proper preparation of your data before uploading it to the OFT. The OFT will return accurate calculations and visualizations for whatever data you upload, but it cannot account for mistakes in your original data worksheet. If you are pulling your data from a standardized gradebook or testing system, it is likely ready to go with very little preparation.

Your spreadsheet should contain one column of data. Copy and paste your historical data into Column A of a fresh spreadsheet. Your data should be in ascending order, meaning the oldest data point is first and the most recent data point is last. The OFT will examine trends and make predictions based on this history.

Remember that you should NEVER upload personally identifiable information for yourself or your students to the internet. While the information uploaded into the OFT is not permanently stored, personally identifiable data is always vulnerable to cyber-attack. Do not upload personally identifiable data for either yourself or your students to the OFT.

When your data is clean and ready, save the file as a .CSV file. CSV stands for comma separated values. This is a common file format for transferring large amounts of data quickly and efficiently. The OFT will only read a .CSV file.

Using the OFT

Using the OFT to analyze your data is simple. First, upload your .CSV file by selecting the “Browse” button in the grey box. This will open a window that will allow you to find the file. Select the file and click “Open”.

The OFT will automatically upload your spreadsheet. This process is normally pretty quick, but the time it takes to upload will vary greatly depending on the size of your file. An upload progress bar will light up under the browse box.

When the OFT has completed its upload, it will automatically generate an analysis. Each of the four statistical models are displayed using tabs at the top. You can change the display by clicking the model you want to view.  For each model, the tool generates a table that includes the next four predicted values and the upper and lower confidence interval at the 80% and 95% levels. Your next score is likely to fall somewhere within this window. The tool also creates a line graph to show your data over time. The black line represents your historical data, the blue dots represent your predicted values, and the blue and grey areas represent the confidence intervals.

All of the information presented by the OFT is static – meaning that you cannot change or customize it. You can, however, copy and paste the information into a document or slideshow presentation to easily share the result with your colleagues. You can also save the graphs by right clicking on the graph and selecting “Save Image As” from the menu.

Interpreting the Results

The OFT will return predicted values using four statistical prediction methods. You will find that the outcomes for each method are slightly different due to the way they handle the historical data. The four methods are described below.

  • Simple Exponential Smoothing: SES methods use exponentially decreasing weights to analyze time series data without a trend.

  • Holt Linear Exponential Smoothing: The Holt method uses exponentially decreasing weights to analyze time series data with a trend.

  • Naive and Random Walk Forecasting: A forecasting method that uses limited manipulation to predict future outcomes.

  • Cubic Spline Forecasting: A forecasting method that uses cubic smoothing splines to predict future outcomes.

The accuracy of each model will vary depending on your data. Generally, the more data you can feed into the model the more accurate the model will be. When using the model for goal setting purposes, education leaders should consider each of the models and their confidence intervals to make an informed decision.