FAQ: Algorithmic Bias in Education

Updated: Jul 16

Welcome to #BeyondTheMean! Check out this post to see what this blog is all about.

In the wake of COVID-19, educators across the nation are turning to data analysis tools that use algorithms to prioritize students for services and predict student outcomes. While tools that are heavily studied and vetted can help teachers direct time and resources to students that need them the most, whenever we turn decision making processes over to algorithms, we introduce algorithmic bias into the mix. In this post, I want to answer some common questions about algorithmic bias and direct you to some helpful resources.

What is an algorithm?

In its simplest form, an algorithm is a set of rules that are used to solve a problem. People often think of algorithms as a new thing, but the earliest algorithms come from the ancient Middle East where the Babylonian people used a series of if-then math calculations to compute the time and place of important astronomical events. As time progressed, hand calculated algorithms would continue to be used to calculate a wide variety of tasks, including NASA’s initial trips to space!

In today’s data rich environment, algorithms run our world. They decide how much your plane ticket costs, which advertisements you see on your social media feed, and which shows you watch on your favorite streaming service. Most people would say that the rise of algorithmic decision making has improved their quality of life and it is virtually impossible to live your life without algorithmic influence.

How are algorithms used in education?

Algorithms are already commonplace in education. We commonly think of algorithms when we think of standardized testing. Algorithms are used to help schools and districts process test scores and identify students at risk of falling behind. Risk prediction algorithms can also look at non-academic data, such as attendance and behavioral data, to help identify which students need extra support.

Algorithms can also be found embedded within many modern standardized tests themselves. Computer adaptive tests, for example, run algorithms in the background of the test and update test question difficulty in real time as students get answers right or wrong. Many technological intervention tools use algorithms in a similar way – by assessing students along the way and selecting games and practice activities for students based on their performance.

What is algorithmic bias?

Many people think that computerized data analysis eliminates bias by taking the personal side out of the decision making process. Unfortunately, that isn’t always the case. Algorithmic bias occurs when repeated and systematic errors result in an unfair outcome. Remember that algorithms are written by human beings, and therefore can reflect the biases of those who develop them. Like most forms of bias, algorithmic bias is rarely intentional, but it can create real harm, nevertheless.

Where does algorithmic bias come from?

Algorithmic bias can be introduced into an algorithm in a few ways. First, algorithms can reflect the existing biases within an organization or society. This type of bias is often difficult to see – it is the kind of bias that exists because “that’s just how things are here”. This type of bias is common when algorithms are designed to replicate existing decision making processes because the results of the algorithm are designed to match the results of the manual process – and they are tested against prior results. The algorithm is deemed fit if the outcomes match outcomes from the past. In this instance, the bias is baked in from the beginning.

Another way that bias is introduced into an algorithm is by what is called its “training data”. When computers run algorithmic programs, developers “teach” the computer about the data using a pre-existing data set. The idea is that the computer will be able to recognize patterns in an existing data set and then be able to spot those patterns in future data. The problem is that if bias exists in the training data – such as a data set that does not include data reflective of minority populations – that bias will eventually be amplified as the computer systems run the algorithms in the future.

Finally, algorithmic bias can make its way into a system when an algorithm is being used in a new context for the first time. When there isn’t a history of using an algorithm for a given task, then there is less information available to the people monitoring the process to be able to tell if the algorithm is off-base or not.

What is an algorithmic “black box”?

Many times, algorithms are referred to as “black boxes”. This term reflects the mysterious nature of algorithms that are developed using machine learning techniques (this is when the developers “teach” the computer to find patterns based on an existing data set). The concern with black box algori