Your Predictive Analytics Are Only as Good as Your Data Literacy

When Donald Trump won the 2016 election, many Americans were shaken. How was this possible if he only had a 30% chance of winning? It seemed like Hilary Clinton was guaranteed to win.

But if you are data literate, you would know this is not true.

Where the 2016 election results and predictive analytics in higher education are similar is that a 30% chance of winning does not mean that you will not win, and a 30% chance of graduating does not mean that you will not graduate. It simply means that you have a 30% chance of graduating based on certain factors.

Like the 2016 election predictions, predictive analytic system findings are just that: predictions. It is essential that the field understand that while a predictive system can pretty accurately predict outcomes, it is by no means a crystal ball. How these findings are interpreted and communicated by end-users is essential to ensuring that predictive systems reach their potential in increasing student success in higher education. Otherwise, even the best algorithm will not help more students graduate.

Here’s how poor data literacy can make that happen: a college counselor sees that their early alert system predicts a student has a 30% chance of graduating if they stay in their major after failing a prerequisite. If not data-literate, this counselor can tell the student that they won’t graduate because they failed the prerequisite. Interpreting and communicating a finding like this can lead to a host of problems for the student: stereotype threat, giving up, self-fulfilling prophecy, tracking and more. Telling a student that they will not graduate based on a predictive system finding can be scary and discouraging to the student and does not take their whole self into consideration.

On the other hand, a data literate counselor would phrase the same finding in a different way. They would tell the student that, after failing a major prerequisite, their chances of graduating in that major have decreased, or are low, and it will be challenging to graduate from that major. This data literate response is both accurate and empathetic to the student. Without data literate end-users, students can suffer the consequences of misunderstood findings, having the opposite effect desired.

Institutions implementing predictive analytics systems and a data-driven decision making culture must ensure that data literacy training is available to its staff and faculty. Just like bias trainings, data literacy training can cover statistical concepts that seem easy to understand but can be easily mis-used in practice, such as probability. Similarly, training on how to communicate these findings is key to successfully using predictive systems for student success. While the added cost and effort to implement such trainings will likely be financially and logistically challenging, this additional investment is important to ensuring predictive systems and big data meet their return on investment and that students are not harmed.

Predictive analytics are only as good as our ability to act upon them effectively, including our ability to accurately interpret findings. In order to ensure that the investment in predictive analytics and big data pays off, higher education also needs to invest in equipping end-users with the data literacy skills and tools necessary to effectively and ethically help students.

Enjoy what you read? Subscribe to our newsletter to receive updates on what’s new in Education Policy!