How Recommendation Algorithms Shape Your Online Experience

Whether you are searching for a pair of shoes to buy, a TV show to binge, a long-lost friend to reconnect with, or simply browsing the internet, your online experience will be shaped by recommendation systems.

Recommendation systems are algorithmic tools that internet platforms use to identify and recommend content, products, and services that may be of interest to their users. Today, platforms are employing personalized recommendation systems for everything from social media to e-commerce to media streaming. These systems are responsible for recommending a range of content, including friends, posts, ads, news articles, trending topics, items to purchase, jobs, and more. In doing so, these systems are able to influence user interests, opinions, and behaviors, as well as their social group formation.

Internet platforms assert that they use automated recommendation systems to enhance users’ experiences on their platforms through personalized and relevant recommendations. Of course, in doing this, internet platforms also seek to retain user attention on their services. This translates to significant financial benefits for the companies, as they can target users with advertisements and recommend further content to consume or items to purchase. In addition, definitions of “relevance” vary across platforms, and are largely based on what a platform believes a user is interested in through its data collection and inference practices.

In some cases, however, the use of algorithmic decision-making tools results in the recommendation of harmful content, such as misinformation, conspiracy theories, and extremist propaganda. In addition, in some instances these recommender systems yield discriminatory outcomes that reinforce societal biases.

Internet platforms that deploy automated recommendation systems do not currently provide meaningful transparency and accountability around how these systems are created and operated. In addition, there is little visibility into how platforms have developed and crafted these recommendation systems, how they operate, and how they make decisions. This makes it very difficult to analyze and combat the problematic recommendations that come from these systems. This lack of transparency and accountability from internet platforms is especially concerning given that recommender systems have a significant amount of influence over how users engage with, and are influenced by, the online sphere. Not only can recommender systems influence product purchases, but they also can determine what content—such as news articles—a user will even see.

In our new report, New America’s Open Technology Institute (OTI) explores how three internet platforms—YouTube, Amazon, and Netflix—utilize algorithmic recommendation systems, and the challenges associated with these practices. The report offers recommendations on how internet platforms and policymakers can promote greater fairness, accountability, and transparency around these algorithmic decision-making practices. Because the First Amendment limits the extent to which the U.S. government can direct how internet platforms decide what content to permit on their sites, the report provides a limited set of recommendations for action by policymakers.

The recommendations presented in this report include:

Internet platforms that utilize recommendation systems should:

1. Disclose to users the situations in which the platform uses an algorithmically-curated recommendation system, and provide comprehensive and meaningful explanations to users around how these recommendation systems work.
2. Explain to users why a recommendation was made to them.
3. Disclose granular data around how the company trains its algorithmic recommendation systems.
4. Enable independent researchers to conduct audits to review and verify relevant internal models and data.
5. Hire independent auditors to conduct regular periodic audits of recommendation algorithms in order to identify potentially harmful outcomes, and take steps to address the findings of these audits, including mitigating discrimination and bias.
6. Share granular data related to how the company tests its recommendation systems and how it determines how effective its systems are.
7. Improve user controls so that users can easily manage whether and how their data is collected and inferred, how this data is used, and how it influences the recommendations that they see.
8. Share the platform’s Terms of Service Community Guidelines related to topics such as content and purchases, and how they are enforced.
9. Publish a transparency report outlining the scope and scale of Terms of Service enforcement actions in all of the regions in which it operates.
10. Explain how the company uses human evaluators to review and train its algorithmic and machine learning models.

U.S. Policymakers should:

Enact rules to require greater transparency from online platforms regarding their use of algorithmic recommendation systems.

This is the fourth in a series of four reports that will explore how internet platforms are using automated tools to shape the content we see and influence how this content is delivered to us.