Media Bias and Slanted Reporting

Words and language form and manipulate our thoughts and opinions. A single event, presented in different ways, is transmitting more than just facts. This is called media bias. In this way, opinions and ideologies can spread subconsciously and lead to rising levels of polarization, splitting people into antagonizing groups and endangering democracy

Right now, where we live in a flood of information that drowns out facts, it is more important than ever to be aware of the impact language has on us. Mindful media consumption and especially critical reading skills are crucial to navigating the media landscape and the attention economy.

Figure 1: Reading news critically

Automatic Detection through Artificial Intelligence

Progress in the fields of Machine Learning and Natural Language Processing has made it possible to automatically detect biased wording. But for this to work, enormous amounts of quality data have to be collected. The BABE dataset (Spinde et al., 2021) used experts to create the labels and showed promising results in doing so, but more data is needed. Thus, the creation of the data set is the limiting factor and the main problem.

The Idea

Figure 2: The idea: creating a game to collect data

This thesis is based on the idea of creating a game that trains critical reading and gathers data as a byproduct of play. For a quality dataset, players have to become experts in detecting bias. By teaching players and collecting their bias labels when they have reached expert level, a win-win situation for both parties is created. The challenging part is to turn the data labeling mechanism into a form that is perceived as fun and, therefore, passes as a game.

The Media Bias Game

Figure 3: MockUps Bias Game: home screen, level up, and bias labeling mechanism

The final result is a strategy game where players own an outlet and publish news. The 'news publishing' is the core mechanic. Depending on what news players choose to publish, their outlet is positioned on the market and attracts different follower groups.

Follower Groups can prefer liberal, conservative, or center content – and different levels of polarization language on a spectrum from factual to polarizing. Later, players can accept offers from ad companies that give extra money, but the ad companies demand different follower groups in return. To maximize their money, players have to strategically publish articles from different positions to match the requirements of the ad companies. Through this mechanic, players understand from the inside how the media system works.

Before publishing anything, players have to analyze the article they might want to publish. This is the core loop of the game and the one that generates the data.

Figure 4: Labelling sentences and receiving feedback on word level and sentences level

To analyze the article, players have to proofread the displayed sentence for media bias. If they find biased words, they tap them. Afterward, they decide if the whole sentence is biased or factual by swiping the card to the right for biased or to the left for factual. If there is enough data on the sentence, they receive instant feedback, enabling players to learn quickly and with delightful feedback. If the sentence is new in the database, it is moved to another section of the game where players collect an even higher reward later. Then they can decide if they want to publish that sentence in their outlet.

Figure 5: Final Version MockUps: Choosing a topic, analyzing sentences, receiving feedback and joining the discussion

The Bias Plant is the players' personal companion that leads them through the first five levels of the tutorial, which are seamlessly integrated into the game. It grows with each level along with the player and motivates or helps them.

Figure 6: Bias Plant growth

Evaluation

The goal of the study was to find out if the form of a casual game can teach players about Media Bias and help them to read more critically. The first seven levels of the game were prototyped into an Android mobile game. The first five levels teach the players the concepts of biased wording on sentences and game mechanisms that get harder with each level. The last two levels test if the players can apply the learnings. Furthermore, qualitative feedback on the game experience was gathered through a UX survey.

The game can be downloaded here

Players who played the tutorial did 5.4% better than the control group, felt more competent, and enjoyed the experience more. While everybody loved the Bias Plant, some perceived the game to be survey-like and too reading intensive. The final tutorial design cuts down on words and teaches through showing and doing while focusing on clear, fun feedback. Additionally, more game modes were added to create enjoyable options for the different player types.

Figure 7: Bias Game study prototype screenshots

Outlook

In the long term, we will build tools with the help of our AI to aggregate news and visualize bias, making it easier for all readers to get the facts straight. 

This thesis served as a starting point. My Ph.D. research proposal was accepted at the IEEE ICDM Conference on Data Mining where I will present my future research plans regarding the Bias Game and bias visualizations. The international feedback as well as the study results will flow into the next version of the Bias Game that will be finished in late January.

Figure 8: Possible news article visualization with AI-powered bias detection

Figure 9: Final game design main page with different game modes

References

Spinde, T., Plank, M., Krieger, J.-D., Ruas, T., Gipp, B., & Aizawa, A. (2021). Neural media bias detection using distant supervision with babe—Bias annotations by experts. Findings of the Association for Computational Linguistics: EMNLP 2021.

Official Presentation

Student: Smilla Hinterreiter

First Supervisor: Prof. Steffi Hußlein

Second Supervisor: M.Sc. Timo Spinde

Date: 3. Dezember 2021 // 08:30 am

Zoom: https://hs-magdeburg.zoom.us/j/87392863094