The work that we are contributing to, in addition to our A.I. bias meter, is to create an analyst-driven bias meter that rates news stories based on individual author leanings. To understand which way an article leans, a variety of factors are considered including:
- Diction and language
- Author check
- Tone and leaning
- Tendency of leaning
- Selection and Ommission
- Expediency Bias
- Accuracy of Facts
These factors are judged by Biasly analysts who are well-versed in current issues and are able to make informed decisions on biases. Each article is rated equally with a numerical score by a liberal, moderate, and conservative-leaning analyst to provide greater accuracy. These scores are then averaged to create a composite score bias rating for the article. After multiple articles are rated, we can then determine an overall bias score for the news source. These ratings can be found on our media ratings page.
Additionally, every article rating is reviewed by a second analyst to ensure further accuracy. This measure of double-checking reduces the likelihood of the personal political bias of a reviewer to affect the ratings.
There may be questions as to why the analyst-driven rating system is binary, in that, there are only left and right leanings instead of measures such as libertarian and authoritarian. However, for the purposes of what the bias meter is supposed to accomplish, these extra measures would only serve to complicate the issue at hand, as our algorithms aren’t designed to calculate party leanings but rather conservative vs liberal leanings.
What are Analyst-Driven Bias Ratings?
The Bias Meter itself is primarily focused on American politics with a few other countries’ articles mixed in. This focus on American politics allows the Bias Meter to be more exact, as definitions of conservative and liberal biases vary across national lines. Therefore, even when looking at news sources such as the BBC or The Guardian, articles on American politics would be chosen and judged on that basis. However, there are some exceptions to this rule with certain organizations or articles. In such cases, articles are viewed from the perspective of American politics. For example, if the article is about the Israeli-Palestine conflict, then the left would generally support Palestine, while the right would generally support Israel. The article would then be judged on the basis of American politics. This allows for a consistent rating across a diverse range of subjects, ensuring that the Bias Rating is accurate.
Why is it Important?
The importance of having a tool such as the Bias Meter being developed by Biasly cannot be overstated, especially looking at the events of the past few years. The issue of fake news and attacks on journalists have gotten the spotlight in recent years, especially regarding issues surrounding increased polarization and the Coronavirus pandemic. Due to these issues, readers will have a lot to gain from the bias meter as it can reveal which way various news sources lean. An especially important component will be the accuracy measure that is implemented within the ratings, which looks at the sources that the article uses to get its information.
While the express purpose of the bias meter is not to judge how reliable a news source is, looking at how much bias the news article has could go a long way in revealing to the reader to be wary of the information they are reading, as a high bias rating may indicate that the author has the intention to misinform the reader to create more support for their political stance. Through the large number of articles we have read through, we can personally confirm that there are some malicious actors within the media landscape that twist and change the facts of an event to suit a certain narrative – both on the left and the right.
The Rating System
Each article rating is based on a binary score from -100 to 100 with negative numbers being liberal and positive numbers being conservative, just as if on a number line, and 0 indicating neutrality.
One of the most important aspects of the Bias Meter at Biasly is its accessibility. Even to readers that may not be well-versed politically, the Bias Meter is simple to understand. All articles have a percentage scale from 0-100 with 100 being the most extreme. Conservative articles will have a positive integer, and liberal articles will have a negative integer. However, just because one number is negative or positive, it does not mean that one side is better or worse. If you look at a number line, negative numbers will be on the left, and positive numbers to the right, thus it is not a normative statement about any side. Because the rating is visible before even clicking on the article to read it, readers will be able to immediately identify what kind of bias they may be exposed to when reading the article. Additionally, because news organizations themselves are also rated, it is made much simpler to find news that is neutral. Although neutrality may not guarantee accuracy, it would be immensely useful in narrowing down what news sources to consider/read.
Who Can Benefit?
These ratings could have different values for different readers. We see this bias meter being especially useful for students, as they may have to find and read news articles for an assignment. In that case, the Bias Meter would be of great help in guiding students to sources to provide for their specific needs, whether that be a neutral perspective on events or a certain viewpoint that they are trying to find. In terms of the general public, as stated above, the Bias Meter promotes the reduction of disinformation on the internet. Especially looking at social media platforms, disinformation is spread by malicious actors, which has sowed considerable discord among political discussions, as many are acting under false information.
Another hope with the Bias Meter is that readers can find out what kind of news they are usually exposed to. Some people may be disproportionately exposed to news of a certain bias, especially with the advent of algorithms on social media that recommends users with similar content with what they have already watched, creating an “echo chamber”. Everyone can relate to watching one video on Youtube, or clicking one link on another social media site and getting recommended similar content for a long period of time. This is the danger of having a polarization effect in which media consumers will be increasingly pushed to one side of the political spectrum. Therefore, using the Bias Meter will be greatly helpful in letting users realize that they may be disproportionately consuming information of a certain political position. Therefore, utilizing the Bias Meter, they can get a quick understanding of how they may have been affected by algorithmic decisions. The only issue would be if consumers purposely and willingly choose news sources that they agree with. Hopefully, the Bias Meter could either get people to understand this polarization before they get to that point, or realize that they have fallen into a cycle of news that only looks at one side of political issues.
Personal Reflections
Some of the personal takeaways we have had from working on the Bias Meter is not only how much the news is affected by the personal bias of either the author or the organization, but the smaller ways in which that bias is seen. While it is obvious to see bias if the author makes normative statements such as “X is wrong for doing Y”, some of the quieter ways in which bias creeps into articles. One of the most difficult factors to account for is selection or omission bias. This is especially hard to detect as we have to have knowledge on the issue at hand or be able to research it effectively to recognize that either some information is missing or other information is focused too much upon.
An additional factor that may complicate issues is that the diction may not in and of itself seem biased. Therefore, to consumers who are learning about the issue from the articles themselves, they would have no idea that they are getting filtered information, and contemporary readers usually will not have the time or the patience to look through multiple articles on the same issue to understand the full picture. Therefore, having a Bias Meter that makes it immediately apparent where articles lie on the political spectrum, which saves users time and does not require expertise.
In conclusion, the Bias meter performs a useful role in political media by cutting through the cluttered and confusing world of news media to show which articles are siding with which stance and thus giving users critical additional information. From students to the general public, the Bias Meter has the potential to help a wide cast of media consumers. This is achieved through a painstaking process of reviewing multiple articles from each news organization to find out exactly what kind of biases the author and organizations are acting under – from looking at the author’s social media to looking at the sources that the article uses, We hope that the work that has gone into the Bias Meter is compensated by reducing and combating the large amount of disinformation that is proliferating on the internet.