16 Nov 2018

Sexism and political bias are easier to spot in nonsense news

The iconic “Media bias chart” creator, Vanessa Otero, about how to spot bias towards women across the political spectrum, how her passion project turned into a start-up, and being smarter about the news.

Tijana Cvjeticanin
Tijana Cvjeticanin Femfacts NewsMavens, Europe
Sexism and political bias are easier to spot in nonsense news - NewsMavens
Vanessa Otero, provided by Ms Otero

The “Media bias chart” started as an image in which about 40 media sources from the United States were ranked for political bias and journalism quality. With time, the chart went through several stages of development, getting more complex, more comprehensive and more serious. Today, its creator Vanessa Otero is one of the leading authorities on media bias.

The original chart, titled “news quality”, was informative, interesting and a bit snarky. USA Today and CNN, for example, received a less-than-flattering description of being just "better than not reading news at all". The viewers were advised to steer clear of the sources which ranked low in quality and extreme in bias, with labels such as “Don’t read this” and “Just no”. The media placed at the liberal and conservative ends of the bias axis were described as “Utter garbage/conspiracy theories”.

News Quality.V5 (Source: http://www.adfontesmedia.com/)

This was Vanessa Otero’s first take on the credibility and trustworthiness of the media in the US, after the chaos that overtook the US presidential election campaign in 2016.

“I was really alarmed with the news sources that people would post and rely on. It was shocking to me how people couldn't tell the difference between a good, reliable source which reported facts, and really biased ones, which do nothing to challenge people’s opinions.”

She also realized that the people who are prone to trust unreliable (and sometimes truly terrible) sources, would probably never dedicate time to reading complex analysis provided by media research organizations or universities. So she started her "passion project", working nights and weekends to create an easy to consume visual guide to the American media landscape.

Otero is a patent attorney with a degree in English. She put both sets of skills to use in making the chart. 

"Being a lawyer and making arguments is important to me and so are academic rigor and methodology. Everybody who’s putting information out there that’s meant to influence has a responsibility to make sure that it’s as good and reliable as you can possibly make it."

"I really take responsibility to make sure that the methodology and the data match the conclusions on the chart, because people rely on this. Teachers teach it in their classes, college professors teach it in media literacy courses, people talk about it at libraries.” 


In the issues which dominate US politics, several are directly related to women and women’s rights. But sexist attitudes more often appear as a “side mention” in articles with a strong political bias.

“When ranking an article for bias I'm looking for a particular political position. We have a baseline of about 20 issues which are the most common in the US: immigration, economy, taxes, gun laws, reproductive rights, LGBT rights. Those span the spectrum. Usually an article is directed towards one or two of those topics, and then brings in one more other thing. That’s typically how bias appears and it's easy to spot if you're attuned to it.

For example, an article might concern the migrant caravan, which is primarily about immigration and secondarily about national security. But then, the articles on the 'right side of nonsense' are about a lot of these women being pregnant, and waiting to cross the border to have their babies here. It's an extra dig at somebody, not just about being an immigrant and/or a threat to this country, but also this promiscuous, uncontrollably reproducing type of woman. It adds an extra layer of right-wing bias."

"When you're looking at nonsense, it is much easier to spot than stuff that's just slightly biased. It's harder to detect bias in the 'middle' sources, [and] much easier to detect the bias in outright nonsense kind of sources.”

The language used to talk about women’s issues is very indicative for where the media stands on the left/right bias axis.

“It’s the easiest to detect it linguistically, in the way in which the two sides characterize issues. For example, the issue of abortion -- you can refer to it in a lot of different ways. 'Reproductive freedom' or 'reproductive choice' are the left-leaning terms. You can also refer to it as 'reproductive rights', which is still left, but it's more neutral. 'Abortion' is the clinical, technical term, so it tends to be used by both sides. But terms like 'abortion proponents' or 'abortion rights activists' tend to be more skewed right. There are definitely linguistic patterns and those are the things that we anchor with our methodology.”

It’s also an ever-moving target that needs to be “recalibrated” regularly.

“Words go through various phases of being reclaimed. For instance, 'queer' was successfully reclaimed -- it was a neutral 'nothing word'; then it was a pejorative word; now it's a term preferred by the community to which it refers. 'Feminism' was a positive term for the left; then the right successfully turned it into a negative. It still retains a largely negative connotation, but I think it's in the process of being reclaimed by feminists.

Besides the linguistic patterns, Otero also noticed storytelling patterns in the left/right framing of women’s issues.

“When you get to more extreme sources to the right, you tend to see more factual inaccuracies about issues like contraception and birth control. They tend to be less detailed, less scientific, less about how a particular kind of birth control works."

"The other thing that shows up as a pattern is the omission of individual women's stories. Most left-leaning stories about why it's important to have those rights, focus on things that happened to a particular woman -- like rape, or a medical condition -- where she really needed an abortion. Whereas anti-abortion type of writing will typically be more general and less specific about an individual woman's plight in a particular situation. It's not always true -- sometimes we see individual stories about, for example, a woman who regretted having an abortion.”

Overall, overt sexism is rare, especially in the media placed in the center and high on the chart. Everyone knows they would “get in trouble by their peers” if they said or wrote something explicitly sexist or racist. But, when it comes to top ranking publications, the problems are still there, they are just deeper under the surface.

“It's like racism in the US, it’s built in the system. The #MeToo movement revealed a lot of structural sexism that doesn't necessarily come out in an overt way. You look at examples like Matt Lauer or Charlie Rose. They didn't say things that were necessarily anti-women, but when you step back, you see patterns -- like how Lauer treated Hillary Clinton in a debate. When his background of sexual assault came to the surface, it made more sense and people could spot the sexism more easily.”

Journalism at the top, Otero says, is still a male-dominated field, as are business and politics. Men still hold most of the positions of power, being the majority of editors, owners and publishers.

To tackle the problem, it’s important for everyone to acknowledge it and work together to bring change -- but it can put an additional strain on women who succeed in the field.

“That's something that becomes very apparent as I start this business. The people who are in a position to help me expand this into a data company are mostly men. And I have some really wonderful men on my team who understand these issues. They realize that there are not a lot of women that are their peers, that are in positions of power in terms of journalism, academic, start up and finance experience. So they are actively working to change that.

There are just not a lot of women in those positions. Those who are, work really hard to mentor people like me, who are trying to make a difference in this male-dominated field. That's a lot to ask from them.”


Otero's Media Bias chart has already become a learning tool, received media mentions and made its way into academic papers. It has also continued to resonate with readers and viewers, who backed Otero’s crowdfunding campaign with over 32,000 dollars on “Indiegogo”. The funds will go into development of her data company and making an interactive platform which will have more media sources, more detailed analysis and more data which can’t be placed on two-dimensional images.

The reach of each individual media, for example, is an important piece of the puzzle that is currently missing from the chart:

“If certain news stories get many readers, followers and engagements, it’s really impactful. If something very extreme to the right, for example, is highly read and followed, that impacts discourse across the board.”

Media Bias Chart: Version 4.0 (Source: https://www.adfontesmedia.com/)

The ultimate goal is to apply the principles of consumer protection, similar to the ones in patent law, to media “products”. Otero insists that she wants to keep her public benefit company independent from ads and corporate donations, connecting to people directly and gathering their support on the basis of what their real needs and genuine interests are.

“When I put it on the internet, I started getting a lot of inquiries and the biggest and most important ones to me were 'What is your methodology?' and 'Add more sources!' Since then, I started creating and updating refined versions, to the point where we’re going be able to bring entire teams of analysts to improve its quality. I want to pursue this as a project that can be ongoing and continue to help people.”

The media sources in Otero‘s chart are ranked for bias and quality based on things like sentence structure and wording, and what type of information they mostly provide -- news, analysis of news, opinions or persuasion. These elements shift the source left/right for political bias and up/downin terms of quality. Otero ranks them by analyzing individual pieces of news they produce. The overall source ranking is a weighted average of individual articles’ scores.

The criteria for the placement of each media have changed in the process. Designations such as “liberal and conservative” were replaced with “left and right” to encompass a wider range of the political spectrum. The vertical axis -- one that ranks journalistic quality -- was also modified.

“At the very top of the first chart I initially put 'complex analysis'. Another type of good journalism is the one where journalists get facts on the ground. I switched those two and put original fact reporting on the top, because complex analysis wouldn't exist without the base of actual journalism and facts.”

Otero says that it’s excellent news that a lot of investigative journalism can be found in media sources placed in the “complex analysis” category and above. She also points out that factual reporting needs to be consistent and continuous for the media to score high. Basing some pieces on facts while also using “hyperbole and extreme rhetoric”, does not cut it in journalism, Otero says.

“Even on InfoWars, a lot of things are based in fact. It’s not all outright fake news or hoaxes -- but the ones that are, are a really big deal."

"If anyone is presenting themselves as news, they should be correct 99% of the time or more and publish a correction when they make a mistake." 

"Everything that’s above the middle of the chart has really high fact reporting percentages. They’re not reporting things that are false on a regular basis -- even five percent would be terrible.”

The interactive version will respond to many of the viewers’ requests that she received in the past two years. It’s also going to make the methodology easier to understand without reading the “long form posts” which already amount to an entire book.

“The static image is very popular, but you can only fit so many sources in there and my latest version has a bunch that overlap, so it's a little hard to find things. People want to know even more sources because there are thousands out there. The better the interactive version gets, the more people will see the methodology without having to read it. I appreciate everyone who spends a lot of time reading it, but I also want to make it possible to get answers just by clicking on it. Then the methodology becomes apparent and they can have more trust in it.”


Some of the initial ranking criteria were related to the history and reputation of the analyzed sources. How long have they been around? Have they existed in print? Were they an “old school” broadcast media -- television before cable and radio before satellite? These criteria speak to a time when it was harder to establish a news outlet and keep it running. More effort had to be put into building a reputation as a quality and fair news source, to maintain readership or viewership.

The same approach, however, would be hard to apply in many European countries, some of which still struggle with media freedom. In countries where most or all media were owned by the state until the turn of the century, the individual existence of media outlets is either fairly short, or tainted by overt political interference.

So, could the chart be replicated in places with a “media history” significantly different from that in the United States, in places where independent media enterprises were only developed in the past few decades? In the later versions, Otero says, she moved away from factors which lean heavily on “tradition” and made the approach more universal.

“I moved more strictly towards content analysis and away from things like reputation and long-standing existence. They’re not the most telling and we don’t need to rely on them. They can provide some kind of baseline in the US, but they would be different in the former Eastern Block, or other places. I talked to a group from India, where there’s been a proliferation of internet media which has been helpful in terms of uncovering truth, when in the US it’s been the opposite -- a part of the problem. But what translates across countries in terms of how we rate quality is veracity (is something accurate and truthfull.) and expression (is something stated as fact or as an opinion). Those factors can be analyzed in sentences written across the world.”

Otero’s chart does rank a few UK-based media, but only in terms of their coverage of US-related news. For example, Otero says, the BBC is viewed as quite a neutral source in its coverage of US politics, but that’s not necessarily true for how people in the UK see its coverage of local politics. Placing media outlets on the axis of political bias also depends on the local context, as ideas about what’s “left”, “right” and “center” differ from country to country. These categories would also need to be adjusted if one were to develop a similar chart for some other place in the world.

"It's obviously not going to be the same everywhere, but you can anchor most things on a left/right spectrum. In Syria, for example, if we take 'pro-Assad' to be one side and 'anti-Assad' to be another, you could arbitrarily pick which of those is left or right, but generally authoritarianism and strongmen type of rule would be more associated with right-leaning politics across the world."

“The left-right spectrum in the US is anchored by our current Republican and Democratic parties and political positions of our elected officials. Politicians state their positions in platforms that are written down and they evolve from year to year, so I can recalibrate the left-right political spectrum for the US accordingly. For ranking things internationally, you’d have to pick a region and look into its particular politics. 'Europe' might be too big of a region, you'd have to go country by country."


Project #Femfacts co-financed by European Commission Directorate-General for Communications Networks, Content and Technology as part of the Pilot Project – Media Literacy For All

The information and views set out on this website are those of the author(s) and do not necessarily reflect the official opinion of the European Union. Neither the European Union institutions and bodies nor any person acting on their behalf may be held responsible for the use which may be made of the information contained therein.

NewsMavens is a media start-up within Gazeta Wyborcza, Poland's largest liberal broadsheet published by Agora S.A. NewsMavens is currently financed by Gazeta Wyborcza and Google DNI Fund.
Is something happening in your country that Newsmavens should cover?
Zuzanna Ziomecka
Zuzanna Ziomecka EDITOR IN CHIEF
Lea Berriault-Jauvin
Lea Berriault Managing Editor
Jessica Sirotin
Jessica Sirotin EDITOR
Ada Petriczko
Ada Petriczko EDITOR
Gazeta Wyborcza, Agora SA Czerska 8/10 00-732, Warsaw Poland
The e-mail addresses provided above are not intended for recruitment purposes. Messages concerning recruitment will be deleted immediately. Your personal data provided as part of your correspondence with Zuzanna,Lea, Jessica and Ada will be processed for the purpose of resolving the issue you contacted us about. The data provided in your email is controlled by Agora S.A. with its registered office in Warsaw Czerska 8/10 Street (00-732). You can find more information about the processing and protection of your personal data at https://newsmavens.com/transparency-policy