Post-Trump Facebook Ecology:
The 'False' Solution To The Fake News Problem

Team Members

Paul Bugeja, Maria Lompe, Yumeng Luo, Richard Rogers, Tatiana Smirnova, Ilian Velasco, Nina Welt

Contents

1. Introduction

In 2016, Craig Silverman of Buzzfeed News reported that fake news was outperforming mainstream news on Facebook in the months leading up to the 2016 election. In 2020, Richard Rogers revisited Silverman’s methods and findings and concluded that the results depended on how fake news was classified. When following Silverman’s definition, the problem was found to have worsened, but when applying a narrower definition of fake news, the problem’s scale was not as large. We built on Roger’s study and found that, since March 2019, mainstream news has outperformed fake news when applying both definitions, meaning Facebook’s content moderation efforts have been somewhat successful; however, mainstream and problematic sources have concerningly become almost indistinguishable in terms of engagement.

Essay summary

• Engagement on Facebook was determined by classifying the top 300 results of a series of chosen election-related keywords and dividing those into two main categories: mainstream and problematic, where mainstream was subdivided into conservative, progressive, or neither, and problematic was subdivided into questionable, clickbait, satire, conspiracy, and imposter.

• In 2016, when Craig Silverman’s report came out, fake news performed better than mainstream news. In contrast, this study shows that when applying the same definition (including (hyper)partisan sources) mainstream news is outperforming fake news. It follows that this is also the case when using a narrower definition of fake news, where (hyper)partisan source are excluded.

• Problematic information is basically indistinguishable from mainstream information in terms of engagement and (hyper)partisan conservative sources have far more user engagement than (hyper)partisan progressive ones, particularly during the first six months of 2020.

• As Facebook is making efforts to stop fake news, our study indicates that one of the strategies - “Ranking Improvements” (Facebook for Media) - that Facebook has used to prevent fake news from spreading might have the opposite effect since it could make this content more likely to appear in the news feed of users who are most engaged with fake news, especially (hyper)partisan conservative sources.

3. Research Questions

1. How pervasive was the problem of fake news, as defined by Silverman’s article, in Facebook’s top content during the 2020 election cycle?

2. In comparison to the 2016 election cycle, how problematic is fake news during the 2020 cycle, using both Silverman’s definition and the narrower one?

3. On Facebook, is there more engagement with (hyper)partisan conservative or progressive sources? What does this mean for the politicization of fake news?

4. Could Facebook’s content moderation efforts be considered effective?

4. Methodology and initial datasets

To answer the research questions regarding the effectiveness of content moderation, whether algorithmic, spot-checking, or otherwise, we curated a list of the following topical 32 issues: [“abortion”], [“affordable housing”], [“assault weapons”], [“background checks”], [“biden”], [“campaign financing”], [“carbon emissions”], [“charter schools”], [“climate change”], [“coronavirus”], [“covid”], [“daca”], [“death penalty”], [“election security”], [“fake news”], [“gun control”], [“health insurance”], [“immigration”], [“infrastructure”], [“medicare”], [“minimum wage”], [“oil and gas drilling”], [“paid leave”], [“private prisons”], [“sanders”], [“securing 5g”], [“social security”], [“student debt”], [“teacher pay”], [“trump”], [“veterans”], and [“wealth tax”].

We then queried BuzzSumo for each keyword with the following filters:

• Date range: from 23 March 2019 to 5 January 2021

• Language: English

Keywords that were more than one word were queried in “quotation marks” to maintain search precision. We then sorted the results for each query by Facebook engagement and stored the top 1,000 results. Following this, we transformed the URLs to just their host names using the Harvester Tool and classified the top 300 results. For some keywords, fewer results were stored and/or classified because there were a limited number of returned results. Additionally, if non-English results were included for whatever reason, then we manually excluded them.

The categories that we used to classify up to the top 300 results were:

• mainstream, in which case they were sub-categorised as either conservative or progressive for (hyper)partisan sources, or neither for neutral sources; or

• problematic, in which case they were sub-categorised as either conspiracy, imposter, questionable, satire, or clickbait

To assist in this classification, we used four resources: Media Bias/Fact Check, Ad Fontes Media Bias Chart, AllSides Media Bias Ratings, and NewsGuard, as well as analysed the website of the source itself when the aforementioned resources were insufficient or in disagreement. Further, the imposter results were automatically classified according to a pink slime list, which contains domains that are known to be purporting to be local or business news publications.

Additionally, user-generated sources were generally classified according to the platform (for example, Medium was classified as mainstream because it has content policies); however, the exception to this was when the content had been removed for violating these very guidelines, in which case it was classified accordingly. Further, user-generated sources were unclassified in instances where there was significant inconsistency on the platform (for example, YouTube) due to time limitations.

These categories allowed the formulation of the following definitions of fake news:

• False news: problematic (narrow fake news)

• Misleading news: problematic and conservative and progressive (broad fake news)

We then compared the sources of these narrow and broad definitions of fake news to the mainstream sources; that is, we compared problematic and conservative and progressive versus mainstream (excluding conservative and progressive) and problematic versus mainstream (including conservative and progressive). This allowed us to determine how different classifications of fake news shape the way we view Facebook’s problem. Further, we also compared conservative versus progressive to analyse the (hyper)partisan news problem and the politicisation of news sources.

We then divided the date range into eight quarters and charted both the narrow and broad comparisons of fake news, as well as the (hyper)partisan comparison, to visually depict the findings. Further, we also charted these comparisons for each keyword to reveal more granular patterns per issue. Finally, we compared these findings to the findings in Craig Silverman’s seminal article in 2016.

It is important to note there were some limitations to our study. Mainly, we classified the top 300 results for each selected keyword. Further research might benefit from classifying a larger amount of data per keyword. Also, sources in other languages were excluded from the results. Further research might reveal different findings in different regions due to geopolitical and/or geocultural differences.

5. Findings

Findings

The study compared the ratio of misleading news to mainstream news under Silverman's broad definition, which includes (hyper)partisan sources, and false news to mainstream news using a narrow definition, which does not include (hyper)partisan sources. We compared the results presented in 2016 by Silverman prior to the presidential election and those from March to December 2020, which also includes the 2020 United States presidential campaign.

Finding 1. If one applies Craig Silverman’s original definition of misleading news, then the problem has decreased. The ratio of misleading news to mainstream news was lower during the 2020 presidential campaign period compared to the 2016 presidential campaign period. In the run-up to the 2016 presidential election, the ratio of misleading news to mainstream news was 1:0.8 (Silverman 2016; 2017), while in the same period in 2020 the ratio was 1:1.8. Further, in 2020, misleading news never exceeded mainstream news, hence the problem has objectively improved.

Figure 1: Misleading news (includes (hyper)partisan sources) vs. mainstream news (excludes (hyper)partisan sources)

Finding 2. Further, if one applies the narrow definition of false news, then the problem is even less pronounced. From April to June 2020, the ratio was at its lowest at approximately 1:20 and from July to December 2020 it was stable at approximately 1:10. Thus, this suggests that Facebook’s content moderation has likely been somewhat effective.

Figure 2: False news (excludes (hyper)partisan sources) vs. mainstream news (includes (hyper)partisan sources)

Finding 3. (Hyper)partisan conservative sources are far more engaged with than (hyper)partisan progressive sources, suggesting that fake news can become politicised. The ratio of conservative to progressive (hyper)partisan sources remained around 1:2 for the duration of the study, peaking at 1:2.4 from April to June 2020. This means that misleading news, in the sense of sources that include (hyper)partisans websites, are usually associated with conservative sources. Thus, in terms of engagement, (hyper)partisan conservative sources and mainstream news are virtually indistinguishable.

Figure 3: (Hyper)partisan conservative sources vs. (hyper)partisan progressive sources

Finding 4. If we look at the ratio of misleading news to mainstream news for individual searches from a broad definition perspective, misleading news often receives more coverage than mainstream media. This is particularly evident for searches such as "Abortion", "Election security", "Gun control", "Securing 5G", and, ironically, “Fake news”. Mainstream news, on the other hand, outperforms misleading news in most categories, notably "Carbon emission", "Coronavirus", "Health insurance", and "Paid leave".

Figure 4: Misleading news (includes (hyper)partisan sources) vs. mainstream news (excludes (hyper)partisan sources) per issue

Finding 5. There are a couple of topics that, even under the narrower definition of false news, can gain more engagement than mainstream news. These are searches for "Oil and gas drilling" and “Social security”. Interestingly, Rogers’ 2020 study did not see such a trend, where, in the narrow perspective for each of the queries, mainstream news always outperforms false news. This may indicate an even greater polarization of these topics and possibly an increase of false news in certain issues.

Figure 5: False news (excludes (hyper)partisan sources) vs. mainstream news (includes (hyper)partisan sources) per issue

Finding 6. (Hyper)partisan conservative sources outperform (hyper)partisan progressive sources, especially for queries "Abortion", "Death penalty", "Gun control", and "Fake news"; however, there are some cases where (hyper)partisan progressive sources outperform (hyper)partisan conservative sources, such as "Medicare", "Oil and gas drilling", "Paid leave", and "Social security". This relationship points to specific areas that gain more coverage in particular political groups; however, it may also suggest that these topics have a political tinge.

Figure 6: (Hyper)partisan conservative sources vs. (hyper)partisan progressive sources per issue

Finding 7. We may see a reversal in some of the trends compared to the March 2020 findings. Back then, for example, "Death penalty" searches were dominated by (hyper)partisan progressive sources, whereas now, much more engagement in this category falls to (hyper)partisan conservative sources. In contrast, "Sanders" queries from January to March 2020 received more engagement from conservative sources, while they are now dominated by progressive ones.

Finding 8. We encountered only one source of so-called pink slime (Burton & Koehorst, 2020), which was not highly engaged with, suggesting that users are savvy with respect to imposter content; however, this means that pink slime can still appear on Facebook, while no such sources were found in the March 2020 findings.

6. Discussion

In contrast to fact-checking efforts implemented by Twitter and a ban on political ads from this platform, as well as others such as TikTok and Snapchat, Facebook has opposed such measures, bringing in an estimated $264 million dollars in the third-quarter of 2020 from American political advertisers (Levy, 2020). It was not until January 7, 2021 that Facebook announced that they were temporarily blocking the 45th president of the United States, Donald Trump, from the social network, following the events of the previous day when a mob of conservative extremists stormed the United States Capitol in an attempt to stop the confirmation of Joe Biden (Isaac and Conger, 2021).

A 2019 Pew Research Center report revealed that 55% of people in the United States of America "often" or "sometimes" obtained their news on social media, an 8% increase from the previous year (Suciu, 2019). If the trend has continued at the same pace, then we could estimate that around two-thirds of Americans now use social media as one of their primary news sources.

As Silverman demonstrated, in 2016 the social network was rampant with misinformation, which, as we see now, can have devastating longer-term consequences. To study how pervasive problematic information was in this past election cycle, we followed in the steps of previous research: starting with Silverman’s report on the prevalence of fake news during the 2016 election and following with Rogers’ research on the period from March 2019 to March 2020 that replicated Silverman’s method. As Rogers' report demonstrated, the classification of fake news changes the scope of the problem.

In Silverman’s article, the term fake news encompasses conspiracy and imposter sources, as well as (hyper)partisan sources, which refer to those “extremely biased in favor of a political party” (definition.org). It also included clickbait sources, which Rogers defines as vernacular sources that are presented in a way that entices consumption and computational propaganda; that is, “dubious news circulation by bots and troll-like” (Rogers). On the other hand, fake news as defined in Rogers’ study refers only to conspiracy and imposter sources, leaving out the (hyper)partisan ones. Looking at the issue from these different definitions changes how pervasive the problem is and, therefore, how it should be addressed.

Following this previous study by Rogers (2020), we classified our sources into two main categories: mainstream and problematic. Mainstream refers to widely accepted sources, which we then divided into conservative or progressive to encompass those that tended to be more (hyper)partisan and neither for those that were more neutral. We also divided the problematic sources into five sub-categories to help us understand the nature of fake news on Facebook. The subcategories were conspiracy, imposter, questionable, satire, and clickbait.

While Zuckerberg has maintained a position that Facebook “shouldn’t be an arbiter of truth” (Roose), it has taken some steps to deter the spread of misinformation in response to sharp criticism. According to Roose from The New York Times, millions of dollars have been spent by the platform to prevent what happened in 2016, mainly to deter foreign intervention. Although the exact measures taken by Facebook are unknown to the public, this study evidences that some of these efforts have likely been successful. As will be presented in the findings, when using either definition of fake news, mainstream sources have outperformed problematic information since 2019, suggesting that Facebook’s content moderation policies have been somewhat effective; however, more troubling is that this narrower definition of false news also implies that a bulk of (hyper)partisan sources that were previously considered fake news have now moved on to becoming mainstream.

Following Rogers, “[l]abelling hyperpartisan news as ‘fake’ or ‘junk’,[...], may lead to greater political backlash” (2020); however, the issue of redefining fake news as false news to remove the stigma from (hyper)partisan conservative sources also calls for a redefinition of mainstream news and newsworthy reporting altogether. Following the Dutch press codex for journalists by the Dutch Association of Journalists, the first and foremost rule in ethical reporting is striving for objectivity (Nederlandse Vereniging van Journalisten). In this sense, neither (hyper)partisan conservative nor (hyper)partisan progressive sources should be considered as news sources.

7. Conclusions

For educational purposes, our findings showed that fake news will still be an ongoing problem on social media. Research from the Stanford History Education Group has indicated that nearly 90% of high school students lack basic skills of assessing the news quality (Breakstone, et al., 2019). It is important to incorporate fake news education into the current media literacy curriculum to cultivate students’ ability to spot false and misleading information.

Similarly, users of social media should be aware that false, misleading, and (hyper)partisan information are prevalent on these platforms; for example, one study has demonstrated that fake news and rumors travels six times faster on Twitter than truthful information (Vosoughi, Roy, & Ara, 2018).

For Facebook’s content moderation, our study suggests that Facebook should have a clearer system to define and classify news sources; for instance, Facebook could label (hyper)partisan and imposter sources when such articles have been shared on its platform. Further, although content moderation, be it algorithmically automated or spot-checked manually, is designed to benefit society at large, it also raises wider-reaching questions, such as what content is considered “false” or “misleading”, as well as who is considered to be an authoritative source on such matters. In this regard, arguments could be made that Facebook is allowing one side of the discussion to flourish, while suppressing non-established sources from its platform. This may have the unintended consequence of driving these discussions underground where they may grow and fester, rather than discussing them on a more mainstream platform where unsubstantiated claims are more likely to be exposed and refuted.

8. References

Bengani, P. (2020, August 4). As election looms, a network of mysterious ‘pink slime’ local news outlets nearly triples in size. Columbia Journalism Review. https://www.cjr.org/analysis/as-election-looms-a-network-of-mysterious-pink-slime-local-news-outlets-nearly-triples-in-size.php.

Breakstone, J., Smith, M., Wineburg, S., Rapaport, A., Carle, J., Garland, M., and Saavedra, A. (2019). Students’ civic online reasoning: A national portrait. Stanford History Education Group & Gibson Consulting. https://purl.stanford.edu/gf151tb4868.

Burton A.G. and Koehorst D. (2020). Research note: The spread of political misinformation on online subcultural platforms. The Harvard Kennedy School Misinformation Review.
https://misinforeview.hks.harvard.edu/article/research-note-the-spread-of-political-misinformation-on-online-subcultural-platforms.

Facebook For Media (n.d.). Working to Stop Misinformation and False News. https://www.facebook.com/formedia/blog/working-to-stop-misinformation-and-false-news.

Isaac, M. and Conger, K. (2021, January 7). Facebook Bars Trump Through End of His Term. The New York TImes. https://www.nytimes.com/2021/01/07/technology/facebook-trump-ban.html.

Levy, A. (2020). Why political campaigns are flooding Facebook with ad dollars. CNBC. https://www.cnbc.com/2020/10/08/trump-biden-pacs-spend-big-on-facebook-as-election-nears.html.

Nederlandse Vereniging van Journalisten. (2008 Code voor). de journalistiek, door het Nederlands Genootschap van Hoofdredacteuren. NVJ. https://www.nvj.nl/ethiek/ethiek/code-journalistiek-nederlands-genootschap-hoofdredacteuren-2008.

Rogers, R. (2020). Research note: The scale of Facebook’s problem depends upon how ‘fake news’ is classified. The Harvard Kennedy School (HKS) Misinformation Review. https://doi.org/10.37016/mr-2020-43.

Silverman, C. (2016, November 16). This analysis shows how viral fake election news stories outperformed real news on Facebook. Buzzfeed News. https://www.buzzfeednews.com/article/craigsilverman/viral-fake-election-news-outperformed-real-news-on-facebook.

Suciu, P. (2019, October 1). More Americans Are Getting Their News From Social Media. Forbes. https://www.forbes.com/sites/petersuciu/2019/10/11/more-americans-are-getting-their-news-from-social-media/?sh=715e09aa3e17.

Vosoughi, S., Roy, D., and Aral, S. (2018). The spread of true and false news online. Science, 359(6380), 1146-1151.

Funding

No funding to declare.

Competing interests

No conflicts of interest.

Ethics

No ethical issues.

Copyright

This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided that the original author and source are properly credited.

Data availability

Data available via: Post-Trump Facebook Ecology (23 Mar 2020 - 5 Jan 2021) - Google Drive