The European Refugee Crisis in the Dutch Media Sphere

Team Members

Andrea Benedetti, Silvia Geurts, Judith Harmsen, Monique Hindriks, Anne Jochems, Lieve Keizer, Thomas Poell, Bernhard Rieder, Leonie Ruizendaal, Puck Stallinga, Kirsten Zwanenburg

Contents

Summary of Key Findings

Subproject one focused on the Facebook pages of Dutch news shows EenVandaag and Nieuwuur, analyzing user reactions in particular. The research showns that people tend to express themselves more negatively when problems or events occur within their own environment. Regarding the refugee crisis, we can conclude that the overall sentiment on Facebook is angry, and therefore quite negative.

Subproject two focused on two emblematic events in during the European Refugee Crisis, the death of Alan Kurdi and the mass sexual assaults in Cologne on new year's eve, and analyzed both the content of Dutch news show EenVandaag and reactions on its Facebook page. The two studied events indeed seem to represent turning points in the media debate. Not only did EenVandaag give double as much attention to the refugee issue in the episodes after the two events, there also is a clear shift in the vocabulary before and after, especially in the case of the new year's eve in Cologne. Facebook reactions, on the other side were particularly sensitive to the death of the young boy and the shocking photograph that accompanied it.

Subproject three shows a wider topic focus in the news program EenVandaag when compared to Facebook comments. Dutch politics and Dutch identity are much more of a topic on Facebook than in the TV news pogram, while the latter reserves more space to the international dimension of the crisis. The transcripts also contain much more factual descriptives and fewer adjectives such as "good" and "better" than the comments.

1. Introduction

The European Refugee Crisis, which reached public attention in 2013 and continues to this day has been a defining political event in European and national politics. Initially triggered by war and political instability in Syria, Afghanistan, and a number of other countries, the refugee crisis connects to a range of issues, from disputes on immigration to the rise of populist politics and European integration. While clearly a matter of transnational importance, the crisis has provoked intense debates in national media spheres, the Netherlands being no exception.

While highly controversial subjects are regularly studied by media scholars, only few projects have been able to research how the different streams of an emerging "hybrid media system" (Chadwick 2013) that includes traditional media institutions (newspapers, television, etc.) as well as social media platforms such as Facebook, Twitter, or YouTube. Due to its high levels of visibility and controversiality the European Refugee Crisis constitutes an interesting opportunity to study different media streams transversally and, in particular, to compare the (once) dominant electronic medium, television, with different social media in terms of coverage, framing, and tone. This project is, at least in part, driven by broader availability of audiovisual data through the CLARIAH media suite. While Television has been investigated in many different ways by media scholars in the past, access to rich metadata and transcripts has been difficult to obtain.

The European Refugee Crisis constitutes a perfect example for gauging the potential of such data in the context of cross-media research as well as research seeking to cover longer timespans.

2. Broad Research Questions

The initial focus of this research project concerns the relationships, similarities, and differences between Dutch television coverage and social media services, in particular Facebook and Twitter.

Research shows that social media communication on current affairs issues is often triggered by television news coverage. Indeed, early findings from our datasets suggested that television coverage of the crisis did not follow the same attention cycle as social media debates. One of the goals is thus to investigate these relationships in more depth.

At the same time, research also indicates that social media communication includes a wider diversity of perspectives, actors, and types of talk than mainstream news coverage. While these forms of diversity are of particular concern to this project, we are equally interested in comparative language use and in the subtler differences between words and their connotations, including their ideological and affective dimension. This includes the question how we can use social media data to better understand the relationship between media production (e.g. of news programs) and audience reception, as it express itself through gestures such as liking, retweeting, or commenting.

These broader concerns frame a number of more specific research questions:

• What are the commonalities and differences between the different media channels in how they discuss the European Refugee Crisis?
• What is the role of public broadcasters in the coverage of the crisis and how do they mediate the debate beyond their own coverage on social media platforms?
• How can we connect and compare the different media channels between each other?
• What are the respective attention cycles, also with respect to the political process as it relates to parliamentary debates?

The three subprojects we eventually settled on for this Winter School approach these questions from specific directions.

3. Initial Data Sets

The project relies on a combination of datasets, including Television transcripts, Facebook Pages and data captured from Twitter trough keywords. The Twitter dataset represents a the main constraint: since historical tweets need to be purchased, we relied on a collection started in 2013 by researchers from the UvA ’s Digital Methods Initiative using DMI-TCAT (Borra & Rieder 2014), that covered hashtags and keywords ([#nos, 1vandaag, brandpunt, buitenhof, dwdd, eenvandaag, jinek, journaal, kvdb, kvdbtv, nieuwsuur, nosjournaal, politiek24, pownews, vandaagdedag, zembla]) representing Dutch television news programs, namely EenVandaag, Brandpunt, Buitenhof, De Wereld Draait Door, Jinek, Journaal, Knevel en Van Den Brink, Nieuwsuur, PowNews, Vandaag de dag, and Zembla. The collection for this dataset was started on May 13, 2013 and counted 8,5M tweets at the end of May 2018. For reasons of convenience and symmetry we set our observation period from May 15, 2013 to May 15, 2018.

Starting from this selection, we identified the news programs in the NISV's (Netherlands Institute for Sound and Vision) Television collection via the CLARIAH media suite. While this collection contains the usual metadata fields for all of these programs, full transcripts were only available for a part of them. We thus made use of the Media Suite’s emerging capacities in Automatic Speech Recognition (ASR) to create textual representations of their spoken content. Due to the heavy computational requirements for text processing, we used individual news programs rather than the full set of transcripts for most of our explorations.

In a final step, we looked for the Facebook Pages and of the news programs in question. In most cases, we were able to identify a Page that corresponds to the TV program in question; but for three cases (Knevel en Van Den Brink,NOS Journaal, and Vandaag de dag), we had to fall back on the Page of their channel. In these cases, one strategy would be to filter the posts that concern the shows in question. In all cases, we were able to gather large amounts of historical data, going back to the Page’s starting dates. But two caveats apply. First, we are limited to "a maximum of 600 ranked, published posts per year" according to Facebook's developer documentation. And second, comments have to be taken with a grain of salt, both because of privacy limitations and because the large number of comments on certain pages forced us to fall back on Facebook’s “top comments” feature, which again provides a ranked selection.

The following chart gives a broad overview of the overall attention the issue received in TV news, Twitter, and Facebook. We see the relative number of "items" - TV program episodes (grey), Facebook posts (dark blue), tweets (light blue) - over five years, as well as the engagement numbers (comments, likes, and shares) relevant posts received.

While the overall issue dynamics are comparable, we notice that the TV news programs follow a less "spiky" curve: the terms refugee (vluchteling), migrant (migrant), and asylum seeker (asielzoeker) appear in a relatively larger number of content units both before and after the main attention peaks in 2015 and early 2016.

4. Subproject 1: The Language of Emotions

Introduction

The refugee crisis has been a highly discussed topic in the Dutch mediasphere and on social media for the past years. Almost every television news programme and newspaper reported on the issue and shared snippets of this discussion on their social media platforms. Social Networking Sites (SNS) such as Facebook are a popular environment to discuss and distribute news, both for news organizations and individual users. Mainly with controversial topics opinions are exchanged and debates arise in the comment section (Basile et. al. 12). Since March 2016, Facebook offers the user a set of emotional responses that one can give to a post. These so called reactions are designed as moving emojis and cover a large part of the emotional spectrum; "love", "haha", "wow", "sad" and "angry" are implemented in the feature. They can be seen as an addition to the classic Facebook like, which has grown into an iconic characteristic of the platform. It is the "love" button that is clicked on the most in the first year, as it accounted for over half of the reactions used (Molloy).

The introduction of the reactions meant a big change to the interface and algorithm that controls the newsfeed. Facebook released a statement at the time, in which they said that if people took the time to place a reaction on a post, instead of a "simple" like, they would most likely like to see more of those posts on their newsfeed. The feature obviously provides Facebook with even more and specified data on their users, which will eventually result in better customer targeting, and therefore bigger profit (Russell).

Besides that social media is used to share information, it is also an opportunity for people to express their emotions. The newly added reactions feature can say something about the overall sentiment a user holds at a certain topic in a Facebook post (Tian et. al. 11). Especially in the case of controversial topics it may be useful to understand which subtopics (within the larger controversial topic) people talk about the most. As Basile et. al. indicate in their article: "Journalists and news agencies may pay additional attention in the framing of certain news, government officials and policy makers may be more aware of the issues involved in specific laws, and social media managers might be more careful, in order to avoid the spreading of hate speech" (12).

This research aims to understand why and how Facebook users use the relatively new reactions feature regarding the refugee crisis, and what language they use to endorse their reactions in the comment section. By doing this, the goal is to find out which subtopics people are most concerned about within this larger topic of the refugee crisis.

Methodology

Due to time- and other limitations we only looked at two Dutch news television programmes: EenVandaag and Nieuwsuur. We chose these programmes, because they are placed in the middle of the political spectrum, and broadcast their shows daily. We used the Facebook data of both the news organizations as corpus for our research and specifically looked at the data between March 2016 (when the reaction feature was launched) and October 2018 (a timespan of 2.5 years).

After data capture via Netvizz (Rieder 2013), we had an excel sheet with the ‘fullstats’ of the Facebook page at our disposal, which included the statistics on each posts, i.e. the frequency of the different reactions used per posts. In addition, we had access to an excel sheet which included all the comments per facebook post. By using the filter function in excel, we managed to easily filter out the posts of our interest.

First, we looked at the average use of the reactions on all the facebook posts on the EenVandaag and Nieuwsuur page, within our timeframe. After that, we filtered this on the three keywords that are most used in Dutch media when commenting on the refugee-crisis: migrant (migrant), vluchteling (refugee), and asielzoeker (asylum seeker). These are considered the most common and neutral words to address this issue. We compared these averages to each other to find out which reaction was used more, less or the same on posts regarding the refugee crisis. In addition, we made a graph of the use of the feature over time, to see if the use of it is increased.

After that, we made a top five of posts for each distinct reaction. In this way, we were able to find out which posts people were the most sad, angry etc. about. To take it a step further, we filtered the excel sheet with the comments on these specific post and created a text file of all the comments for each top five posts, per reaction. These text files were put into an online software called Voyant Tools. This tool provided us a list with the most used words in a specific text file. For the sake of readability, general words that are not relevant (e.g. you, I, me, we, the, it) were filtered out of the lists. Also, verbs that are meaningless without the sentence they were placed in, were left out. By analysing these lists we were able to see which topics are discussed the most, under which reaction.

Findings

This section describes the results we found within the set time frame (March 2016 till October 2018) on the Facebook pages of EenVandaag and Nieuwsuur. EenVandaag posted 1944 posts on Facebook, with 42 posts that contained the words refugee/migrant/asylum seeker (2,2%). The Nieuwsuur page posted about refugees/migrants/asylum seekers 74 times out of 1797 of total posts (4,1%).

Average use of reactions

The calculation of the averages shows us that compared to the overall use of the buttons the button ‘angry’ is used way above average on topics that are about refugees, on both the EenVandaag as Nieuwsuur Facebook pages. The buttons "love", "sad", and "angry" appear to be the most popular buttons for average use.

Use of reactions on filtered posts

By looking at the use of the reactions we can see some peaks over time, and can detect a small increase on the use of the buttons on the Nieuwsuur page (fig. 3). Only "haha" seems to be a straggler. Besides that, we see the most peaks in the use of the "angry" button.

To narrow and specify our findings, we focussed and elaborate only on the top five peaks per reaction in the above graph. By looking at these top five peaks, we were able to find out which topics resulted in the most reactions. In the discussion section these topics will be discussed.

This graph shows the use of reactions on the filtered EenVandaag posts. We can identify three relevant peaks that stand out during our time frame. The first peak takes place in March 2016. In this period, the "love" button was used significantly more than other reactions. The post that received that many responses was about a column the Dutch writer Kader Abdolah wrote about the refugee crisis. The next day, a post about a chef that hands out fruit to refugees in Greece also got a significantly bigger amount of "love" reactions compared to the other posts. The second peak is an obvious peak of "angry" reactions during July and August 2016. Respectively, 54, 41 and 132 "angry" reactions were placed in response to posts about the mass-assault and aftermath of the event in Cologne during new year’s eve in 2016, and other situations where migrants caused troubles. Again, the third peak shows an increase in "angry" reactions, about an happening were refugees caused trouble.

Connect to the comments

To see if the comments correspond with the overall sentiment of the post ("angry", "sad", etc.) we compared the language used in the comments to the reactions used. We used the top five posts for every reaction. To concretize these findings, we created a word-map (see fig. 7).

As can be seen in the lists above, the same words appear under multiple reactions. This is visualized in the bubble-map for EenVandaag. For example, the word "kinderen" (children) is ranked high under the "love" and "sad" reactions (see fig. 8). Also the word "vluchten" (fleeing) is associated with both "love" and "sad" reactions. If one looks at the other side of the emotional spectrum, one can see that terms such as "gelukszoekers" (luck-seekers; negative term used to describe economic migrants) and "VVD" (Dutch right-wing political party) are used in relation to "angry" and "wow" reactions. The visualisation also shows that words like "mensen" (people), "land" (country) and "Nederland" (The Netherlands) come across all reactions.

As can be seen in fig. 9, the words are much more clustered and categorised. Words that get mentioned in relation to "angry" reactions are "UWV" (government body that is in charge of benefits), "uitkering" (benefits) and "Polen" (Poland). Looking back at the top five "angry" posts, the words match the text used in the posts itself. On the other side, words like "democratie" (democracy), "Hongarije" (Hungary) and "iedereen" (everybody) are often used in relation to "love" and "wow" reactions.

Discussion

Firstly, we have seen a peak in the use of the angry button when the post is about refugees, migrants or asylum seekers. This confirms the theory that social media can work very polarizing. However, it does not align with the statement that the "love" button is the most used button. Statistically this can be true, but practically it does not say that much about the influence of the feature. Judging by this statistics, one can conclude that the reaction feature brought a "positive vibe" to the platform, whilst with controversial topics, like the refugee crisis, it seems to be the other way around.

Secondly, it is important to point out that the reaction feature seems to be used more over time. This is probably because people first have to get used to the new feature and it takes some time before people actively start using it. Therefore, a logical consequence is that the topics that are marked as the most angry are posts about events that happened lately and are not the issues of two years ago. However, in the graphs we also saw peaks in the use of the reactions when the feature was just launched. For example, in the Nieuwsuur graph, in June 2016 we see a peak involved "sad" reactions that were placed under a post that describes the situation of a young refugee trying to contact his family for days. The second peak-moment exists of two peaks in the "love" and "wow" buttons in September 2016. This is an interesting peak-moment as they are both about the same post, namely a post about Hungary that does not want to receive migrants and refugees anymore because it would harm the hungarian identity. It is also interesting because of the high frequency of "love"; this could indicate that people agree with Hungary and think a harsh policy is the right way to approach this problem. The "love" button is used here whilst it is not necessary a positive topic, as we see at the other topics were ‘love’ is used the most. The last peak took place around September 2018. This was an ‘angry’ peak about the fraud scandal with the uwv, were a lot of labor migrants still receive benefits while they are living and working in their home country again.

Finally, from this research, we can conclude that it seems that if there are issues around this refugee topic that do not occur in the Netherlands and therefore not concern them, people mostly feel sad about it and feel pity for the refugees. Also "love" is remarkably used on posts about events happening in foreign countries and people who set up charity actions or stood up for refugees. On the other hand, when refugees are involved with issues in the Netherlands or Europe, as the uwv scandal, refugees squatting houses in Amsterdam or migrants who easily travel illegally through Europe, it seems that people switch sentiment and become more angry about the topic. In other words, when it is a remote issue people feel pity for the refugees, but when they come too close and cause trouble, people are angry at them.

As described in the discussion section above, the use of reactions differs when looking at time-period, Facebook page, and nature of the post. We can conclude that people are polarized on this subject. Polarization is not a new phenomenon on the internet and social media, but, with the limitation of just a "like" or a "share", one had to write and express their true feelings and emotions via words. Nowadays, people are getting used to and preferring to expressing their emotions in minimalistic and quick ways (Pool & Nissim 1). Next to emojis, the Facebook reactions offer the user an easy way to express their feelings, even though this may not always happen in a balanced perspective. For example, there is no ‘slightly annoyed’ button, so people may go for an ‘angry’ reaction, even though they are not that angry at all.

This research has shown that people tend to express themselves more negatively when problems or events occur within their own environment. Regarding the refugee crisis, we can conclude that the overall sentiment on Facebook is angry, and therefore quite negatively. Looking back at Larsson (138), who said that people are more likely to share what upsets them, rather than to share what makes them happy, this should be taken in regard when looking at this study. If people express more negative than positive feelings on social media in general, this could also be the case with the refugee crisis debate.

If we truly want to understand and examine this topic, further research should be conducted. This research could look into the effect (and maybe even the cause) of the reactions on polarization within the topic (e.g. does the reaction feature increase the polarization on Facebook?). This could also be expanded to different topics, using different Facebook pages and keywords. This study will function as a basis for research on reactions and language used in relation to those reactions.

5. Subproject 2: Symbolic Events

Introduction

This subproject looked at the question how widely-mediatized, symbolic events affect both media coverage and larger debates about the refugee crisis. Here, we have looked at two moments we considered as ‘turning moments’ during the refugee crisis. The first one concerns Alan Kurdi, a three-year-old Syrian boy whose image made global headlines after he drowned on 2 September 2015 in the Mediterranean Sea. He and his family were Syrian refugees trying to reach Greece amid the European Refugee Crises. Photographs of his body were taken and quickly spread around the world, leading to an outpouring of reactions. The second moment concerns the mass sexual assaults of 1,200 women by 2,000 men on New Year’s Eve in Cologne. These events were linked to the influx of refugees.

In this research we tried to measure if there is a change in the public discussion regarding the refugee crisis in 2015-2016, according to the Dutch news broadcast show EenVandaag, both on terms of coverage and user reactions on its Facebook page. Have these 2 moments changed the tone and the vocabulary of the way the refugee crisis is discussed in mainstream media and online?

Methodology

We started by separating the two moments by month: one month before the incident and one month after the incident. We chose this time frame because of the limited time for working on the project. It’s easier to look at one month at the time and both events were at the beginning or end of that month. Thus, we got the most reliable results by analysing the whole month.

We used the program LineMiner to analyze both the transcripts of EenVandaag and Facebook comments. With this programme we were able to look at the most common words that EenVandaag used in their reporting style. We did this for both crucial moments, four months in total. After that we searched for the episodes of EenVandaag that mentioned the refugee crisis (searched by dutch terms of migrant, vluchteling and asielzoeker). Within these transcripts we manually extracted the news segments that actually conerned the refugee cricis. We put all of these relevant transcripts into the tool TextAnalysis, which allowd us to extract the most frequent bigrams. Bigrams are words that often co-occur together and are generally considered to deliver more meaningful insights than single word frequencies.

For Facebook comments, we selected posts mentionning the refugee crisis and analyzed both the changes in the most common words and overall reaction numbers.

Findings

The following visualisation shows bigrams before and after the death of Alan and the spread of his photograph throughout the news sphere.

In the middle, one can see the bigrams that are used in both time periods, for example the name of Angela Merkel. She was one of the key figures in the refugee crisis. On the "before" side on the right, we see bigrams that are most used in items about refugee crisis like Europe, helping, Athens in Greece and about the German state Saxony where public opinion was particularly opposed to taking in refugees. This images shows a general approach of the crisis. The left cluster (after) shows more bigrams in discussing the amount of refugees, and that this is growing. But the bigrams also show that media put a stronger emphasis on children and women after the photo of Alan.

The second visualization shows the results for the events in Cologne. We can see that the reporting style on the crisis has developed in the meantime. In the reporting of EenVandaag a more "human" approach is visible. There is also a focus on Jesse Klaver, a left-wing politician who made welcoming refugees a political agenda point.

In the "after" cluster on the right, the focus shifts to the frame of New Years Eve in Cologne and stresses the incidents of sexual assault. Common words are "Cologne police", "back on the boat", "criminal fence", "german social media" and the pepperstray that was hot issue in Germany as a means of protection from sexual assault.

In the last visualisation shows the respective comment spheres on Facebook. The rectangles show the number of reactions on posts and the most frequently used words. In both cases, we can see that references to the specific events ("Aylan" and "women") leave an imprint on the debate. Alan's death, in particular, seems to be connected to a large growth in the number of reactions.

Discussion

While hard causality is hard to attributed, the two studied events indeed seem to represent turning points in the media debate. Not only did EenVandaag give double as much attention to the refugee issue in the episodes after the two events, there also is a clear shift in the vocabulary before and after, especially in the case of the new year's eve in Cologne. Facebook reactions, on the other side were particularly sensitive to the death of the young boy and the shocking photograph that accompanied it.

6. Subproject 3: Language in TV Transcripts and Social Media

Introduction

This subproject was also focused on the public discourse about the refugee crisis in Europe. We looked at the television transcripts of EenVandaag from 2014 till 2017 and the Facebook comments from EenVandaag in those same 4 years. Our questions were: What associations are made when talking about the European Refugee crisis? How do these associations change over time? Is there a difference in discourse used in text if you compare television and the comments of the Facebook users. And when did specific words as “vluchtelingencrisis” first appear? To answer this question we have looked at what words are often used in combination with migrant, asielzoeker/asylum seeker and vluchteling/refugee in news reporting and on Facebook.

Methodology

This subproject relied mainly on LineMiner to analyze both the most common (frequency) and specific (tf-idf) word appearing in the four years under scrutiny, both in TV transcripts and Facebook comments. Results were visualized using RankFlow and custom-made graphics.

Findings

The first visualization shows the most common words associated with the terms "migrant", "asylum seeker" and "refugee" in the coverage of EenVandaag.

First of all you can see a big peak in the sheer frequency of terms co-appearing in 2015. One can of course say this was "the year of the refugee", but it is interesting to see because it is such a high peak and television programs did not stop reporting about refugees after 2015. "PVV" and "Wilders" (PVV is a right wing political party, and Geert Wilders there leader) stand out, as these terms do not appear before 2015, but are really high in 2015. The whole discussion seems to politicize here, all words that have to do with politics come from this period. There is also a big focus on "us" as the Netherlands, and on Dutch politics and Dutch solutions.

The second visualization focuses on the language in Facebook comments.

In the Facebook comments, the most frequent words are more similar over the four years, but they change quite a bot in terms of ranking.

In addition to the associations over time we also took a look at the most common associations in EenVandaag transcripts and Facebook comments in general. To compare the news coverage with the discourse in the comment section, we divided terms into categories as "(Dutch) politics", "international players", "Dutch identity", "descriptors", "adjectives or other" and "verbs".

From this we can conclude that Dutch politics and Dutch identity is much more of a topic on Facebook than it is in the transcripts, as one can see from the size of the bubble. Common associations made in Facebook comments are PVV, Wilders, VVD, Netherlands and "people", while these do not or barely appear in the transcripts.

On the other hand, international players such as Greece, Turkey, Syria and Germany are mentioned a lot in EenVandaag transcripts, while people commenting on Facebook only or mostly talk about what the refugee crisis means for "us" - people in the Netherlands. Furthermore we can conclude that the transcripts contain much more factual descriptives and fewer adjectives such as ‘good’ and ‘better’ than the comments.

Last, we expected to see more emotional words in the Facebook comments focused on fear or terrorism, but this was not the case. An explanation could be that emotions are mostly expressed through emoji buttons as the like-, angry- or sad-button. But of course this is just an assumption.

7. Overall Conclusions

Next to the individual findings, we can generalize that rather than online vs. offline or “traditional” vs. “social” media we see the emergence of a “hybrid media system” (Chadwick) marked by commonalities and differences that do not (necessarily) align with media types. But one general difference is that social media spaces seem to have higher “contrast”: they are spikier in terms of temporal distribution and more narrow in terms of subject focus (e.g. the strong connection with Dutch politics and local perspectives). Working with textual material allows for comparison over channels where metrics are sparse but also raises the uncomfortable question about conditions of enunciation and their implications for analysis.

8. References

Basile, Angelo, Tommaso Caselli and Malvina Nissim. 2017. "Predicting Controversial News Using Facebook Reactions." Proceedings on the Fourth Italian Conference on Computational Linguistics, 11-12 December 2017, Rome. Turin: Accademia University Press.

Borra, Erik, and Bernhard Rieder. 2014. "Programmed Method: Developing a Toolset for Capturing and Analyzing Tweets." Aslib Journal of Information Management 66(3): 262-278.

Chadwick, Andrew. 2013. The Hybrid Media System. Oxford, New York: Oxford University Press.

Larsson, Anders. 2014. “Everyday elites, citizens, or extremists?”. Medienkultur: Journal Of Media & Communication Research 30(56): 61-78.

Molloy, Mark. 2017. "Why should you 'love' instead of 'like' the Facebook posts that really matter to you." The Telegraph, 28 February 2017, https://www.telegraph.co.uk/technology/2017/
02/28/should-love-instead-like-facebook-posts-really-matter/. Accessed 10 January 2019.

Pool, Chris and Malvina Nassim. 2016. "Distant supervision for emotion detecting using Facebook reactions." Proceedings of the Workshop on Computational Modeling of People’s Opinions, Personality, and Emotions in Social Media, 2016, Osaka. N.p.: n.p., 2016.

Rieder, Bernhard. 2013. "Studying Facebook via Data Extraction: The Netvizz Application." In: Proceedings of the 5th Annual ACM Web Science Conference. New York: ACM, 346-355.

Russell, Jylian. n.d. “Facebook reactions: what they are and how they impact the feed”. Hootsuite, https://blog.hootsuite.com/how-facebook-reactions-impact-the-feed/. Accessed 9 January 2019.

Tian, Ye et. al. 2017. "Facebook Sentiment: Reactions and Emojis." Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media, 3-7 April 2017, Valencia. N.p.: Association for Computational Linguistics, 2017.

-- BernhardRieder - 10 Jan 2019