Investigating the Information Landscape of the Russia-Ukraine Conflict on YouTube. Comparisons of the emerged content from distinct queries on YouTube surrounding the political conflict

Team members: Anasthasya Mathilda, Danique Vijlbrief, Harneet Bahal, Li Jiang, Tianyi Li

Summary of Key Findings

This paper observes a degree of disparity between the kind of information one can seek to gain on YouTube based on the keywords they use, especially in the context of the Russia-Ukraine conflict. The keyword ‘Ukraine War’ reveals more general news coverage, whereas the keyword ‘Ukraine Military Operation’ reveals a more subjective coverage of the conflict.

1. Introduction

On the 24th of February 2022, Russia started an unprovoked invasion of Ukraine. At the time of writing, Russian troops are still occupying Ukrainian territories and actively attacking both Ukrainian soldiers and civilians. This war has left Ukraine devastated, while Russia, under Vladimir Putin’s policy, isolates itself further from the west.

While western media widely condemn the Russian invasion and refer to the Russia-Ukraine conflict as a ‘war’, Russia avoids this term and rather refers to it as a ‘special military operation’ (McMahon, 2022; Troianovski & Safronova, 2022; Wesolowski, 2022). Russia creates its own narrative about the Ukrainian war and actively spreads this propaganda through state media (Mozur, Satariano, and Krolik, 2022). More than that, on the 4th of March, Putin signed a censorship law which threatens imprisonment for any journalist who deviated from the Kremlin’s portrayal of the conflict in Ukraine (McMahon, 2022). It should be clear that this conflict has reinforced the opposition between the western news media and the Russian propaganda news media.

Naturally, people search for news items to get more information about the subject. One place where people search for this information is YouTube. With more than two billion users, “!YouTube is one of the most dominant sources of online information” (Li et al., 2020, p. 1). YouTube’s dominant role in providing online information comes with an editorial responsibility to ensure that that information is, at least to some extent, based on facts. When it comes to the Russia-Ukraine conflict, this means that YouTube is expected to react in some way to misleading propaganda from the Kremlin. Indeed, YouTube has taken on the role of the dissident platform in this conflict when it took a political stand and deleted 70,000 videos and 9,000 channels that described the conflict as a ‘liberation mission’ (Milmo, 2022). In this way, YouTube has actively worked to moderate misinformation about the conflict.

In this context, it is interesting to investigate how YouTube mediates the Russia-Ukraine conflict and what the information landscape around this conflict looks like on YouTube. This research is based on two keyword queries that represent the two sides of the conflict: ‘Ukraine War’ and ‘Ukraine Military Operation’. These keyword queries are used when collecting datasets with YouTube Data Tools (Rieder, 2015). A quantitative analysis of these datasets is then performed to answer the research question.

2. Initial Data Sets

For this research, we were looking at a total of 9 datasets, in which 1 was the starting point, 6 were analysed in detail, and 2 yielded insubstantial results for this research. All datasets that were observed and used during this research process can be found in this linked folder.

The initial point of interest of our investigation was the first dataset, which stemmed from the Video List dataset that our project leader extracted surrounding the query ‘Ukraine War’, extracted through the Video List Module on YouTube Data Tools (Rieder, 2015) on two separate occasions (December 2022 and January 2023) in which one of the findings showed the discrepancies of the number of videos published per week from the two sessions of data extraction. This finding, combined with the event of YouTube’s removal of videos, led us to obtain from YouTube Data Tools (Rieder, 2015) the following data sets:

1. From the Video List Module, based on search queries of ‘Ukraine War’ (1 dataset) and ‘Ukraine Military Operation’ (1 dataset); Set on iteration 1, dated from 1 February 2022 to 31 December 2022, with the search for each day, timeframe opted in ranked by date. These datasets were the operational groundwork to answer our main Research Question and the subsequent sub-questions, enabling us to observe timely trends, publishing behaviours, and activities of the actors (i.e. channels).

2. From the Video Network Module, based on search queries of ‘Ukraine War’ (2 datasets; video network and channel network) and ‘Ukraine Military Operation’ (2 datasets; video network and channel network); Set on iteration 1, dated from 1 February 2022 to 31 December 2022, ranked by relevance, and set on a crawl depth of 0, and each query generated both video networks and channel networks. The video network enabled us to simulate user activity if they are searching for each query in their watching session and further shows what is being co-watched in the same session. Furthermore, initially, we also obtained the same datasets with a crawl depth of 1, however, through processing the data, the results yielded were also not substantial and thus not used in the findings section of this research report.

3. From the Channel Info Module, using seeds of channel IDs from the most active channels (based on the numbers of videos published, in accordance with the Video List datasets) from both the ‘Ukraine War’ and ‘Ukraine Military Operation’ datasets. These datasets were used for further investigation into the top 20 channels of each query as we noticed distinct categories of actors between the two queries. We were then able to observe the channel age difference and countries of origin between the active actors in each query.

3. Research Questions

The main question that guides this research is: What does the information landscape in the Russia-Ukraine conflict look like on YouTube?

In order to be able to answer this main question, three smaller sub-questions are answered. They are as follows:

1. Which kind of content circulates in the information landscape around the Russia-Ukraine conflict?

2. Who are the actors contributing to this information landscape?

3. Are there temporal dynamics to both the contents and the actors?

For the first sub-question, we hypothesise that there are mostly videos in the category News & Politics for both queries since the Russia-Ukraine conflict is a political conflict that is reported on by news media on a daily basis. However, this category could be less present for ‘Ukraine Military Operation’ due to YouTube ’s removal of channels and videos that spread misinformation about the conflict. Related to the first question, we expect that the actors contributing to the information landscape on YouTube are mainly news media that report on the conflict regularly. Again, this might be different for ‘Ukraine Military Operation’ for the same reasons as above. Our hypothesis for the third sub-question is that there could be changes in the kind of content and actors that might match key events in the conflict.

4. Methodology

Firstly, based on the research question and its subsequent sub-questions, we decided to compare two different queries with possibly opposing views and connotations and obtain datasets for each that will be compared and contrasted in parallel to each other. In choosing the queries, first, we must acknowledge that within our research sub-group, there was no researcher fluent in or is a native speaker of both Ukrainian and Russian; hence we have limited linguistic and cultural context that may lead us to choose terms (i.e. in the native languages) as a query that possibly shine a better light into the landscape of the conflict in YouTube. Consequently and as mentioned in earlier sections, we arrived at the decision to compare and contrast the terms ‘Ukraine War’ and ‘Ukraine Military Operation’ for this research.

We chose ‘Ukraine War’ as a query as it represented the general global news narrative of this conflict, and ‘Ukraine Military Operation’ was chosen as the other query as from preliminary research done on news articles and reports, Russia evaded and even banned calling this conflict ‘war’, and instead referred to the conflict as ‘liberation operation’ or ‘military operation’ (McMahon, 2022; Troianovski & Safronova, 2022; Wesolowski, 2022). Hence these two queries with distinct connotations were chosen in the hope of showing us both ‘sides’ of this conflict and how it is mediated, portrayed, or moderated on YouTube.

As mentioned earlier, all datasets for this research were obtained through YouTube Data Tools (Rieder, 2015). To recap, the datasets used in this research were: Video List from both ‘Ukraine War’ and ‘Ukraine Military Operation’ queries; Video Network from both ‘Ukraine War’ and ‘Ukraine Military Operation’ queries; and the Channel List from the top 20 (most actively posting, based on the number of videos published) channels from both ‘Ukraine War’ and ‘Ukraine Military Operation’ queries.

For the analyses done in the context of the Video List and Channel Info datasets, Google Sheets was used as the main platform to process the data, in particular, the Pivot Table and Charts function were utilised to process and visualise posting trends (monthly), category share, temporal dynamics of the category share, channel activity rank based on the number of published video, channel country of origin, as well as channel age share comparison between the two queries. As for Video Network datasets, the .gdf files obtained from YouTube Data Tools were processed in Gephi using the Force Atlas 2 algorithm. Statistical processes (average degree and modularity) were also run to detect clusters within the network.

5. Findings

Video List Module Findings

Video Quantity

There is a significant difference in the number of videos generated for each of our queries from YouTube Data Tools (Rieder, 2015). The ‘Ukraine War’ query generated 6,578 videos, whereas the ‘Ukraine Military Operation’ query generated 2,667 videos. The gap in the quantity of videos could be a result of YouTube’s removal of channels and videos that were falsely spreading pro-Russia propaganda.

Video Length

The duration of the videos for each query was also analysed for the purpose of this paper. Videos from the ‘Ukraine War’ query are timed between 0 seconds to 3,599 seconds with an average of 666 seconds, or approximately 11 minutes. Each video with the resulting video duration of 0 seconds is due to a YouTube LIVE recording that started at the onset of the conflict and is still ongoing.

Furthermore, the duration of the videos from the ‘Ukraine Military Operation’ query ranges from 2 seconds to 3,567 seconds, with an average of 370 seconds, or approximately 6 minutes.

Video Categories

An analysis of the overall video categories was conducted for each of the search queries. Figure 1 displays the overall dataset generated for the ‘Ukraine War’ query and indicates a share of 73.7% of videos belonging to the News and Politics category. Additionally, the People and Blogs category belongs to 8.4% of the videos for the ‘Ukraine War’ dataset. Due to the nature of the query, the People and Blogs category is deemed important as it could signify vlog-like reporting, citizen journalism or coverage done by political commentators.

Figure 1. Video categories for the ‘Ukraine War’ dataset.

However, Figure 2 displays the video categories for the dataset obtained from the ‘Ukraine Military Operation’ query. The most prominent category is People and Blogs at 39.5%, followed by News and Politics at 38.1%. Based on this finding, it can be deduced that the conflict has been equally covered by news media outlets and individuals or political commentators.

Figure 2. Video categories for the ‘Ukraine Military Operation’ dataset.

Moreover, an analysis of the temporal dynamics was conducted for the video categories of each of the queries. Figure 3 displays the category share per month for the ‘Ukraine War’ and displays a steady dominance of the News and Politics category over time. Additionally, an increase in the People and Blogs category can also be observed, which may denote an increase in live coverage, commentaries or citizen journalism.

Figure 3. Category share per month for the ‘Ukraine War’ dataset.

Furthermore, the category share per month for ‘Ukraine Military Operation’ can be observed in Figure 4. While the majority of the videos were categorised under News and Politics at the beginning of the conflict, a gradual change can be observed over time. There was a brief period of time wherein the People and Blogs category dominated the category share, and this could be attributed to YouTube ’s removal of thousands of channels and videos that were spreading misleading pro-Russia propaganda. However, after this period, the two primary categories appear to be equally contributing to the information landscape.

Figure 4. Category share per month for the ‘Ukraine Military Operation’ dataset.

Actor Analysis

In order to ascertain the most prominent contributors to the information landscape of the Russia-Ukraine conflict, an overview of the channels that uploaded content pertaining to our two queries was analysed. Table 1 depicts a difference in the top channels for the queries ‘Ukraine War’ and ‘Ukraine Military Operation’, wherein the first query appears to be dominated by official news outlets, and the second query appears to be dominated by individuals.

Table 1. A comparison of the publishing actors for ‘Ukraine War’ and ‘Ukraine Military Operation’.

Furthermore, an analysis of the channel age for the top 20 channels of both queries was also conducted. The results indicate that due to the top channels being official news outlets, the channels for ‘Ukraine War’ tend to be older. However, the channels for ‘Ukraine Military Operation’ seem to be comparatively younger, with 40% of them having started in 2022.

Figure 5. Channel age share for the top 20 channels in the ‘Ukraine War’ query.

Figure 6. Channel age share for the top 20 channels in the ‘Ukraine Military Operation’ query.

Video Network Findings

The video network for both queries, ‘Ukraine War’ and ‘Ukraine Military Operation’, were analysed to discover the different ways the information landscape pans out on YouTube. The comparison of the video network analysis for both queries resulted in the discovery of a large cluster for the ‘Ukraine War’ query featuring video titles like ‘Savage War Crimes’, ‘The KGB and Why It Is Feared’, along with titles featuring dictators like Hitler and Kim Jong-Il. The presence of video titles depicting this sentiment is largely absent in the video network for the ‘Ukraine Military Operation’ query.

Figure 7. Cluster from the video network for ‘Ukraine War’.

Furthermore, the analysis also revealed the presence of a large cluster of videos in the video network for ‘Ukraine War’ that feature live camera footage, HD live streams and on-ground recordings of the conflict. Moreover, the same occurrence of live footage videos was largely absent in the video network for the ‘Ukraine Military Operation’ query.

Figure 8. Cluster from the video network for ‘Ukraine War’.

Channel Network Findings

The channel network for ‘Ukraine War’ intrinsically links to more news media outlets such as Sky News, BBC and CNN, as compared to the channel network for ‘Ukraine Military Operation’, which featured channels such as ‘War Leaks Military Blogs’ and ‘Divine Justice’. However, channels for news media outlets were also present in the ‘Ukraine Military Operation’ channel network.

We also collated the channel IDs of each channel from the channel network for ‘Ukraine War’ and ‘Ukraine Military Operation’ into the channel network module of the YouTube Data Tools. Then, the generated dataset was imported into Gephi to detect the presence of any correlation between the channels for ‘Ukraine War’ and ‘Ukraine Military Operation’. There was no linkage discovered which could also signify the difference in the way different channels presented their coverage of the Russia-Ukraine conflict.

6. Discussion

YouTube provides the audience with unlimited choice and access to produced political content (Munger & Phillips, 2022, p. 188). In the current conflict between Ukraine and Russia, YouTube has been perceived as an alternative place of news consumption, a fertile ground for user-generated content that may arguably give a relatively objective overview of the conflict from different perspectives. However, in addition to producers that produce and upload content, YouTube as the mediator and governor, plays a significant role in displaying the information landscape.

By mapping out the existing content overview of the Russia-Ukraine conflict with conceivably opposing queries of the two sides, there is an intuitive presentation of YouTube’s moderation of content. In detail, the distinct gap in video numbers between ‘Ukraine War’ and ‘Ukraine Military Operation’ may be a result of the aforementioned YouTube’s removal of channels and videos that spread pro-Russian propaganda, coinciding with Putin's Censorship Law on domestic Russian media users. Additionally, apart from the removal and moderation of content, our study’s findings illustrated how YouTube ’s automated recommendation, which matches audience taste to content (Munger & Phillips, 2022, p. 187), facilitates different viewer experiences in terms of the information landscape. With different English queries, users are fed very different content from different video categories, producer types and video topics.

Furthermore, with a large user base and a high degree of free expression, YouTube has become a significant place for political expression (Halpern & Gibbs, 2013). An overview of the information landscape on YouTube regarding certain issues is necessary for verifying YouTube ’s governance over content, identifying potential bias and breaking the filter bubble. Traditional media, such as TV channels and newspapers, have long been criticised for their discursive nature and sometimes biased representation. At a time when people are turning to YouTube for alternative political information consumption, it is important to realise that with YouTube’s control over its platform activity and recommendation system, ‘what is out there on YouTube’ is also highly mediated. It is undeniable that YouTube does provide room for discussion. By identifying the key actors in two different queries, we see that YouTube helps validate and spread alternative arguments from individual actors (i.e. ‘Ukraine Military Operation’) complementary to dominant media organisations (i.e. ‘Ukraine War’). However, users should also avoid falling into the echo chamber created by the recommendation algorithm.

7. Conclusion

In order to map out the information landscape of the Russia-Ukraine conflict on YouTube, we have developed two queries and examined their datasets. By quantifying the video content and examining the key actors, we shone a spotlight on YouTube ’s role in terms of content moderation and, to a certain extent, evaluated YouTube’s performance as a site for alternative political expression. As growing political discussion and consumption have been grounded on the platform, we argue that the user consumption of information on YouTube is mediated by the platform and its algorithm. While YouTube does offer the opportunity to discover content with differing perspectives, users should avoid falling into the echo chamber.

In our attempt to capture different stances of the information landscape, we have used representative English queries. However, a limitation here is the use of a single language, we assume that queries in native languages (i.e. Ukrainian or Russian) may provide more distinct results, or may uncover more substantial findings. Furthermore, by using iteration 1 to run both queries, we have obtained a dataset that is large enough to answer our research question. While at the same time, other researchers have obtained larger datasets on the Ukraine War with more iterations, which may yield a more thorough information landscape.

Another important point is that the Russia-Ukraine conflict is still ongoing, while our research captured the data of the last 11 months, there are possibilities, especially if there are any other developments of the conflict, of dynamic shifts or trends emerging on YouTube as the conflict evolves further.

Considering the points above, future research is suggested to utilise different languages, especially Ukrainian or Russian, to observe and investigate more ‘native’ or culturally relevant landscapes in relation to the language. Furthermore, in the context of understanding YouTube ’s role in moderating context, future research may also investigate the removed videos, if possible, and see if there are any common grounds or characteristics within the content that is moderated by YouTube. Lastly, frequent runs of the same queries over time may also further reveal if there are any other acts of moderation done by YouTube and perhaps whether the removal of content is systematically done in a given period as an act of maintenance or if any actions, events, or regulations trigger the moderation.

8. References

Halpern, D., and Gibbs, J. (2013) Social media as a catalyst for online deliberation? Exploring the affordances of Facebook and YouTube for political expression. Computers in Human Behavior. 29(3), 1159–1168.

Li, H., Bailey, A., Huynh, D., & Chan, J. (2020). YouTube as a Source of Information on COVID-19- A Pandemic of Misinformation? BMJ Global Health, 5, 1-6.

McMahon, R. (2022, March 18). Russia Is Censoring News on the War in Ukraine. Foreign Media Are Trying to Get Around That. Council on Foreign Relations. https://www.cfr.org/in-brief/russia-censoring-news-war-ukraine-foreign-media-are-tryin g-get-around

Milmo, D. (2022, May 22). YouTube removes more than 9,000 channels relating to Ukraine war. The Guardian. https://www.theguardian.com/technology/2022/may/22/youtube-ukraine-invasion-russi a-video-removals

Mozur, P., Satariano, A., & Krolik, A. (2022, December 15). An Alternate Reality: How Russia’s State TV Spins the Ukraine War. The New York Times. https://www.nytimes.com/2022/12/15/technology/russia-state-tv-ukraine-war.html

Munger, K. & Phillips, J. (2022). Right-Wing YouTube: A Supply and Demand Perspective. The International Journal of Press/politics, 27(1), 186–219. https://doi.org/10.1177/1940161220964767

Rieder, B. (2015). YouTube Data Tools (Version 1.30) [Software]. Available from https://tools.digitalmethods.net/netvizz/youtube/ Troianovski, A., & Safronova, V. (2022, March 4). Russia takes censorship to new extremes, stifling war coverage. The New York Times. Retrieved January 17, 2023, from https://www.nytimes.com/2022/03/04/world/europe/russia-censorship-media-crackdow n.html

Wesolowski, K. 2022. Fake news further fogs Russia's war on Ukraine. DW. Retrieved January 17, 2023, from https://www.dw.com/en/fact-check-fake-news-thrives-amid-russia-ukraine-war/a-61477 502

-- BernRieder - 30 Jan 2023