Retraining the machine: Addressing algorithmic bias

Digital Methods Summer School 2018

---

---

Digital Methods Summer School
2–13 July 2018

Everyday location
Digital Methods Initiative
University of Amsterdam
Turfdraagsterpad 9, 1012 XT Amsterdam
https://wiki.digitalmethods.net/Dmi/SummerSchool2018
bit.ly/dmi18-ss-call

Important Links

Shortlink	Description
bit.ly/dmi18-ss-call	Call for participation and general information.
bit.ly/dmi18-ss-welcome bit.ly/dmi18-ss-welcome-folder	The main welcome package document and folder as sent to you by email. Contents: Welcome note, day-to-day schedule, maps of venue locations, coffee, and lunch places, preparations, and further instructions. The welcome package folder has the reader, face book, and other linked documents.
bit.ly/dmi18-ss-schedule	The day-to-day schedule as included in the main welcome package document.
bit.ly/dmi18-ss-projects	Tentative descriptions of projects to be pitched on the opening days (Monday afternoons).
bit.ly/dmi18-ss-reader bit.ly/dmi18-ss-reader-folder	The reader and a reader folder with PDF documents.
bit.ly/dmi18-ss-facebook	The face book with a short bio and a profile picture of all staff, facilitators, keynotes, designers, and international participants.
bit.ly/dmi18-ss-locations	Google My Maps with venue locations, nearby bars and restaurants, coffee, and lunch places.
bit.ly/dmi18-ss-template-gdoc bit.ly/dmi18-ss-template-wiki	The templates to be used for writing up research project reports. Formats: Google Docs and DMI Wiki (please find further use instructions in the main welcome package document.)
bit.ly/dmi18-ss-notes	The collaborative note-taking document that we will use for keynotes on the opening days (Mondays).
#dmi18	Official Twitter hashtag.

Projects (Work Sessions)

Week number	Project number	Project title	Project room(s)	Project wiki page(s)
1	1	Remembering and forgetting the Syrian war on Twitter hashtag and Facebook page publics	BG1 0.04	Remembering and forgetting the Syrian war on Twitter hashtag and Facebook Page publics
1	2	One game, multiple visuals?	BG2 0.08	One game, multiple visuals?
1	3	Collaborative research practices and bullshit accumulation tactics on 4chan /pol/	BG1 0.13	Collaborative research practices and bullshit accumulation tactics on 4chan /pol/
1	4	Decoding the decoders	BG1 0.16	Decoding the decoders
1	5	App stores and their bias	BG2 0.02	App stores and their bias
1	6	Visual analysis and controversies	BG2 0.08	Visual analysis and controversies
1	7	YouTube as an archive for the end of life	BG2 0.12	YouTube as an archive for the end of life

Week number	Project number	Project title	Project room(s)	Project wiki page(s)
2	1	Containing homophily	BG1 0.16	Containing homophily
2	2	Mapping war atrocities across platforms	BG1 0.04	Mapping war atrocities across platforms
2	3	Gab's path to radicalization?	BG2 0.02	Gab's path to radicalization?
2	4	Technobiographies of an imagined past	BG1 0.04	Technobiographies of an imagined past
2	5	The internet of miracle cures	BG1 0.13	The internet of miracle cures
2	6	Mapping the multiverse of alternative facts in the age of post-truth	BG2 0.02	Mapping the multiverse of alternative facts in the age of post-truth
2	7	Pipeline politics in Canada’s colonial context	BG2 0.08	Pipeline politics in Canada’s colonial context
2	8	Hunting the bakers	BG2 0.02	Hunting the bakers

* Project rooms are announced on the first day of the week, Please find a list and map of venue locations in the welcome package (section IV).

Please click on the WikiWord to create a new wiki page for your project report. Further instructions may be found in the welcome package (section XI).

Digital Methods Summer School

This year’s Digital Methods Summer School is dedicated to approaches to studying so-called machine bias. Discussions have been focusing on how to hold algorithms accountable for discrimination in their outputting of results such as in the notorious cases of query results for ‘professional hair’ (white women’s hair-do’s) and ‘unprofessional hair’ (black women's hair-do’s). Recently, it was found that search engine image results for ‘pregnancy’ and ‘unwanted pregnancy’ are similarly divided, with the pregnancy queries returning white skinned women (mainly bellies, privileging the baby over the woman). ‘Unwanted pregnancy’ results in diverse ethnicities. These are new variations on classic, and still urgent, search engine critiques (once known as ‘googlearchies’) which questioned the hierarchies built into rankings, asking who is being authorised by the engine to provide the information. That work moves forward at the Summer School, building on examinations of the volatility of engine results, as in the Issue Dramaturg project, which put on display the drama of websites rising and falling in their rankings after algorithmic updates, meant to fight spam, but having unintended, epistemological consequences. More recently, Facebook newsfeeds have been the source of critique for their privileging and burying mechanisms, however much they — like the engine returns preceding them — are not easily captured and documented. Saving engine results has been against the terms of service; making derivative works out of engine results also breaks the user contract. Saving, or recording, social media (newsfeed) rolls seems even less practicable given how feeds are even more personalised, presumably resisting generalisable findings. User surveys pointing out unexpected newsfeed results have led to calls for ‘algorithmic auditing’, a precursor to machine bias critique. As reported in the technical press, querying social media ad interfaces shows highly segmented audiences (including racist ones such as publics to target for ‘jew haters’ among other available keyword audiences for sale).

These ad interface results could be repurposed to show which population segments (as defined by the platforms) are driving the content choices reflected in the results served. How large are these discriminatory segments? Capturing, auditing, or repurposing results are diagnostic practices, identifying under which circumstances machines could or ought to be retrained. The larger question, however, concerns how to retrain the machine. One approach lies in query design — fashioning queries so as to ‘re-bias’ the results. Others concern corpus development. For example in stock photography efforts have been made to reimagine (‘re-image’) women (in the well-known case of Getty Images' ‘Lean In Collection’), however much the images are often used out of context, as has been found. Yet another one concerns training and maturing research accounts to trigger controlled algorithmic responses.

The Digital Methods Summer School is interested in contributing not only to interpretations of celebrated cases of algorithmic or machine bias, but also providing diagnostic, query-related, research account and corpus-building research practices that seek to address the matter more conceptually.

Expanding the case study collection is also of interest; age discrimination in Facebook ad interfaces (an American theme) is a recent example of a telling case study of in-built rather than organic machine bias, but the international landscape may contribute more to bias detection, as is the aim of the Summer School. In Twitter there are feminist bots striving to keep the #metoo space serious, since the spam has arrived. Which other practices of remaining on topic may be found, and how may their success and complications be characterised? There is also the question of the ramifications of conceptual contributions to re-biasing for big data science. Which practical contributions could be made to big data critique?

Applications & Key Dates

To apply for the Digital Methods Summer School 2018, please use the University of Amsterdam Summer School application form (applications are now closed!). If that form is not working, please send (i) a one-page letter explaining how digital methods training would benefit your current work, (ii) enclose a CV (with full postal address), (iii) a copy of your passport (details page only), (iv) a headshot photo, and (v) a 100-word bio (to be included in the Summer School welcome package). Mark your application "DMI Training Certificate Program," and send to summerschool [at] digitalmethods.net.

• 4 May: Deadline for applications.
• 7 May: Notifications. Accepted participants will later receive a welcome package in mid June, which includes a reader, a day-to-day schedule, and a face book of all participants.
• 18 June: Deadline for summer school fee payments. Participants must send a proof of payment by this date.

The cost of the Summer School is EUR 895 and is open to PhD candidates and motivated scholars as well as to research master’s students and advanced master’s students. Data journalists, artists, and research professionals are also welcome to apply. Accepted applicants will be informed of the bank transfer details upon notice of acceptance to the Summer School on 7 May. Note: University of Amsterdam students are exempt from tuition and should state on the application form (under tuition fee remarks) that they wish to apply for a fee waiver. Please also provide your student number.

Any questions may be addressed to the Summer School coordinators, Esther Weltevrede and Fernando van der Vlist: summerschool [at] digitalmethods.net. Informal queries may be sent to this email address as well.

Scholarships

The Digital Methods Summer School is part of the University of Amsterdam Summer School programme, which has a video giving a flavor of the Summer School experience. Students from universities in the LERU and U21 networks are eligible for a scholarship to help cover the cost of tuition for the DMI Summer School. Please state LERU or U21 university affiliation under tuition remarks when applying to the Summer School. Dutch universities are not eligible.

Please find information about available scholarships at the UvA Summer School page.

Accommodations & Catering

The Summer School is self-catered, and there are abundant cafes and sandwich bars nearby. We previously made a map of nearby lunch and coffee places, see bit.ly/dmi17_ss_lunch.

We have organised studio apartments for a reasonable rate. These are 25 minutes away from the Summer School by public transport. We also have arranged a discount at Hotel Casa Amsterdam. For those who prefer other accommodations, we suggest Airbnb or similar. For shorter stay, there is Hotel Le Coin, where you may request a university discount.

• Studio apartments. Cornelis Lelylaan 3 ('Little Manhattan'). eur 450 total. 25 minutes away. Register with Summer School, and pay for accommodations with the tuition fee. First come, first served.
• Hotel Casa Amsterdam, Eerste Ringdijkstraat 4, 1097 BC Amsterdam, Netherlands. 25% discount off hotel rates. 15 minutes away. Register directly with hotel.
• Hotel Résidence Le Coin, Nieuwe Doelenstraat 5, 1012 CP Amsterdam, Netherlands. 'University discount.’ 2 minute walk. Register directly with hotel.

Successful Completion & Completion Certificates (incl. 6 ECTS when necessary)

To successfully complete the Summer School and receive a completion certificate (and 6 ECTS when necessary), you must complete a significant contribution to two Summer School projects (one in week one and the other in week two), evidenced by co-authorship of the project reports as well as final (joint) presentations. Templates for the project report as well as for the presentation slides are supplied.

Schedule

The Summer School meets every weekday. Please bring your laptop and your VGA adaptor for the projector. (An iPad is not enough.) We will provide connectivity. We start generally at 9:15 in the morning, and end around 17:15. There are morning talks as well as tutorials one to two days per week. All other time is devoted to project work with occasional collective and individual feedback sessions. Each Friday there are final presentations. On the second Friday we have a festive closing with a boat trip on the canals of Amsterdam.

Preparations: Online Tutorials

For your Summer School to be especially successful we would recommend highly that you watch (or listen to) the digital methods tutorials. The DMI YouTube channel has copious materials, and we would very much like for you to watch the social media tool tutorials.

Summer School Philosophy

The Digital Methods Summer School is exploratory and experimental. It is not a setting for ‘just’ tool training or for principally tool-driven research. Substantive research projects are conceived and carried out. Participants are encouraged to ‘span time with their issue’ and the materials. In other words, we heed Alexander Galloway’s admonition about data and tool-driven work: “Those who were formerly scholars or experts in a certain area are now recast as mere tool users beholden to the affordances of the tool — while students spend ever more time mastering menus and buttons, becoming literate in a digital device rather than a literary corpus” (Galloway 2014: 127). We encourage device and corpus literacy! The device training we ask you to do prior to the Summer School through online tutorials, and at the Summer School itself, in a kind of flipped learning environment (if you'll excuse the overused phrase), we would like to believe that you have familiarised yourself already with the tools and completed the tutorials available online. During the Summer School we will discuss and tinker with the nitty-gritty, aim to invent new methods, techniques and heuristics and create the first iterations of compelling work to be shared.

About the Digital Methods Summer School

The Digital Methods Summer School, founded ten years ago, in 2007, together with the Digital Methods Initiative, is directed by Prof. Richard Rogers, Chair in New Media & Digital Culture and Department Chair at Media Studies, University of Amsterdam. The Summer School is one training opportunity provided by the Digital Methods Initiative (DMI). DMI also has a Winter School, which includes a mini-conference, where papers are presented and responded to. Winter School papers are often the result of Summer School projects. The Summer School is coordinated by PhD candidates in New Media at the University of Amsterdam, or affiliates. The Summer School has a technical staff as well as a design staff, drawn from the ranks of Density Design in Milan. The Summer School also relies on a technical infrastructure of some nine servers hosting tools and storing data, which recently (and intrepidly) moved to the cloud. In a culture of experimentation and skill-sharing, participants bring their laptops, learn method, undertake research projects, make reports, tools and graphics and write them up on the Digital Methods wiki. The Summer School concludes with final presentations. Often there are subject matter experts from non-governmental or other organizations who present their analytical needs and issues at the outset and the projects seek to meet those needs, however indirectly. For instance, Women on Waves came along during the 2010, Fair Phone to the 2012 Summer School, and Greenpeace International and their Gezi Park project in 2013 as well as the COP21 Lima project in 2015. We have worked on the issue of rewilding eco-spaces with NGOs in the 2014 Summer School. More recently we have sought to repopulate city dashboards (Summer School 2015 and Winter School 2017).

• Previous Digital Methods Summer Schools, 2007–2017, https://wiki.digitalmethods.net/Dmi/DmiSummerSchool.
• Previous Digital Methods Winter Schools, 2009–2018, https://wiki.digitalmethods.net/Dmi/WinterSchool.
• What's it like? Digital Methods Summer School flickr stream 2012 and flickr stream 2013. There is also a 2015 flickr collection.
• More recently we have created Summer and Winter school shorts: Promo video about the Digital Methods Summer School (2014); Summer School in 2015 (tip!), and Winter School 2016 video.
• 2016 keynote lecture by Prof. Richard Rogers on Critical Analytics. 3-minute interview on the same subject at Johns Hopkins Bologna (2018)

About Digital Methods as a Concept

Digital methods is a term coined as a counterpoint to virtual methods, which typically digitize existing methods and port them onto the Web. Digital methods, contrariwise, seek to learn from the methods built into the dominant devices online, and repurpose them for social and cultural research. That is, the challenge is to study both the info-web as well as the social web with the tools that organize them. There is a general protocol to digital methods. At the outset stock is taken of the natively digital objects that are available (links, tags, threads, etc.) and how devices such as search engines make use of them. Can the device techniques be repurposed, for example by remixing the digital objects they take as inputs? Once findings are made with online data, where to ground them? Is the baseline still the offline, or are findings to be grounded in more online data? Taking up these questions more theoretically (but also practically) there is also a Digital Methods book (MIT Press, 2013) as well as a complementary Issue Mapping book (Amsterdam University Press, 2015), and other digital methods publications.

About the Digital Methods Initiative

The Digital Methods Summer School is part of the Digital Methods Initiative (DMI), Amsterdam, dedicated to developing methods for Internet-related research. DMI was founded a decade ago with a grant from the Mondriaan Foundation, and the Summer School has been supported by the Center for Creation, Content and Technology ( CCCT), University of Amsterdam, organized by the Faculty of Science with sponsorship from Platform Betatechniek. It also has received support from the Citizen Data Lab, Amsterdam University of Applied Sciences as well as "Media of Cooperation," University of Siegen.

Social

• We are using the #DMI18 hashtag as the backchannel for communication (to use a throwback term for Twitter usage).
• Here is the Facebook Group from one year, and from the last Summer School.
• Here are pictures from a variety of DMI Summer and Winter School flickr streams.
• We will have a list of Summer School participants and make an old-fashioned face book with the headshots and bios you send to us.

Suggestions for Evening Hangouts

• Amsterdam Evening Hangout Suggestions