* Paper: Algorithmic Accountability: Journalistic Investigation of Computational Power Structures. By Nicholas Diakopoulos. Digital Journalism, 2015
Excerpted from a discussion by Stefanie Knoll:
“How can the power of algorithms be understood and, when called for, controlled? We are only starting to understand how these strings of computer code are shaping our view of the world. As researchers point out, inherent biases in algorithms can lead to startling discriminatory possibilities, with important consequences.
Exposing the workings of algorithms to understand their deeper impact may yet become an important part of investigative journalism. In a 2015 paper published in Digital Journalism, “Algorithmic Accountability: Journalistic Investigation of Computational Power Structures,” Nicholas Diakopoulos of the University of Maryland examines journalistic strategies to gain insight about the inner workings of algorithms. While transparency of algorithms might be a first step to solve the problem, Diakopoulos is especially interested in a strategy called reverse engineering — “the process of extracting knowledge or design blueprints” by studying and then emulating the behavior of an algorithm.
The author discusses five case studies in which journalists used reverse engineering to examine algorithms, including a story in the Daily Beast, on the iPhone’s language-related algorithms; ProPublica on the 2012 U.S. election campaign and targeted email strategies; the Wall Street Journal on website pricing differentiation and on stock trading by executives; and one story by Diakopoulos himself. Based on these stories, Diakopoulos identifies the scenarios journalists typically encounter in their reporting on algorithms as well as the challenges emerging from these investigations in terms of human resources, legality and ethics.
The paper’s key points include:
* When using reverse engineering, journalists are interested in three aspects of an algorithm: the input, the output and the transformation from one to the other. There are often cases in which some elements in this relationship can or cannot be observed, and different strategies of reverse engineering may be necessary.
* When inputs are not available, “figuring out how to observe or simulate those inputs is a key part of a practical investigation…. Figuring out what the algorithm pays attention to as input becomes as intriguing a question as how the algorithm transforms input into output.”
* In this process, journalists need to keep in mind that external evidence of algorithms’ behavior might be disturbed by A/B testing — the practice of randomly assigning different treatments or content to various groups to optimize for the best response rate or return. The entities that use the algorithm are “already running experiments on their sites, and to a reverse engineer it might look like noise, or just confusing irregularities.”
Furthermore, algorithms “may be unstable and change over time, or have randomness built in to them, which makes understanding patterns in their input-output relationship much more challenging. Other tactics such as parallelization or analysis of temporal drift may be necessary in order to control for a highly dynamic algorithm.”
To successfully achieve algorithmic accountability reporting, media will have to “take dedicated efforts to teach the computational thinking, programming and technical skills needed to make sense of algorithmic decisions.” Given a legal framework that is growing more complex, “more work is also needed to explore the legal ramifications of algorithmic accountability through reverse engineering by journalists.”
New ethical questions may arise in the context of studying algorithms. The author suggests a focus on questions such as, “How might the investigation allow the algorithm to be manipulated or circumvented?” or “Who stands to benefit or suffer disadvantage from that manipulation?”
Diakopoulos underscores the computational skills needed for achieving algorithmic accountability. However, “reporting is still a key part of finding a story in a reverse-engineering analysis.” Even in an environment as technical as this, “knowing what makes something a story is perhaps less about a filter for statistical, social or legal deviance than it is about understanding the context of the phenomenon, including historical, cultural and social expectations related to the issue — all things with which traditional reporting and investigation can help.”
For more information, see the report
* Nicholas Diakopoulos. Algorithmic Accountability: On the Investigation of Black Boxes. Knight Foundation and the Tow Center on Digital Journalism at Columbia Journalism School.