Algorithm audits are powerful tools for studying black-box systems without direct knowledge of those systems’ inner workings. While they have been effectively deployed to identify harms and biases in algorithmic content, algorithm audits’ narrow focus on technical components stop short of considering users themselves as integral and dynamic parts of the system, to be audited alongside its algorithmic components.
After an overview of the state of the art in algorithm auditing, this talk will introduce sociotechnical auditing: evaluating algorithmic systems at the sociotechnical level, focusing on the interplay between algorithms and users as each impacts the other over time. Metaxa will also present Intervenr, a platform developed to conduct browser-based, longitudinal sociotechnical audits, and a case study in which we deployed Intervenr to investigate the central claim of online targeted advertising systems: that targeted ads perform better with users. Finally, the talk will conclude by discussing some of the group’s current work, expanding the auditing method to novices and youth, and developing post-auditing tools for collective action.
Bio:
Danaë Metaxa is the Raj and Neera Singh Term Assistant Professor of Computer and Information Science at the University of Pennsylvania, with a secondary appointment in the Annenberg School for Communication. Metaxa specializes in issues of bias and representation in algorithmic systems and content, across high-stakes social settings including politics, employment, and education. Most recently, their work has focused on online ad targeting, including its effectiveness and its impact on marginalized populations. Their work has been published and awarded in top-tier publication venues including the Association for Computing Machinery’s (ACM) Conference on Human Factors in Computing Systems (CHI), and the ACM Conference on Computer-Supported Cooperative Work (CSCW). They have also worked in an advisory role with legal, industry, and policy groups including the NYC Commission on Human Rights, Halt AI, and PolicyLink. Prior to Penn, Metaxa received their Ph.D. from Stanford University in computer science.