Auditing AI and Sociotechnical Systems
The past, present, and futures of impactful audits.
Workshop at CHI 2027 Pittsburgh, PA
Conference: May 10–14, 2027 Workshop date: TBA
Algorithm audits have surfaced harms in sociotechnical systems and driven real-world change, from product updates and removals to lawsuits and legal reforms. Auditing is now at a critical moment: audit requirements are being written into law, yet access for independent auditors is narrowing, systems span complex supply chains, and the resources needed to audit are concentrated in a few hands.
While past AI audits often focused on relatively bounded systems with identifiable inputs, outputs, and vendors, today’s systems are increasingly complex, distributed, opaque, and continuously changing. These changes have created new challenges for conducting AI audits, and require methods that look beyond an analysis of the technical properties of the AI artifact to incorporate additional sociotechnical assessments of the individual, organizational and regulatory contexts that define downstream outcomes.
We see auditing as an ecosystem involving many actors and stakeholders: those who notice harms, those who verify them, those who can respond, those who are affected, and more. This approach also means looking beyond Large Language Models (LLMs): an LLM is just one tool which a 21st-century citizen might interact with on a given day. In addition to LLMs, people interact with recommender systems, risk scores, ad delivery, as well as more complex pipelines that combine several systems.
This in-person workshop will gather researchers, journalists, community auditors, legal experts, and practitioners to create shared strategies, tools, and methods for impactful audits. Through short talks and interactive activities, we aim to connect those who identify harms with those who can address them. We also want to reach people who may not realize they are already auditing: many civil society and community-based auditors may not call their work “auditing.” This workshop welcomes everyone who supports testing, assessing, inspecting, or auditing sociotechnical systems (e.g. red-teaming, algorithmic impact assessments, investigative journalism, and everyday informal reporting), no matter what they call their work.
Topics include:
We invite papers of 2 to 4 pages (excluding references) in the ACM template, such as field notes, reflections on past audits, tool or dataset demonstrations, method proposals, or provocations. Please submit via the A2S2 website by the posted due date. Organizers will review submissions for relevance and potential to spark discussion; accepted papers will be published on the workshop website. Authors are free to extend and publish their work at other venues.
At least one author of each accepted paper must attend in person. Since CHI workshops are in-person only, we will explore asynchronous ways to contribute for those unable to travel. This may include submitting field notes or short provocations that organizers present on their behalf, joining an online pre-workshop meeting, and/or co-authoring the post-workshop report via the Algo Audit Network Slack and mailing list.
As part of the registration process, the workshop will survey participants to collect information about accessibility needs. Submissions must comply with SIGCHI’s guide to accessible submissions.
Submission instructions and the workshop date will be announced here.