Two Glasgow Researchers on Putting Non-Experts in Charge of AI Audits

Why participatory auditing

NervNowParticipatory AI auditing puts non-experts at the center of evaluation. What gap in how AI was being audited convinced you this was needed, and how did the framework first take shape?

Participatory AI auditing emerged in response to a gap in traditional AI auditing approaches, which relied exclusively on technical experts, overlooking the perspectives and lived experiences of the people who actually use AI systems. There have been numerous examples of spontaneous, crowd-sourced audits where everyday users, without formal technical expertise, have successfully identified important issues in deployed AI applications that expert-led evaluations had missed. Therefore, people without AI expertise have been successful at effectively uncovering blindspots overlooked by AI experts. Research on sociotechnical auditing has shown why this happens. People embedded in the context where AI is used bring valuable knowledge about social dynamics and real-world impacts that experts may not have access to. Experts can be limited by their own cultural assumptions or by evaluating systems outside of their everyday context of use. Participatory AI auditing complements traditional approaches by involving end users and decision subjects in the evaluation of AI applications as a whole rather than being limited to the assessment of AI models. Our project takes this idea one step further by developing tools and methodologies that make participatory AI auditing more systematic, enabling communities to contribute their contextual expertise in a structured way.

NervNowYou worked with 17 people who had no AI background, including patient representatives, teachers and parents. How did you decide who to involve, and why those groups specifically?

Our research focuses on specific AI applications as potential audit subjects, so we recruited participants based on their relationship to those applications. We were interested in involving either end users of an AI system or decision subjects, that is people whose lives could be affected by an AI system even if they never interact with it directly. For example, one of our case studies involved an AI application designed to predict the likelihood that a Scottish citizen would be admitted to hospital within the following year. Since the outputs of this system could influence healthcare decisions affecting patients, we recruited Scottish citizens, as potential patient representatives, whose perspectives could help evaluate its real-world implications. Another case study focused on an AI application that predicts whether a child has a secure or insecure attachment style. We recruited teachers because such a tool could be used in educational settings and parents of young children, as they are among the people most likely to use, be affected by, or make decisions based on this type of application. By selecting participants with direct experience of the contexts in which these AI applications are deployed, we aimed to capture contextual knowledge that is often unavailable through technical audits alone.

NervNowThe work brought together researchers from Glasgow, Sheffield, Stirling, Strathclyde and York, with funding from EPSRC through Responsible AI UK. How did a project this size come together, and what does that backing signal about where responsible AI research is heading?

This project brought together a group of researchers with diverse expertise spanning AI, human-computer interaction, social sciences, and responsible AI. Participatory AI auditing is inherently interdisciplinary, therefore it requires not only technical knowledge of AI systems but also expertise in user-centred design, governance, and the social contexts in which these technologies are deployed. Bringing together researchers from multiple institutions allowed us to draw on these different perspectives to develop a robust participatory auditing framework. The support from EPSRC through Responsible AI UK reflects a growing recognition that building trustworthy AI requires looking beyond model performance to understand how AI systems affect people in practice. This means developing approaches that combine technical evaluation with meaningful public participation and stakeholder engagement. Our project contributes to this direction by exploring how the knowledge and experiences of people affected by AI can become a systematic part of AI auditing.

What the audits surfaced

NervNowEva, you noted that participants easily identified potential problems, sometimes flagging issues developers did not appear to have considered. Can you share a concrete example that surprised the team?

One example that stood out came from our audit of an AI application designed to predict children’s attachment style from speech. The group of parents participating in the audit raised several questions that were surprisingly pertinent and wouldn’t be typically considered during a technical audit. They wanted to know how the system’s ground truth had been established. They asked who had determined whether a child’s attachment style was “secure” or “insecure,” what their qualifications were, and how they had reached agreement on those labels. These are fundamental questions about the quality and validity of the training data, that do not often arise in traditional audits, which are mainly focused on model performance metrics. They also highlighted potential unintended consequences of the application itself. They questioned whether children could feel disrupted by and react differently to the testing procedure, which could create anxiety or alter their behaviour, potentially affecting the speech samples and leading to inaccurate predictions or false positives. Their observations demonstrated how people with lived experience can identify practical and contextual issues that may not be apparent from a purely technical perspective.

NervNowParticipants found it harder to work out how to measure an impact than to spot it in the first place. What does that tell you about where tooling and structured guidance matter most?

One clear finding from our study was that participants were generally very capable of identifying potential impacts based on their own experiences and knowledge of the context in which an AI system would be used, but they needed more support in translating those concerns into concrete metrics that could be used in an audit. Defining meaningful measures of concepts such as fairness is difficult even for experts, and participants often found it challenging to determine what evidence would be needed or how evidence could be used to formulate a measure in practice. In some cases, they proposed metrics that relied on data that was not readily available, while in others they suggested qualitative forms of evaluation that existing auditing tools are not well equipped to support. This highlights where tooling and structured guidance can have the greatest impact. Rather than helping people identify issues, which they were already very good at, tools should provide step-by-step support for turning those observations into robust audit criteria. That includes helping participatory auditors formulate measurable questions, identify appropriate evidence, understand what data is needed, and accommodate both quantitative and qualitative forms of assessment. In other words, the key contribution of future participatory auditing tools is to scaffold the process of transforming auditors’ expertise into a rigorous and actionable audit.

NervNowParticipants pushed for transparency about an application’s purpose and who is conducting the audit, and for outcomes that capture both benefits and harms rather than a simple pass or fail. How should those principles shape the way audit tools are built?

Our participants consistently wanted greater transparency about the AI application itself, particularly its intended purpose, who had developed it, and under what conditions. Understanding why an application exists and what problem it is intended to solve helped them assess not only whether it was functioning as intended, but also whether they believed it provided sufficient value to justify its use. Participants also wanted audits to document potential benefits alongside potential harms, rather than producing a simple pass-or-fail judgement, reflecting a broader view of AI auditing as an exercise in weighing trade-offs. An AI application may introduce certain risks while also delivering important benefits, and participants wanted audit outcomes to make those trade-offs explicit rather than reducing them to a binary assessment. These findings suggest that audit tools should be designed to support this broader form of evaluation. They should provide clear contextual information about the application, prompt participants to consider both positive and negative impacts, and support balanced judgements about whether an AI system is appropriate, necessary, and valuable, not just whether it is technically accurate. In other words, participatory auditing encourages us to ask not only “Does this AI system work?” but also “Should this AI system be used in the first place?”

NervNowYou tested the approach on SPARRA, the NHS Scotland readmission tool, and SAM, the school attachment prototype. Did the two very different systems surface different kinds of concerns from participants?

Although the two AI applications are meant to operate in very different domains, participants raised many of the same overarching concerns, including privacy, security, bias, and social exclusion. What differed was how those concerns were expressed in relation to the specific context in which each application would be used. For example, in the SPARRA case study, participants were concerned that information about a person’s predicted risk of hospital admission could be leaked or misused, for instance by insurance companies to increase premiums or discriminate against individuals. In contrast, participants auditing SAM worried about the consequences of attachment predictions being disclosed beyond their intended purpose, such as influencing future educational opportunities, employment, or even government decision-making. Concerns about fairness and inclusion also reflected the different application domains. In the SPARRA audit, participants highlighted the risk that marginalised groups with poorer access to healthcare might be underrepresented in the data, leading to less accurate predictions and potentially delayed or inadequate care. In the SAM audit, teachers and parents focused on whether the training data adequately represented different children and whether biases in the data could lead to misclassification of attachment styles. Overall, one of the key findings was that while the broad categories of concern were remarkably consistent across applications, participatory auditors drew on their contextual knowledge to identify how those risks would manifest in practice. This illustrates why involving people with lived experience is so valuable, it grounds AI audits in the realities of the domain in which the technology is deployed.

Taking it inside a company

NervNowPatrizia, you have described participatory auditing as a potential selling point, an application vetted by a diverse range of people from the start. How would you make that case to a product or engineering leader weighing it against speed to market?

I would argue that a slightly delayed introduction of an AI application to the market after a rigorous testing process is easier to recover from than reputational loss from issues of a product arising because important stakeholder views have been forgotten at the development stage. Participatory auditing can support product and engineering leaders in identifying which aspects of the AI application are important to the users and if these fulfil expectations, uncovering potential blind spots of the developers, before they can become a potential reason for a whole user group to avoid switching to other products.

NervNowWhat would it realistically take for a company to fold participatory auditing into an existing development cycle, in terms of time, cost and where in the process it sits?

Participatory auditing can be done at any time in the development cycle of an AI application, starting from business case discussions, where the focus would be on potential societal impacts and data needs to ensure diversity, all the way to post-deployment, where active impact of the AI application can be assessed by stakeholders with experience in using the AI application. The time and cost are very dependent on the scope the audit would take and at which point in the cycle the audit would be conducted. Auditors should receive some training to set expectations, and audit instigators would need some resources to prepare the information for auditors in an easily understandable way. Companies will get the best use of these preparations if they do iterative participatory audits, at different stages of the development cycle and after each major update, as materials and processes can be re-used and adjusted.

NervNowInternal AI ethics teams and red-teaming already exist inside larger firms. Where does participatory auditing fit alongside those, and what does it do that they cannot?

Participatory auditing involves people without auditing and technical expertise, but with domain knowledge and lived experience in the area in which the AI application will be deployed. Internal AI ethics and red teams typically do not involve these user groups. As such, they might not be able to investigate as rigorously if the AI application considers edge cases and biases as the participatory auditors can. The AI application acts in the world of the participatory auditors and they know the ins and outs, which makes them the best-placed people to understand the impact that the AI application can have on society and communities.

NervNowIs there a risk this becomes a compliance checkbox rather than a genuine input into design? What separates meaningful participation from box-ticking?

In our vision of participatory auditing, audit criteria would predominantly be decided on and created by the participatory auditors. While audit instigators, the ones setting up the audit, might suggest audit criteria, the participatory auditors are ultimately in charge of what they think the AI application should be audited on. As such, for auditors to properly engage with the audit, they will have to critically think about the AI application and define these audit criteria. This aspect works fundamentally against the audit becoming a checkboxing exercise, unless auditors only purely assess the AI application on criteria provided solely by the audit instigators or the auditor group is uniform and similarly biased one way or another. The audit instigator is in charge here, making sure that the auditors they invite are diverse and engaged and that they encourage them going beyond criteria they provide (or even refrain from doing so).

Regulation and what comes next

NervNowProfessor Stumpf pointed to the EU AI Act as a driver for limiting AI harms. Do you see participatory auditing as something regulators could eventually expect, or does it stay voluntary?

We hope that participatory auditing will become a standard practice, to feed into compliance and assurance. With the almost uncontrolled rise of AI applications and their use, participatory auditing can be a powerful tool for AI providers to differentiate themselves in the eyes of the users and nurture more trust in their applications in a time in which bad practice and misuse lead to almost daily reports of harm. It is hard to say in which direction regulators might go, as we have seen that regulations around AI have come under considerable pressure from powerful tech companies and countries supporting deregulation in the space. Yet on the other hand there have been groundbreaking lawsuit decisions lately that could lead to a more cautious approach, certainly on the side of the users. I think that users could really make the difference here: if they start to expect proper scrutiny of AI applications, of which participatory auditing could be an influential bit of evidence, then tech companies will eventually start to catch on. I believe that early adoption of this good practice will provide an advantage for companies.

NervNowWhat stage is the work at now, and what comes next: pilots with industry, open tooling, policy engagement?

Since publication of the paper, we have now implemented the first version of an auditing tool, the PHAWM Workbench, currently focused on the auditing experience based on our work discussed in the paper and work done by our colleagues in parallel. We are conducting research on the audit instigator side at the moment and will update the workbench once we have analysed those findings. In addition, we have created a guiding framework, the PHAWM Methodology, which discusses guiding principles around participatory auditing and supports both auditors and audit instigators through the process. The methodology document is not meant to be read front to back, but as a look-up document to dip in and out of at different points within the auditing process. We are currently testing both in a field trial with three public service partners and will improve both with the feedback we are collecting. The workbench and methodology are open source and can be accessed via the PHAWM website (https://phawm.org/)

NervNowIf a CXO reading this wanted to experiment with participatory auditing on a single product in the next year, what is the smallest sensible first step you would point them to?

I would probably point them towards the methodology on our website, as it provides support for thinking about and setting up a participatory audit.