There’s a lot going on at the interface of AI and speech – both recognition and production – and some of it draws on ideas from ethnomethodology and conversation analysis. But is it any good? Stuart Reeves runs the rule over some of the issues.

Artificial Intelligence is a big deal now. We’re told that AI systems are reaching and even exceeding human performance at things like playing games, hearing and transcribing words (Xiong et al., 2017), translating between languages, or recognising faces and emotions (Lu and Tang, 2014). This means we are entering a world where it’s possible for people to have conversations with AI agents (a Google researcher recently claimed a chatbot as “sentient”), or get computers to understand what’s in a picture and even generate their own art when prompted.

The problem with this picture is that while it is technically accurate, it is also conceptually wrong.

Why?

It’s not that the AI systems built aren’t impressive. Voice transcription or language translation services—like many AI systems—are vastly more powerful than they were 10 years ago. This is because of significant hardware improvements around massively parallel processing (specifically, the use of Graphics Processing Units, or GPUs). This has made neural network techniques developed long ago now very tractable, and, coupled with huge data sources made available by the internet to ‘train’ these systems, it’s claimed AI can now readily ‘learn’ how to ‘recognise’ things. 

But there are two major self-deceptions going on here (hence the quotation marks in last paragraph). Firstly, it’s that, as Emily Bender says, people are confusing “the form of an artifact with its meaning”. Sure, an AI system can ingest text and produce news articles ‘automatically’, but that doesn’t make it a journalist or even what we’d think of as a ‘reader’. Secondly, when people talk about AI they tend to exclude all the human effort that is involved in designing, building and even performing its execution; and I say ‘performing’ because the running of AI is often very much the spectacle. (Nothing new here, just look at the hubbub around IBM’s Deep Blue chess playing AI from the 90s.)

This erasure of people subordinates the human to the machine. It offers a neat sidestep away from responsibility for all the potentially prejudicial and biased outputs that the huge data sets ingested by AI systems train them to generate, whether it’s reinforcing stereotypes or systematically excluding people from jobs. These problems are a feature of data sets because they are a feature of society—“stochastic parrots” as Bender et al. (2021) put it.

Ethnomethodology and CA’s long association with AI research

For ethnomethodologists and conversation analysts, however, the last decade’s resurgence of AI is increasingly demanding our attention because of AI’s spread into ever more mundane everyday circumstances, whether that’s voice-operated assistants like Apple’s Siri or self-driving cars. But AI is not new for EMCA.

It’s humbling to think that back in the 80s and 90s, when I was but a child, EM research from Lucy Suchman (Plans and Situated Actions, 1987/2006), and Graham Button and colleagues (Computers, Minds and Conduct, 1995), had already effectively pulled apart AI of the time. Suchman built a critique of the view that humans, as modelled by AI systems, mistook formal ‘plans’ as adequate descriptors of mundane interactions. Button et al. in turn deconstructed the notion that while AI systems might do things which appear human-like, it is a conceptual mistake to confuse them with human practical action. 

The problems they found have not gone away, no matter what anyone says about the ‘new AI’ which is sweeping the world. In fact the problems are bigger. It takes an army of people to train AI systems to ‘recognise’ anything, whether that thing is a sheep or that I’ve said the word ‘hello’. If EMCA researchers are going to pick apart human-AI ‘interactions’ (if we may even call them that), we need to look at the rest of this iceberg. Following Suchman, Mair et al. (2020) argue that EMCA should be “treating all the work that goes into and is done with AI, including descriptions of what a given system might be said to be or be doing, as being as much part of the ‘assemblage’ as the hardware and software”. 

Tread with care

Secondly, if EMCA wants to get stuck in with AI research(ers) en masse, we must also learn the hard lessons from EMCA’s prior engagements with technology-driven research, specifically in the field of human-computer interaction (HCI).

HCI’s ultimate interests are driven towards the (re)design of technologies, just like much of AI’s is. For HCI, this design orientation quickly cast EMCA into a ‘service’ capacity, which it will have for AI research if it is only concerned with delivering “implications for design” (Dourish, 2006). Alternately, EM and CA work could be rethinking its hybridity with design, respecifying, or deflating conceptual confusions in / of AI. 

HCI research also tried to domesticate EMCA, to bring its concerns for the social organisation of technology use into line with conceptual orderings of the field. Suchman’s EM-informed critiques of AI became “situated action theory” (Vera and Simon, 1993). You may also find a catalogue of pretty odd descriptions of EM and CA in various HCI textbooks (naming no names!). If we are researching and publishing on AI I can see EMCA being domesticated again.

One last thing….

The last reminder is about EM in particular. EM’s work in HCI has sometimes relitigated its many arguments with sociology (and other disciplines); replaying these in HCI has derailed the ability of EM (less for CA) to explore what hybridity, conceptual respecification, etc. could mean to its fullest. Critique is important, but it’s also useful to remember whether your audience is as invested in an argument as you are. Baggage should probably be avoided in engagements with AI research.

Stuart has been practicing ethnomethodology and (to a lesser extent) conversation analysis within human-computer interaction (HCI) and collaborative computing (CSCW) research for most of his academic career. Read more on his thoughts about EM/CA and AI in his piece in Medium.