A recent blog by Dariusz Galasiński about the poverty of some qualitative analysis has prompted me to dust down an old paper from back in 2003. Written mostly by Michael Billig, it was something of a success d’estime among those who could find it – but it got lost when the online journal it was in folded. Its worries about discourse analysis are easily generalised to qualitative research more broadly; and, as Galasiński’s blog shows, still all too topical.

This is an abridged version; if you’d like to read the properly referenced full thing, probably the easiest place to get it is as a .pdf from Loughborough University’s repository.

Discourse analysis means doing analysis: A critique of six analytic shortcomings

Citation: Antaki, C., Billig, M.G., Edwards, D. and Potter, J.A., (2003) ”Discourse Analysis Means Doing Analysis: A Critique of Six Analytic Shortcomings”, Discourse Analysis Online, 1

Abstract

A number of ways of treating talk and textual data are identified which fall short of discourse analysis. They are: (1) under-analysis through summary; (2) under-analysis through taking sides; (3) under-analysis through over-quotation or through isolated quotation; (4) the circular identification of discourses and mental constructs; (5) false survey; and (6) analysis that consists in simply spotting features. […]

Introduction In the past fifteen years, discourse analysis has had a major impact on social psychology, especially in Britain. It has introduced new methods of research, new ways of conceptualising research questions and new ways understanding the nature of psychology itself. In this time it has gone from a marginal perspective developed by a handful of scholars to an approach that is represented in wide range of different empirical and theoretical journals, seen in different conference presentations, and developed in a growing body of PhDs. For an increasing number of academics discourse analysis is the prime way of doing social psychological research. At the same time, there has been a proliferation of forms of discourse analysis. […] Our aim here is not further to rehearse these debates and issues, but to highlight some methodological troubles that are visible from whatever discourse perspective one adopts. […].  All of these have positive things to say about doing analysis. But they leave implicit what is not analysis. That is what we want to make explicit in this paper.

[…]

(1) Under-Analysis Through Summary

Qualitative analyses share something important with quantitative analyses in that they both want to do something with the data. Neither is content merely to lay the data out flat. A quantitative researcher who merely presents the raw data from subjects in an experiment without putting it to some sort of statistical testing would hardly be said to have analysed it. So it is with qualitative data.

An interview, doctor’s consultation or television talk show might be transcribed […] Transcription prepares the data for analysis. However, it is not analysis in itself. Analysis must mean doing something with the data, but not just anything. A quantitative analyst who presents a selection of their raw data in some graphical form, hoping that the reader might see a trend or a pattern, would not have done anything statistical on their data. A qualitative analyst will be doing the equivalent if they present their data as a prose summary. However, summarising the themes of what participants might say in an interaction typically does not involve any analysis of the discourse that they are using. A summary is likely to lose the detail and discursive subtlety of the original. The summary will be shorter and tidier. It will be phrased in the analyst’s words, not those of the original speakers (or writers). It will lose information and add none.

[…]

(2) Under-Analysis Through Taking Sides

 If data analysis requires that the analyst offers something additional beyond presenting or summarising the data, then this does not mean that every additional offering is analysis. It certainly does not mean that every added element of analysis is discourse analysis. In some writing one sees the additional offering of the analyst’s own moral, political or personal stance towards what the quoted speaker or text is saying. This on its own is not discourse analysis.

There is a debate amongst discourse analysts whether analysts should take positions with respect to the material that they study. It is not our intention to enter into that debate. Nor, indeed, do we agree amongst ourselves on this issue. What we do insist upon, however, is that position-taking – whether analysts align themselves with, or critically distance themselves from, the speakers whom they are studying – is not analysis in itself. Sympathy and scolding (either explicit or implicit) are not a substitute for analysis.

[…]

[O]ne can say that under-analysis can occur when the analyst substitutes sympathy or scolding for detailed examination of what the speakers are saying. A particular danger is that the desire to sympathise or censure, when not allied to careful analysis, can lead to the sort of simplification that is the antithesis of analysis. […]  [It can] produce a flattening of the discursive complexity, as the analyst selects quotations for the rhetorical effect of appealing to the readers as co-sympathisers or co-scolders. The result is enlistment, not analysis.

(3) Under-Analysis Through Over-Quotation or Isolated Quotation

[…] Under-Analysis through Over-Quotation is often revealed by a low ratio of analyst’s comments to data extracts. If extract after extract is quoted with only the occasional sentence or graph of analyst’s comment, then one might suspect this type of underanalysis is happening. In the example of [an] interview about marriage, an analyst might think of chopping up the whole extract into quotable extracts, omitting the interviewer’s questions. After presenting the quotations, the analyst might summarise the collection of quotes with a comment such as ‘so we can see that the respondent had strong views about the importance of marriage and commitment’. This would not be analysis. The list of quotes divorces the utterances from their discursive context, with the result that it would not be possible to analyze them as responses to questions.

More typically, Under-Analysis through Over-Quotation is liable to occur when the analyst is piecing together responses from different speakers. For instance, the analyst might wish to show that a number of interviewees had responses rather like the one in our extract. Selective quotation from such respondents might be given. There can be analytic and theoretical reasons for presenting profiles based on piecing together such quotations. However, this profiling is not normally of itself discourse analysis, for again it does not of itself get down to the business of actually analysing in detail the discourse that is used. Indeed, as has been mentioned, the over-quotation may impede certain forms of discourse analysis by removing utterances from their discursive context. Two tell-tale signs of Under-Analysis through Over-Quotation would be the small amount of analyst’s writing in proportion to the large amount of quotation, and the tendency of the writing to refer to the quotations rather than analyse them.

[…]

(4) The Circular Discovery of (a) Discourses and (b) Mental Constructs

 Compiling quotations into a profile can be part of a discourse analysis. For instance, an analyst might be seeking to investigate whether speakers, in framing their individual utterances, are using commonly shared discursive resources. Some analysts examine how particular rhetorical and conversational devices are used in specific contexts. Some researchers examine how speakers may be using shared patterns of understanding or interpretation. There are a variety of terms to describe the sort of discursive resources that speakers may share […] [S]ome discourse analysts will consider it a matter of theoretical and methodological importance to show how particular utterances are themselves formed out of wider, socially shared ‘repertoires’, ‘ideologies’, ‘discourses’ etc. The analyst might present a profile of quotes in order to show how different speakers might be drawing upon common repertoires etc.

In theory, such profiling would seem to fit the requirement of discourse analysis. An analytic extra is being added. The reader is not merely being informed that the speakers made these utterances, but the additional claim is made that all these utterances have something in common, being manifestations of a shared pattern of talking. The problem comes when care is not taken to substantiate the claim. Again, the data cannot be left to ‘speak for itself’, as if a series of quotes is sufficient in itself to show the existence of the repertoire, ideology or discourse. Moreover, the analyst runs the risk of circularity if the socially shared entities are cited in explanation for the utterances.

[…]  If that is all the analyst is doing, then these terms function merely as summaries. They add little if anything to the analysis of the utterances, for they are only handy ways of describing the common features that the analyst is claiming to summarise. However, if the analyst then moves towards an explanation of the quoted discourse in terms of these entities, then a step towards circularity is taken, and we have Under-analysis through Circular Discovery. The quotes, which provide the justification for claiming the existence of a ‘marital commitment discourse’ (or repertoire, or ideology) are then explained in terms of this entity. Such circularity would occur if the analyst, having quoted extracts to claim the existence of a ‘marital commitment repertoire / ideology / discourse’, then goes on to imply that the speakers made those particular utterances because they shared this discourse, repertoire or ideology.

[…]

(5) False Survey There is a danger of extrapolating from one’s data to the world at large. This error is not unknown in quantitative research, of course. It may be avoided by explicitly survey-oriented studies, but is not uncommon in experimental social psychology when findings are subtly generalised from the sample of the experiment (say, a set of North American undergraduates) to the universal categories they are supposed to represent (women, high achievers, people with a certain attributional style). Discussion sections of experimental papers sometimes use such unqualified terms, with the logical implication that they encompass all members of that category.

The same danger of False Survey lurks for qualitative work that discovers that certain respondents use certain discourses or ways of speaking. It is fatally easy to slip into treating one’s findings as if they were true of all members of the category in which one has cast one’s respondents. For example, an analyst reading our interview extract might see, in the respondent’s way of talking, a ‘traditionalist discourse of marriage’. They might then be tempted to attribute that discourse to all people of their alleged kind  (‘non- University-educated young women’, if that was the demographic information supplied along with the extract). This attribution might be done explicitly, but is still more likely to happen unconsciously, in the way the writer uses demographic categories to refer to the people in their data.

Probably few discourse analysts want or intend explicitly to be reporting surveys; but without care, their reports may give that impression. Such a fault makes the work an easy target for the quantitatively-minded, who will properly see it as failing to supply appropriate evidence for its claims. If a survey is wanted, survey tools must be used.

(6) Under-Analysis through Spotting If discourse analysis demands an attention to the details of utterances, this does not mean that all such attention qualifies as satisfactory discourse analysis. Analyses provided by discursive, conversation and critical discourse analysts have, over the past twenty-five years, noticed and labelled a wide variety of conversational and rhetorical procedures. Anyone engaging in these sorts of analyses should properly acquaint himself or herself with such work. They should be able to recognize these conversational features in data extracts. The same is true of rhetorical tropes in printed persuasive materials and so on. However, the recognition of features does not constitute analysis, at least at a research level. It may be appropriate in training exercises as one seeks to acquire the skills of analysis. But research does not, and should not, consist principally of feature-spotting, just as analysing the history and functions of the railway system cannot be accomplished by train-spotting.

[…].

An analysis that consisted primarily of such spotting would not count as original research. It would be like a training exercise in running a well-known illusion such as the Müller-Lyer or administering a well-established personality test. Original analysis should seek to show how established discursive devices are used, in new sets of material, to manage the speakers’ interactional business. What is required is to show what the feature does, how it is used, what it is used to do, how it is handled sequentially and rhetorically, and so on. To remark: ‘that’s a 3-part list’ for example, is to identify a wellknow discursive feature of talk and text; but the interest is in unpacking it and show what it’s doing in this particular set of materials. Good analysis always moves convincingly back and forth between the general and the specific.

Concluding Comments

[…] We should be at pains to say that we do not think that identifying these inadequacies tends positively toward any one particular level or style of discourse analysis. What it does is show up how some ways of writing have the sheen of analysis without its substance. We have deliberately stopped short of saying what does count as analysis, because of the variety of directions in which analysis can go, and because much more has been written on this elsewhere. Perhaps it is safe to say that analysis means a close engagement with one’s text or transcripts, and the illumination of their meaning and significance through insightful and technically sophisticated work. In a word, Discourse Analysis means Doing Analysis.

[References have been stripped from this abridged version]