Frame Analysis
theory
measurement
concepts
software
bibliography

Frame Measurement

qualitative detection

quantitative detection

validation

Identification and Measurement of Frames

Thomas König

Since framing is a theoretically demanding concept, it has proven elusive to measure (Maher, 2001: 84). The bulk of framing studies, starting from Goffman's initial thesis (Gamson, 1975: 605), offer no explicit measurement model.

Qualitative Identification of frames

Qualitative framing studies hardly ever reveal their measurement models. Even in otherwise well documented studies, it is often difficult to tell, which mechanisms were used to arrive at particular frames and, how they have been measured empirically. To give you a few examples: In one otherwise exceptionally well argued study, the frame identification process is laconically described in a footnote with "[f]rames were analyzed from the actual language of the reported claim (direct and reported speech)" (Statham & Mynott, 2002: 10, Fn. 6). Another article describes the frame detection process as follows: "After a first reading of the material, five thematic dimensions have been identified, [… this process was than] complemented by an in-depth qualitative analysis of the data " (Triandafyllidou, 2002: 3.7 & Fn. 10). A third paper flatly proclaims "during the data collection process, we coded all evidence of acknowledgment of or reactions to the ideas" (McCaffrey & Kayes, 2000: 49) without mentioning, how the coding was done.

In praxis, most studies use techniques borrowed from discourse analyses and sociolinguistics to identify frames. The repertoires of this methodologies are too proliferate to survey here. There are, however, a few heuristic devices that are worth mentioning.

One of the most revealing ways, in which identity frames are implicitly constructed, is a collectivization through the use of deictics, particularly in form of the usage of the first, and to a lesser extent third, persons plural (Billig, 1995: 70; Fairclough, 1995: 145; Reisigl & Wodak 2001: 83; van de Mieroop, 2005: 112). Therefore all instances "we" or "us" or their equivalents in other languages may give strong hints at the most important actors in the narrative.

Another typical speech figure, which establishes relevant collective actors are particularizing and generalizing synecdoches (Reisigl & Wodak 2001: 57). Typical cases for generalizing synedoches is the "collective singular", in which an action of a person is explained through one of his or her attributes, such as nationality or gender.

No matter which interpretative devices are used, the analyst should try to avoid creating a new set of frames for every study. Instead, interpretation should be guided by already established masterframes.

Quantitative Measurement of Frames

Quantitative studies in frame analysis are still quite rare, but their measurement models are usually more explicit. Usually, the reader is presented with a list of more or less parsimoniously identifiable frame terms, "attributes" or "devices," which were used as manifest indicators for the identification of frames (e.g., Ferree et al., 2002; Koella, 2003; Semetko & Valkenburg, 2000).

A number of studies uses multi-scale items to code the data material with the help of trained coders (d'Haenens and de Lange, 2001: 853; Semetko and Valkenburg, 2000; de Vreese et al. 2001). High intercoder reliabilities (de Vreese et al. 2001: 112f) suggest that this method is very reliable, if costly and time consuming.

More frequently, keywords are used as indicators (Entman, 1993: 53; Triandafyllidou & Fotiou, 1998: 3.7; Miller & Riechert, 2001a: 61ff). This might come as a surprise, because on a conceptual level, frames, more often than not, are latent, read: not spelled out in their entirety. Yet, it seems reasonable to assume that parts of frames become manifest in speech. If, say, a speaker has adopted or keyed an ethno-nationalist frame, i.e., the conception that quasi-primordial culturally fairly homogeneous groups of people can be delineated, we would expect this speaker to refer to some components of that frame in speech. She or he might, for instance speak about peoples, might allude to some historical continuities, or refer to specific (ethnic) nations, such as "the Dutch," etc. These speech figures in turn can be identified by keywords (Entman, 1993: 53; Triandafyllidou & Fotiou, 1998: 3.7; Miller & Riechert, 2001a: 61ff). The first task in the empirical investigation of frames thus becomes the detection of these keywords. To avoid a dependence on the creativity of the frame analyst (Maher, 2001: 84) and "researcher fiat" (Tankard et al., 1991: 5; Tankard, 2001: 98), it has been suggested to generate these keywords automatically, simply by mapping the most frequently occurring words or strings within the data (Koella, 2003: 7; Miller & Riechert, 2001a: 70). In this methodology, the most frequent strings in a set of data, for example newspaper articles or press bulletins, are calculated. From the resulting list, stop words such as prepositions are expunged. In a second step, those terms that according to with the highest Chi2-Square rank are most unevenly distributed between the source documents are chosen as keywords that are presumed to identify frames. As this is a purely mathematical method for the choice of keywords, it appears to avoid the researcher bias in the interpretative identification of keywords. In fact, however, its decision is to be made by researcher fiat, namely the determination of the optimal number of eigenvectors (Miller & Riechert, 2001b: 116). This decision sounds more "objective," as a number can be pegged to, but that number is an arbitrary one. Moreover, the procedure is deeply positivist (in a Carnapian sense), assuming that concepts should arise unmediated from the data. Yet, both epistemological and methodological positivism have  been rejectedeven by its former champions ( Popper, 1966). Even if we were to accept positivist logic, most statistical tests are based on a priori probabilities. By basing the decision about the choice of keywords on ex post covariances, these tests become meaningless. While this problem could be circumvented through a split sample, an even more severe problem is that empirically identified keywords clearly cannot be interpreted as an indicator of meaningful frames. Miller & Riechert (1994), for instance, found besides "environmental," "any," and "major" to be identifiers of the "environmental protection" frame. It seems obvious that these are not meaningful framing terms. Indeed, Koella (2003: 8), who most closely follows Miller and Riechert, deviates in this point, wryly noting that "each set of frame terms was reviewed in context." This proceeding, of course, in turn reintroduces interpretative leeway through the back door. Frequency counts do hint at possible keywords, but in the end an interpretative identification of relevant keywords seems to be the more appropriate and more common route (Andsager, Austin, & Pinkleton, 2001: 129; Tankard, 2001: 103; Tedesco, 2001: 2053, more technically centered: Miller, 1997: 369). Reading or listening to a reasonable amount of data, framing researchers should interpretatively uncover frames and their corresponding keywords.

Quantitative Validation of Frame Models

Currently, hierarchical cluster analysis is the most popular method for statistical validation of frames, once frames have been identified via itemized coding or keyword detection. That is, if one can speak of "popular", when merely a handful of references exist (Dyer, 1994; Koella, 2003; Landmann & Züll, 2004: 120; Miller, 1997; Miller & Riechert, 1994; Miller & Riechert, 2001b; Miller & Riechert, 2001b). There are a few problems with this methodology, though. To begin with, it requires specific chunks of data - documents with around 1,000 words -to perform best (Miller, 1997: 369). While this problem could be alleviated by slicing or aggregating data appropriately, the a general problem of all cluster analyses - be it k-means or hierarchical - cannot be circumvented, namely that it does not offer any real goodness of fit tests (Aldenderfer & Blashfield, 1984), which in turn makes it impossible to choose an optimum number of clusters on an empirical basis (Miller & Riechert, 2001b: 116; Trochim & Hover, 2003). That means that any number of frames could be posited throughout the texts, without any possibility to falsify any frame model, which, once again would return us to researcher fiat. On top, hierarchical cluster analysis assumes texts to belong to either one or the other frame. But it is entirely reasonable, and even likely, that speakers use any number of frames in a given text. In fact, many speakers actively engage in frame alignment processes such as frame bridging (Snow et al., 1986), which presuppose the existence of more than one frame in a text. Moreover, cluster analysis assumes a direct measuring model, but as has been discussed in the theoretical part of this paper, keywords are only indicators of latent frames.

Factor Analysis avoids these shortcomings of cluster analysis. It knows well-established goodness of fit criteria, it assumes a measurement model that does justice to the latency of frames, and it can decide on an empirical basis, which frame model is more adequate. Yet, to date we know only of one attempt to use frame analysis in framing studies (Risse & van de Steeg, 2003). While the headway made compared to cluster analysis is considerable, it seems puzzling that the authors do not even discuss the violation of the scale level assumptions of factor analysis, even though it has been shown that this violation can seriously affect the substantial results (Magidson & Vermunt, 2004).

In contrast, latent class analysis exhibits all required features factor analysis offers, but at the same time does not contain the same shortcomings, making it very well-suited for the analysis of quasi-idealtypical concepts (Hagenaars & Halman, 1989) such as frames. It should seem therefore straightforward to introduce it into frame analysis studies. Although the methodological principles of latent class analysis have been already developed in the fifties (Lazarsfeld, 1950), it has remained an esoteric statistical method for many social scientists. Basically, latent class analysis can be considered the equivalent of factor analysis for ordinally and nominally scaled variables (McCutcheon, 1987: 7). It examines, if a set of observable indicators can meaningfully be projected onto a smaller set of latent, that is, unobservable classes. Most important theoretical concepts, among them frames, do not translate straightforwardly into easily empirically observable, that is: measurable, indicators. Latent class analysis that expressly works with latent, read: unobservable, variables (Lazarsfeld, 1950: 363) is therefore in the analysis of frames superior to other log-linear models that operate exclusively with observable data. In comparison to cluster analysis, latent class analysis delivers more unequivocal results, as it allows for a number of well-developed goodness of fit measures. And while it shares with factor analysis the virtue of operating with latent variables, it does not contain the caveat of requiring hard to come by interval scaled data.

References

Valid HTML 4.01 Transitional