Who is missed by a national census? A review of empirical results from Australia, Britain, Canada, and the USA

Ludi Simpson and Elizabeth Middleton, Centre for Census and Survey Research^# , University of Manchester, M13 9PL, UK. June 1997. Email: ludi@man.ac.uk. 

The characteristics of those expected to be missed by censuses can be used to (a) improve census fieldwork procedures, (b) choose efficient stratifying and explanatory variables for post-enumeration coverage surveys, and (c) assess the impact of non-response on applications of census data.

1 Scope and purpose

Census non-response, also referred to as census undercount or under-coverage, refers to the difference between the census count of a population group and its true size. To the extent that use of census data assumes that the census count is the true count at the time the census is taken, census non-response introduces bias into each application. Census non-response when averaged across all groups in 1990/91 in the four countries covered by this review was: Australia (1.8%), Britain (2.1%), Canada (2.9%) and the USA (1.6%).

A person may be missed by national census procedures for various reasons. A household or institution with residents may not be contacted at all, or may complete a form but omit one or more residents. In either case the root cause may be unwillingness to complete the census form, or absence at the time of the enquiry, or oversight, or a breakdown of the census procedures themselves. There is a considerable literature discussing the reasons for non-response in sample social surveys, within textbooks (such as Moser and Kalton) as well as studies of the survey process itself (such as Groves and Couper 1991). Non-response in censuses, which is considerably lower than in social surveys, may have a different mix of causes. Manuel de la Puente (1993) reports ethnographic investigations at the time of the 1990 Census in the USA., but little other research has been conducted on the reasons for non-response to a census.

This review is not directly concerned with the reasons for non-response but with the characteristics of those who do not respond to a census. There are three purposes for this focus:

1. Improve fieldwork. The characteristics of past non-respondents can suggest improvements to fieldwork procedures for future censuses. Publicity and enumeration can be targeted at those groups which are hard to enumerate, or at areas in which they are most likely to live. Knowledge of the nature of non-respondents may suggest avoidable reasons for their having been missed, for example related to the wording of census questions.
2. Design coverage surveys. Post-census surveys to quantify coverage of the census are more efficient if characteristics associated with non-response are used in their design and analysis.
3. Adjust census counts. Estimated characteristics of non-response are required to make reasonable adjustment to convert census data into estimates of population characteristics.

A variety of methods have been developed to estimate the size and nature of census non-response. These methods have been reviewed by Don Kerr (1997) for the same four countries as are covered by this review. They are only referred to here when they affect the interpretation of results.

Section 2 discusses the various ways in which undercount has been expressed in Australia, Britain, Canada and the USA in their censuses of 1990 and 1991.

Section 3 presents the empirical results from the published literature of these censuses and that for 1981 in Britain. Comparison with non-response to social surveys in Britain is included to put the census results in the context of social investigations more generally.

Section 4 summarises and discusses the results.

The Appendix specifies one particular application of the results: the choice of plausible adjustments of 1991 Census data in Britain, in order to assess the likely impact of non-response on census applications.

 2 Definition of non-response

Each country’s census reports define the non-response rate for any group as the percentage net under-enumeration rate,

100 * (P-C) / P

where P is the estimated full population, and C is the published census count.

The non-response rate is usually positive but may be negative where over-enumeration, due for example to residents counted by the census in more than one place, outweighs under-enumeration.

In this review we report results for groups of residents and households of residents. Non-response for visitors tends to be relatively high (14% in Australia for example), but of no interest to most census applications.

It is not straightforward to compare non-response rates from different censuses. Four specific difficulties arise as follows,

2.1 Erroneous response can be confounded with non-response

Where the full population is estimated entirely independently from the census, non-response rates include an element for mis-reporting in the census. For example, if the census report of marital status is different from that derived from marriage registrations since the last census, the difference may lie with mis-reported marital status as well as with non-response.

An element of unreliability of census reporting within the measure of non-response is thus present in most demographic studies, and in the results from the USA and Australian censuses’ post-enumeration surveys. Where the population values are specifically an adjustment to the Census as in Britain and Canada’s reporting of their coverage surveys, then the non-response rates refer solely to non-response and not to the quality of census returns.

In this review the unreliability of census reporting is only thought to dominate the estimate of non-response for some estimates of marital status and some estimates of household composition, which will be mentioned at the relevant points in this review.

2.2 Published census counts include imputed records for some of those missed

Allowances for some of those not enumerated in the census are included in the census counts before they are published, as follows. In all four countries, enumerators report the estimated number of people in households that have not responded, and whole records for these people are imputed and added to the census database (amounting to 1.5% of the population in 1991 in Britain and 1.2% in Australia).

In Canada in 1991, further imputation of whole records into the census database, known there as random additions, was made as a result of sample checks on residents temporarily away from home and not included on their home census form (adding 0.33% of the population, and not undertaken in 1996) and dwellings mis-classified as unoccupied, (adding 0.45% of the population, 0.60% of households).

The non-response rate in census office reports is based on those missed from the published census tabulations, and therefore does not include those census records imputed into the census before publication. Imputed records represent additional people missing from the enumeration; their characteristics may differ from those missed from published census counts.

2.3 Joint probabilities of being missed

With the exception of age and sex, most reports from Census offices measure census non-response one dimension at a time. It is not possible to tell whether the relatively large likelihood of missing unemployed residents is reflecting the same phenomenon as the relatively large likelihood of missing residents in rented accommodation. Whether two categories are independently missed or not - whether their influences on missingness are additive or interact - is quite important to the purposes of this review but can seldom be assessed from the reports.

2.4 Exclusions

Residents of institutions such as prisons, military quarters and residential hospitals are omitted in the coverage surveys from which the socio-economic characteristics of non-respondents are derived, for all four countries. In the USA and Australia those in very sparsely populated or remote areas are also omitted. The small numbers of individuals involved will not affect the estimates of non-response for national classifications. But there are no estimates of non-response for those in institutions. 

3 Characteristics of those missed from censuses in Australia, Britain, Canada, and the USA

The tables in this section summarise results from the following sources:

• Australia 1991 survey. The Post Enumeration Survey reported in Australian Bureau of Statistics (1995).
• Britain 1991 survey. The Census Validation Survey reported in Heady et al. (1994, chapter 4). The survey succeeded in identifying only one sixth of the total estimated to be missing from demographic analysis; its reported non-response rates are relatively low and were generally not statistically significant.
• Britain 1991 demographic. For age and sex (Heady et al. 1994, Chapter 5), marital status (Morris 1997), and ethnic group (Simpson 1996).
• Britain 1981 survey. The Post Enumeration Survey reported in Britton and Birch (1985).
• Canada 1991. The net results from the Reverse Record Check and Over-coverage studies, in Statistics Canada (1994, chapter 9).
• USA 1990 survey. The Post Enumeration Survey reported in Hogan (1993).
• USA 1990 demographic. For age, sex and race, in Robinson et al. (1993).
• Britain 1991 Labour Force Survey. The non-response to this national sample survey was modelled in a logistic regression, in which only the statistically significant determinants of non-response were reported in Foster and Bushnell (1993). Except when stated otherwise, the results reported here are in each case consistent with the four other sample surveys whose response rates were examined in 1991 and reported in the same publication: General Household, National Travel, Family Expenditure and National Food surveys.

The text in this section makes reference to other studies which gave results insufficiently comprehensive or comparable with other countries to present in the tables. Deserving special mention among these is the USA logistic regression model of USA 1990 Census non-response (Alho et al 1993). It is the only study to test a variety of predictors of individual probability of non-response. This study was limited to minorities in central city areas; it nonetheless provides useful evidence of the independence of two or more predictors, and is referred to as the USA 1990 regression in the text below.

The tables below include all the results from the publications above. The results are strikingly few in number. Most coverage studies focus on interregional differences, age and sex, to meet the main demands of population estimation.

3.1 Age and sex

Each country reports the national age-sex structure of non-response in some detail; it is required for the population estimate that is used and updated during the following years. In each country census non-response is highly related to age and sex.

The following common features are apparent:

• Young children are less reliably captured than children in their early teens, for both sexes.
• Young adult men are the hardest group to enumerate.
• Among adults, older adults are more easily enumerated than younger adults.

These patterns are persistent over time as well as between the four countries, and have long been recognised.

In Britain in 1991 and the USA in 1990, elderly women above 75 years of age were more difficult to enumerate than middle-aged women for reasons that neither countries’ reports attempt to explain.

In Britain and the USA a significant proportion of newly born babies have been omitted from census forms (Werner 1984 for Britain).

There are specific differences between countries and over time for some age-sex groups. Some of these may relate to structure of education, work and housing in each country, while others will reflect errors in the measurement of non-response rather than real differences in enumeration.

Marital status of adults: non-response rates

1 Total non-response including those not contacted
² Including children under 16
³ Divorced and widowed
⁴ Not reported: unreliable census responses biased the non-response estimate

There is considerable consistency, that some single people and some divorced men are hard to enumerate, raising the non-response rates for these groups to over twice that of married people.

The association of non-response with marital status is not simply a consequence of the high probability that young adults, who are mainly single, being missed from the census. The only study to report cross-classified results for age, sex and marital status is the 1991 demographic study in Britain. It specifically disentangles the impact of mis-reporting marital status on non-response. It then finds that the higher non-response among single and divorced men compared to married men is not only repeated at most ages, but is most evident for those in their late twenties and early thirties, and those aged over 75.

The USA 1990 regression also finds that independently of age, married people were more likely to be enumerated than others. It seems plausible that marriage usually coincides with stability of address and a household that is more often occupied to be contacted in a census. Divorcees may revert to a singles lifestyle, particularly divorcees without child-care responsibility.

Migration (address one year previously): non-response rates

1 Non-contacts.
² All addresses: those with the same address were not reported separately.

From Britain and Canada there is clear evidence that those who have moved address in the previous year are hard to enumerate in a census. The study of sample surveys in Britain (final column of the table) suggest that this is related to difficulty of contacting such ‘movers’ rather than their unwillingness to complete survey schedules: for those contacted, migration was not a significant predictor of non-response.

There seems no reason why movers during the year prior to the census should in general be hard to enumerate. It seems likely that among all movers those who are hard to enumerate are those frequent movers who may not consider their current address as a permanent residence and who spend a substantial amount of time at other addresses.

Employment status: non-response rates

4 Retired and other inactive

Neither economic nor employment status have been used as dimensions of non-response in Australian or USA coverage reports.

In Britain, unemployed people have had a consistently higher chance of not being counted in a census. In 1981 when this was reported separately for men and women, the difference was particularly great for men. In Canada, a higher non-response was also found for those without a job in the year previous to the census.

The Canada 1991 report also shows that income level was closely related to response rates. Those with negative or no income had non-response of 5.6%, while each of six bands of increasing income had progressively smaller non-response rates.

Sample survey results in Britain in 1991 (Foster and Bushnell 1993) do not show the same relationship of non-response to employment status. Along with self-employed and unskilled manual heads of household, unemployed heads of household tended to be harder to enumerate in other surveys but not in the Labour Force Survey. It is not obvious why this should be so.

Ethnic group, race, country of birth: non-response rates

8 Pakistan, India, Bangladesh. ⁹ Asian and Pacific Islander. ¹⁰ Reservation Indian.

Race and ancestry have been asked in USA Censuses for many decades, and non-response has been consistently higher among the black population and the more recently identified Hispanic population. In Britain each resident’s ethnic group was requested for the first time in 1991, with non-response also higher in the Black and South Asian (Indian subcontinent) minority groups according to the survey report.

Country of birth has been a common proxy for ethnic group in studies in Britain, although it excludes the children of immigrants. In 1991 it allowed demographic comparison of census results with births in the years prior to the census which confirmed considerable extra non-response among children of those born outside Britain.

Race and ethnic group are social classifications that depend significantly on historical, political and legal conditions in each country, as evidenced from the many notes to the table above. Each group’s non-response rate is likely to depend on local conditions including the ways in which census fieldwork adapts to cultural and social diversity. Nonetheless, high non-response among minorities in different countries may have common causes, including separation of communities from the nationally ruling culture of which the Census is part, and poor economic and urban housing conditions which are (from other tables in this review) associated with non-response.

There are also particular difficulties of enumeration for some groups. Geographical isolation adds to the difficulty of enumeration encountered for Aboriginal and Torres Strait Islanders in Australian censuses (Evans et al. 1993) and Indian reservations in Canada and the USA, all of whom had significantly high rates of non-response in recent censuses. Lack of knowledge of official languages in Canada was associated with non-response of 11% in Canada, three times the average of 2.9%.

In some cases, the youthful age structure of minority groups which have developed from recent migration may account for higher non-response rates, as suggested in Australia for those of New Zealand birth (3.5% non-response, against a rate of 2.2% for all groups). However the 1990 USA regression found black residents less well enumerated over and above difficulties associated with age, sex and social factors, and the main USA studies found race differentials in non-response maintained (though slightly reduced) across sex and tenure classifications.

Difficulties of attaining high response among minority race or ethnic groups are also evident in studies of sample surveys, and in registration of voters (for example Smith 1993).

Household size: non-response rates

1 All non-contacts and refusals, people missed; non-contact was particularly likely among residents of one-person households . ²4 or more. ³ 5 or more. ⁴ 6 or more.

Size of household has been a concern of census coverage studies only in Britain. There, the pattern of one-person households being particularly missed was found in 1981 and in 1991.

However the 1991 post enumeration survey found that this was due to a problem of mis-reporting households. People in one-person households were enumerated but sometimes wrongly included with another household. The table shows that for individuals it is those from larger rather than smaller households that are missed. This must be partly because larger households have more people to miss. Additionally, the limit of six people on a single 1991 census form probably hindered the collection of complete information on larger households.

The 1981 post enumeration survey in Britain found that one-person households of one adult under pensionable age (men under 65 and women under 60) accounted for 34% of all missed households. Their non-response rate of 2.2% compared to 1.2% for single adults with children, 0.3% for one adult of pensionable age, and 0.6% or less for each other type of household.

Outside of Britain, only the 1990 USA regression addressed household size, as a continuous variable. It found easier enumeration of people in larger households (and that this was the case particularly for older residents); this is opposite to the finding in Britain.

 3.7 Relationship to household head

In the 1990 USA regression, those unrelated to the first person on the census questionnaire had a lower chance of being enumerated than those related. On the other hand, in the 1991 survey in Britain there was recorded over-enumeration of unrelated members of households, though there were significant numbers of missed persons whose relationship to the head of household could not be determined.

 3.8 Tenure

Tenure: non-response rates

1 The figures given are for non-Hispanic Whites; the differences for Black, Hispanic, and Asian/Pacific Islander groups each show similarly higher non-response for those not owning their housing. 

In Britain, Canada and the USA, tenure has had an enduring relationship to non-response, with owners having considerably lower non-response rates than non-owners. In Britain the distinction between privately rented and socially rented (from the state or a housing association) is important: the private renters had higher non-response rates in both 1981 and 1991.

The 1990 USA survey found that ‘tenure is as important as race in explaining undercount’ (p.1055). The 1990 USA regression supported this result, finding a large decrease in the probability of enumeration for renters after race and other characteristics were taken into account. For three of the four regions studied this decrease was significantly more acute for renters with smaller household size.

Renting in these three countries is clearly associated with low response to the national census, perhaps due to a concentration of poor living conditions, dwelling structures of a difficult nature to enumerate.

 3.9 Type of accommodation

Type of accommodation: non-response rates

4 Duplex, apartment accommodation on two floors. Often the upper floor is rented from an owner who is living on the main floor. 

Non-response is relatively low in dwellings that are occupied by a single household. In Britain, flats or apartments tend to be rather more difficult to enumerate, and shared, converted dwellings and homes which are not permanent buildings have particularly high non-response rates. In Canada, accommodation that was attached to or located within a larger dwelling also stood out as hard to enumerate.

The USA survey (p.1156) also highlights households missed within enumerated dwellings as likely to be erroneously enumerated. The USA 1990 regression found that those living in multi-unit dwellings were more likely to be missed than others with otherwise similar characteristics.

The results here will be related to those for tenure, if these shared dwellings often involve privately rented flats. The results here are very striking and consistent. However, the impact on coverage of people may be over-stated if (as discussed already in section 3.6 on household composition) households were amalgamated in the enumeration rather than missed altogether.

4 Discussion

The review shows that non-response has several social dimensions.

Male single and divorced, migrants, unemployed, minority race or ethnic groups, privately tenants, and sharers of dwellings are each separately associated with recent census non-response rates in the four countries reviewed, in addition to the age-sex pattern of non-response that has long been acknowledged.

Mis-reporting of households within a dwelling confounds the assessment of coverage for household size and composition, because separate households my be counted as one. This requires clarity in census instructions and coverage measurement.

The 1991 Survey results in Britain are shown in the review above to be consistent in many aspects with the results in Britain from the 1981 census, and with those of Australia, Canada and the USA, despite its capture of only a minority of the undercount.

There are differences between censuses and surveys. The relative non-response between social groups appears to be less for sample surveys than for censuses, if groups are compared by taking the ratio of their non-response rates, and the 1991 Labour Force Survey in Britain is taken to be representative of sample surveys.

The census has lower non-response rates than sample surveys. It has to cope with a more extreme set of missing people than surveys and so one might expect the differentials to be bigger than in surveys, as they are.

This is a useful reminder that the non-respondents to a census make up a very small group, and are a minority within any of the groups singled out by this review as susceptible to undercount. A single person, a migrant, a young male, a house sharer, are each more likely to be counted than missed: none of the single categories used in the review, nor for that matter any in the census, captures those missed very specifically.

Therefore, accurate adjustment of census data would attempt to find interactions between demographic and social variables that better identify the nature of people who are missed.

The review has practical ends, as suggested in the introduction. The variables identified in this review as clearly associated with census non-response can be used to target revised fieldwork procedures to avoid non-response in the 2000 round of censuses.

These same variables will be suitable to stratify and to analyse post enumeration surveys.

Finally, in Britain in 1991, the results will be used to adjust census counts to explore the impact of adjustment on census applications. This proposal is discussed further in the Appendix.

Appendix Plausible non-response rates for Britain in 1991

The authors have used the results of the current review to draw up several sets of ‘non-response differentials’ as in the table below. Each set comprises some socio-economic categories for which non-response factors are different. Choosing more than one set of differentials allows us to explore the range of impact of non-response that is plausible, representing uncertainty around any estimation of non-response. It will also show whether particular census applications are affected by non-response in some social groups rather than in others, in order to indicate the non-response that should be a priority to reduce.

The differentials will be applied to the census output for local areas. The age-sex-area non-response that has already been estimated by ONS and the Estimating with Confidence project will be maintained. The procedure to be adopted is specified in Simpson (1997). The non-response already estimated for an age-sex-area population group is divided between socio-economic groups such that their relative non-response factors¹ are as set by the differentials.

The table shows two columns, each showing three sets of differentials. In each case the first column of differentials broadly reflects the GB 1981 and 1991 survey results in this review, and the non-response factors are expressed as differentials from the first category given which has least non-response. The second column gives a more extreme set of differentials from the first category by doubling those of the first column. This reflects the possibility that in 1991 non-response was more extreme than in 1981 in a way that was not identified by the Census Validation Survey; the choice of double is arbitrary.

Six sets of plausible non-response differentials Col 1 Col 2

To these six sets of non-response differentials, we can add a seventh that combines all three variables of the first column, such that the combination of categories gives rise to a multiplicatively higher non-response. The differential for White social renters out of employment will be 1x3x5=15, and so on.

Notes

1 In the specification the differentials refer to the adjustments to be made to the census, ie 100*(Population - Census)/Census. These adjustments are termed non-response factors rather than the non-response rates defined at the beginning of this review which have the Population as denominator. For a given non-response rate, the non-response factor will be slightly larger: for example a group with non-response rate of 10% requires an adjustment by a factor of 1/9 or 11%. The factors can rise over 100% if the non-response rate is over 50%.

 References 

Alho, J. M., Mulry, M. H., Wurdeman, K., & Kim, J. (1993). Estimating Heterogeneity in the Probabilities of Enumeration for Dual-System Estimation. Journal of the American Statistical Association, 88, 1130-1136.

Australian Bureau of Statistics. (1995). Census of Population and Housing Census 91: Data Quality Undercount. Canberra: Australian Bureau of Statistics.

Britton, M., & Birch, F. (1985). 1981 Census Post-enumeration survey: An enquiry into the coverage and quality of the 1981 Census in England and Wales. London: HMSO.

De la Puente (1993) Why are people missed or erroneously included by the census: a summary of findings from ethnographic coverage reports, pp29-79 (including discussion) in 1993 Research Conference on Undercounted Ethnic Populations, Proceedings. US Department of Commerce Bureau of the Census: Washington DC.

Evans, Julie, Kahles, Dietmar, and bate, Catriona (1993) 1991 Census data quality: aboriginal and Torres Strait Islander counts. Census Working Paper 93/6. Australian Bureau of Statistics: Canberra.

Groves, R. and Couper, M. (1991) Towards a useful theory of survey participation, presented at the 1991 meetings of the American Statistical Association.

Foster, K., & Bushnell, D. (1993). Non-response bias on government surveys in Great Britain. Fifth International Workshop on Household Survey Non-Response.

Heady, P., Smith, S., & Avery, V. (1994). 1991 Census Validation Survey: coverage report. London: HMSO.

Hogan, H. (1993). The 1990 Post-Enumeration Survey: Operations and Results. Journal of the American Statistical Association, 88, 1047-1060.

Kerr, Don (1997) Alternate methodologies in the evaluation of Census coverage: Canada, the United States, Britain and Australia. Demography Division, Statistics Canada: Ottawa. Draft.

Morris, M. (1997). The rebased 1991 population estimates by marital status. Population Trends, in press.

Moser, Claus and Kalton, Graham (1971) Survey methods in social investigation. Heinemann: London.

OPCS/GRO(S) (1995) 1991 Census General Report, Great Britain. HMSO: London.

Robinson, J. G., Ahmed, B., Das Gupta, P., & Woodrow, K. A. (1993). Estimation of Population Coverage in the 1990 United States Census Based on Demographic Analysis. Journal of the American Statistical Association, 88, 1061-1071.

Statistics Canada. (1994). Coverage 1991 Census Technical Reports. Statistics Canada: Ottawa.

Simpson, S. (1996). Non-response to the 1991 Census: the effect on ethnic group enumeration. In D. Coleman & J. Salt (Eds.), Ethnicity in the 1991 Census (Vol. 1 Demographic characteristics of the ethnic minority populations, pp. 63-79). HMSO: London.

Simpson, S (1997) Socio-economic differentials in non-response: a procedure for adjusting census data. Working paper no. 1. Centre for Census and Survey Research, University of Manchester, M13 9PL, UK.

Smith, S. (1993). Electoral registration in 1991. OPCS: London.

Werner, B. (1984) Infants aged under 1 in the census, 1861-1981. Population Trends, 38: 18-24.