J . R. Statist. Soc. A (1996) 159, Part 1, pp. S 2 by Juan Pardo

J . R. Statist. Soc. A (1996) 159, Part 1, pp. S 2 0

Political Polling: 95% Expertise and 5% Luck By ROBERT WORCESTER? Market & Opinion Research International Ltd, London, UK [Read before The Royal Statistical Society at a meeting on 'Opinionpolls and general elections' on Wednesday, June 14th, 1995, the President, Professor A. F . M. Smith, in the Chair]

SUMMARY

The record of British election polls was good-until the general election of 1992. The Market Research Society's inquiry into the performance of the polls in 1992 found inadequacies in the implementation of the sampling system used, evidence of a late swing and some reluctance of Conservative supporters to reveal their loyalty; but it generally endorsed the principle of well-conducted quota polling and found that variations in methodological detail had nil effect on the results. The evidence is presented and some possible future developments to counter the 'spiral of silence' are discussed. Keywords: MARKET RESEARCH SOCIETY INQUIRY REPORT; 1992 GENERAL ELECTION, NONRESPONSE BIAS; OPINION POLLS; PUBLIC OPINION, QUOTA SAMPLING; SPIRAL OF SILENCE; SURVEYS; TELEPHONE POLLS

1 . INTRODUCTION

Price (1992) in his book Public Opinion argues that opinions and attitudes have been said to differ conceptually in at least three ways. First, opinions have usually been considered as observable verbal responses to an issue or question, whereas an attitude is a covert psychological predisposition or tendency. Second, although both attitude and opinion imply approval or disapproval, the term attitude points more towards affect (i.e. fundamental liking or disliking), opinion more towards cognition (e.g. a conscious decision to support or oppose some policy, politician or political group). Third, and perhaps most important according to Price, an attitude is 'traditionally conceptualised as a global, enduring orientation toward a general class of stimuli, but an opinion is viewed more situationally, as pertaining to a specific issue in a particular behavioural setting'.

Opinions in my view are those low salience, little thought about reactions to pollsters' questions about issues of the day, easily manipulated by question wording or the news of the day, not very important to the respondent, not vital to their wellbeing or that of their family and unlikely to have been the topic of discussion or debate between them and their relations, friends and work-mates. These normally would for most people include their satisfaction with the performance of the parties and their leaders, economic optimism and pessimism, and the salience of the issues of the day, but not voting intention. Attitudes derive from a deeper level of public consciousness, are held with conviction and are likely to have been held for some time and after thought, discussion, perhaps the result of behaviour (Festinger's cognitive dissonance), and ?Addressfor correspondence: Market & Opinion Research International Ltd, 32 Old Queen Street, London, SWlH 9HP, UK. O 1996 Royal Statistical Society

0035-9238/96/159005

WORCESTER

p a r t 1,

harder to confront or confound. Examples of these are the Scots' support for a separate assembly, held with force over generations and based on strong beliefs that they are not fairly represented either in Parliament or in our system of government, perhaps attitudes to the taking of medicines or exercise, forms of education, local authority service delivery for services used frequently and by large percentages of citizens such as rubbish collection, streetlighting and schools. With some, voting intention derives from their attitudes; with most, from their values. Values are the deepest of all, learned parentally in many cases, and formed early in life and not likely to change, only to harden as we grow older. These include belief in God, attitudes to religion generally, views about abortion or the death penalty, family values and the like. For many people, these include voting intention, learned in childhood from parental influence, class and peer group identification. I define an opinion poll as 'a survey of the views of a representative sample of a defined population7 (Worcester, 1991). What polls (and I use the term more or less interchangeably with surveys, although there are those who use 'polls' only to describe 'political' soundings) cannot tell us well are likely future actions of the public generally and especially the future behaviour of individuals. They are better at telling us what, rather than why (although the latter is the principal function of qualitative research and especially focus groups, which concentrate on the interaction of the group rather than the question and answer, 'expert'-on-respondent, format of the individual depth interview). Polls are not particularly good at exploring concepts that are unfamiliar to respondents, nor are they good at identifying the behaviour, knowledge or views of tiny groups of the population, except at disproportionate expense. They are useless in telling us much about the outcome of an election weeks or even months or years in the future. Nevertheless, the voting intention question is valuable for what it summarizes about people's attitudes and values at the moment. Polls are a marriage of the art of asking questions (Payne, 1980) and the science of sampling. The conduct of survey research is quite a simple business really; all that needs be done is to ask the right sample the right questions and to add up the figures correctly, the latter task including proper analysis and reporting. 2. PUBLIC OPINION IN A DEMOCRATIC SOCIETY

David Hume said in 1741 that 'All government rests on public opinion7, and Aristotle stated centuries earlier that 'He who loses the support of the people is a king no longer7. Somewhat more recently Abraham Lincoln said that 'Public opinion is everything7and went on to avow that he saw his role as the elected leader of the USA as finding out what his electorate wanted and, within reason, giving it to them. Burke's 1774 belief (Hoffman and Ross, 1970) that his vote in Parliament was determined by his own judgment rather than the wishes of his Bristol electors is often quoted admiringly by those who would dismiss poll findings; seldom recalled is that in the following election his electors chose another to represent them in Parliament! Nor is it recalled that in 1780, after his defeat, he said that 'The people are the masters' and he wrote that 'No man carries further than I do the policy of making government pleasing to the people. I would not only consult the interest of the people, but I would cheerfully gratify their humours.'

19961

POLITICAL POLLING

One important point that Lippmann (1922) made is that inevitably our opinions cover a bigger space, a longer reach of time and a greater number of things than we can directly observe. They have, therefore, to be pieced together out of what others have reported and what we can imagine. We do not measure truth, we measure perceptions. But the reality of the world of public policy, as well as the media and industry, is that it is perception, not fact, that determines public opinion. My position on the role of polls, surveys and assessment of public opinion is one not of advocacy of any particular policy, subject or topic, but of the provider of both objective and subjective information, obtained systematically and objectively, analysed dispassionately and delivered evenly (Worcester, 1981). 3. THE ART OF ASKING QUESTIONS

A good question should be relevant to the respondent, easily understood by the respondent, unambiguous in meaning and should mean the same thing to the respondent, the researcher and the client for the results, relate to the survey objectives and not be influenced in any untoward way by the context of the questioning. 4. THE SCIENCE OF SAMPLING

I arrived in the UK some 26 years ago not knowing anything but two-stage probability sampling, and my institute, Market & Opinion Research International (MORI), does more than a third of its survey research by using probability samples. But we also believe that, in the heat of a general election, we are worse, not better, served by the application of probability techniques. The principal reason is time, not cost, although cost is a factor. Probability samples given less than a week to conduct fieldwork rarely rise above a response rate in the 50s; even three weeks' fieldwork achieves samples in the low 60s, and the 70s is thought excellent by most government agencies who waste, in my view, millions of pounds of tax-payers' money by insisting on strict probability samples with many repeated attempts to contact and interview the designated respondents when looser approximations would suffice for the public policy purpose in mind. After all, even if precision to the usual 95% confidence level could be guaranteed, how many times does the Minister care whether the finding is 77% or 73%, so long as he or she knows that about three-quarters of the electorate favour the policy? Certainly, if the proof of the pudding is in the eating, then the test of five elections in the 1970s, comparing nine probability 'forecasts' with election results, with an average 'error' of 6.3%, was hardly to be preferred to the 3.0% error of the 19 quota samples reported simultaneously (Table I). And specifically, at the 1975 European Economic Community (EEC) referendum, when the answer was a straightforward yes or no, four out of the five quota samples were within 1% whereas the probability sample was far out, in an election where the tracking polls showed that there was virtually no shift in the opinions of the electorate over the three-week election period. But perhaps the best test that is independent of an election with its possibility of late swing was at the time of the American bombing of Libya when three polls conducted independently, with different sampling points, different interviewers and different question wordings, found virtually identical results.

[Part 1,

WORCESTER TABLE 1

Comparative record of random and quota polls in British elections Election

Average error (%) in lead for the following types of po1l-f: Random polls

Quota polls

7.3 (3) 3.7 (2) 8.0 (2) 10.0 (1) 1.6 (1)

5.4 (2) 1.7 (4) 3.5 (4) 3.9 (5) 1.7 (4)

6.3

3.0

1970 February 1974 October 1974 1975 (referendum) 1979 Mean error 1970-79

?The number of polls is given in parentheses.

TABLE 2

Accuracy of the final polls, 1945-92

Year

1945 1950 1951 1955 1959 1964 1966 1970 1974, February 1974, October 1979 1983 1987

Mean error gap (%) Average error per party (%)

No. of polls

3.5 3.6 5.3 0.3 1.1 1.2 3.9 6.6 2.4 4.5 1.7 4.5 3.7

1.5 1.2 2.2 0.9 0.7 1.7 1.4 2.2 1.6 1.4 1.O 1.4 1.4

Average

3.3

1.4

1992

8.7

2.7

5 . WHAT THE 1992 BRITISH GENERAL ELECTION TAUGHT

BRITISH POLLSTERS

What is the record of the British polls over the years? On the whole, it was rather good (Table 2), until 1992! Some may quibble, but their record, especially over the past 20 years, has been very good indeed. In 1983, all six were within sampling tolerance, given the sample size, three within f1%. In 1987, all six were within k l i % on average, for the share of each party. Over the 13 British general elections since the war until now, and over the period of election polling, until now, polls have performed, on average, about as well as could be expected. The margin of error on the gap should be approximately double that of the share, according to sampling theory, and indeed it is, according to the empirical evidence of 51 election forecasts over 42 years.

19961

POLITICAL POLLING TABLE 3

Eve-of-poll predictions, April 1992 Results from the following polls: Harrk

MORI

NOP

Gallup

ZCM

General election

2186 Sample 2210 1731 1746 2478 9th April Fieldwork 4th-7th April 7th-8th April 7th-8th April 7th-8th April 8th April 42.8 38 Conservative vote (%) 38 38 39 38.5 35.2 38 Labour vote (%) 40 39 42 38 18.3 20 Liberal Democrat vote (%) 18 20 17 20 4 3.7 4 3 2 3.5 Other parties (%) 0 -2 -1 -3 0.5 Conservative lead (%) 7.6 7.6 8.7 (average) 9.6 8.6 10.6 7.1 Error on lead (%) 2.4 2.67 (average) 2.55 2.75 3.4 2.25 Error on share (%)

About a quarter of the polls since 1945 have been able to estimate the Labour and Conservative share of the vote to within k l % , and over 50% have forecast to within f2%. Of these 51 polls, 29 have overestimated the Tory share of-the popular vote and 29 overestimated the Labour share; 20 underestimated the Tory share and 19 underestimated the Labour share. There is no sign of a bias. On average they have marginally underestimated the Tory lead over Labour, but as polls do not measure postal voters and these are generally agreed to favour the Tories-perhaps by as much as 3:1-the direction of these averages is not surprising, and in any case the slight bias has been only about ;% in each direction, or an error in the lead of 1%. 6.

THE 1992 BRITISH POLLS GOT IT WRONG!

But the polls in the 1992 British general election have much to answer for, for one thing the most exciting election night in a long time! The five major pollsters, Gallup, Harris, International Communications and Marketing (ICM), MORI and National Opinion Polls (NOP) averaged a 1.3% Labour lead; the final result was a 7.6% lead for the Conservatives (Table 3). The polls backed the wrong winner, by a wider margin than can be accounted for by sampling error, and were widely criticized for their inability to perform to expectation. 7. CAMPAIGN POLLS

What were these polls showing? They showed quite a steady picture, steady across polling organization, and, generally, over time. They certainly were not out of line with the final polls -indeed, on average they found a bigger Labour lead. Whatever problems affected the final polls affected the earlier campaign polls as well. The campaign tracking polls were also broadly in line with the two national panel studies that were published at the same time, and the results of all of these face-to-face polls were matched by the small number of telephone polls which were conducted. 8. WHAT WENT WRONG?

Much space in the newspapers was given over after the general election to

WORCESTER

[Part 1,

'Pollsters admit election mistakes' (April 30th' 1992) and 'The invisible voters who fooled the pollsters' (May lst, 1992) in The Independent, as well as to the various letter-writers' expressions of their own prejudices. Just a day or two after the election, letter-writers and critics alike were dismissing any 'late swing lame excuses' from the pollsters. There were many pundits' and psephologists' commentaries and readers' letters in The Times, The Daily Telegraph, The Independent, The Guardian, The Financial Times and elsewhere following the election, expressing various opinions why the opinion polls 'got it wrong' on April 9th. They were all good fun. Their suggestions are set out-and, mostly, easily refuted-in the next few sections. All the polling companies have, naturally, carried out their own reviews of the problems and an exhaustive two-year inquiry by a committee set up by the Market Research Society (MRS), consisting of pollsters, academics and other experienced market researchers who were not involved in opinion polling, was reported in July 1994. Their conclusions and recommendations have a great many lessons for both pollsters and poll readers (Market Research Society Working Party, 1994).

9. SAMPLE SIZE

One of the most popular reactions to the polls' failure was to blame the sample size: it is manifestly apparent that the sample size is not the key to the quality of findings. The ICM poll taken in the third week of the campaign, among a sample of 10460 electors in 330 sampling points and taken over nearly a week including a week-end, was almost identical in its voting intention with the findings of a smaller poll conducted over the same period. Interviewing on March 3lst-April 3rd, ICM recorded 36% for the Conservatives, 39% for Labour and 20% for the Liberal Democrats. At the same time, on March 3 1st-April 1st, NOP interviewed 'only' 1302 electors in 83 sampling points and found 37%, 39% and 19%. The fact that the final polls of the campaign, all taken a day or two before polling day, were so consistent -all five had the Tories at 38% f 1%, Labour at 40% f2% and the Liberal Democrats at 18% f2% -suggests that neither sample size nor sampling variation was responsible for the magnitude of the difference between the final polls and the actual result. Despite this, the British public remain unconvinced: a Gallup survey in April 1992 found 63% of British adults saying that it is not possible 'from a survey of 1000 or 2000 people . . . to give a reliable picture of British public opinion'. This is one area in which the pollsters' biggest challenge is probably to educate the British public.

10. MILLIONS OF EXPATRIATE TORY VOTERS? Another letter-writer claimed government by fraud, reporting that one of the iirst Acts of the 1987 Conservative government was to give the postal vote to 2 million expatriate Britons, but ignored the fact that only 34454 of these expatriates registered to vote. At an average of 53 voters per constituency, they could have made some difference, but not that much -the registered overseas electorate exceeded the majority in only two seats.

19961

POLITICAL POLLING

11. THE MENDACIOUS BRITISH PUBLIC?

A fellow letter-writer spoke up confidently to report that there was 'no eleventhhour swing to the Conservatives', and reported himself to be in a state of profound depression and dismay; no wonder he is depressed, if he is convinced that the British (against all my experience to the contrary) are 'a nation of liars'. One academic critic even titled his paper with implied agreement with this hypothesis (Crewe, 1992a). Other letter-writers also suggest that millions of people lied to interviewers. In the case of the three panel studies that were undertaken and which had comparable results both before and after the election, this would involve repeated and consistent lying. In the campaign tracking polls, it would have to assume that respondents lied consistently on a bank of other, non-voting, questions that they answered as well, since there is no evidence of inconsistency between voting and attitudinal questions. The MRS inquiry found no evidence of widespread lying. 12. RANDOM SAMPLING MYTH

The leader in The Independent on April 30th raised another point, alleging that 'Random polls have proved more accurate' (thari quota polls). This is an enduring myth with the polls' critics, but in fact in error. British election polls conducted by random sampling during past national elections where both methods were used (including the 1975 EEC referendum) were less, not more, accurate, than those conducted by quota, owing to poor compilation of registers and late swing after the bulk of the interviewing had been done. 13. INTERVIEWING IN STREET

Nor is there any evidence that other variations in the sampling design would have improved matters. Differences in interviewing procedures played no role. Although most of the polls were conducted in street, a few were conducted partly or exclusively in home. Nearly identical poll results were obtained (Table 4). 14. TELEPHONE POLLING

Although telephone polling in British elections had some problems in its early days, in 1992 the small number that were published found results that were entirely comparable with those from the face-to-face polls (Table 5). TABLE 4

Comparison of in-street and in-home polls: average party sharest Poll

In street In street and in home In home

Average share (%) for the following parties: Conservative

Labour

Liberal Democrat

Other

38 38 39

40 40 40

18 18 17

5 4 5

tSource: Market Research Society Working Party (1994).

Conservative lead (%)

-2 -2 -1

part 1,

WORCESTER TABLE 5

Comparison of phone polls and face-to-face pollst Average share (%) for the following parties:

Poll

Phone polls (7) Face-to-face polls (47)

Conservative lead (%)

Conservative

Labour

Liberal Democrat

Other

38.8 38.2

39.5 40.0

17.3 17.7

4.4 4.1

-0.7 -1.8

tSource: Market Research Society Working Party (1994). TABLE 6

Comparison of one-day, two-day and longer pollst Average share (%) for the following parties:

Poll

Conservative

Labour

All polls (54)

All one-day polls (15) All two-day polls (24) All longer polls (15)

38 38 38

Excluding telephone polls (47) One-day polls (13) Two-day polls (23) Longer polls (1 1)

38 38 38

Liberal Democrat

Conservative lead Other

I%)

40 40 40 40 40 40

Excluding telephone polls and later waves of panel studies (44) One-day polls (13) 38 40 Two-day polls (23) 38 40 Longer polls (8) 39 40

?Source: Market Research Society Working Party (1994).

15. ONE-DAY, TWO-DAY AND LONGER POLLS

There was no difference in results between those polls conducted as 'one-day quick polls' and those with fieldwork spread over a longer period (Table 6). Nor was there any evidence that it made any difference whether or not fieldwork periods included a week-end (Table 7). 16. SELECTION OF SAMPLING POINTS

But what about the actual constituencies sampled? If they had happened to be biased to Labour, that might have caused an error. The MRS inquiry compared the results in 1987 and 1992-in each polling company's selection of constituencies with the national result. It found that four of the five companies had selections that were very slightly biased to Labour in 1992, and also that the same bias had existed in those constituencies in 1987-it was not simply bad luck in picking constituencies which swung unusually. But the average error was barely 1% on a Conservative lead

19961

POLITICAL POLLING

TABLE 7

Week-end and weekday pollst Average share (%) for the following parties:

Poll

Conservative

Labour

Liberal Democrat

Other

38 38 38 39 39

40 40 40 39 39

18 18 18 18 18

4 4 4 5 4

All polls (54) With some Saturday or Sunday fieldwork (17) With no Saturday or Sunday fieldwork (37) With some Saturday fieldwork (14) With some Sunday fieldwork (10)

Conservative lead (%)

-2 -2 -2 0 0

tSource: Market Research Working Party (1994).

over Labour, and most of the potential effect should have been corrected by the operation of the quota system to ensure a representative social profile. Besides, the company whose selection of constituencies was representative fared no better than the others in 'predicting' the final result. Plainly this was not a major cause of the error. 17. LATE SWING

The MRS inquiry isolated three root causes of the discrepancy between the poll findings and the final result. The first of these was late swing. Polls are snapshots at a point in time, and that point is when the fieldwork was done, not when the results were published. If after that voters change their minds, the 'don't knows' decide to vote after all and one party's supporters become so apathetic that they stay at home, the polls will be wrong. It has become fashionable in certain quarters to decry the late swing explanation (Clifford and Heath, 1994), and even to suggest that there was no movement of opinion throughout the campaign. Late swing was not the only problem (or the exit polls should have been accurate), but no-one at the time doubted that it was happening. Forgotten now are the headlines on election day: 'Late surge by Tories closes gap on Labour in final hours of campaign' was the banner in The Times; 'Tory hopes rise after late surge' was the headline over the splash in The Guardian. In The Daily Telegraph it was 'Tories narrow gap', and in the Financial Times the banner read 'Opinion polls indicate last-minute swing from Labour to Tories' whereas the Daily Express trumpeted 'Tory surge: polls show late boost for Major'. Labour's peak came on the day of its Sheffield rally, eight days before polling day, with published leads sufficient to give them an overall majority. But Labour's triumphant rally proved the beginning of the end for Labour and its leader Neil Kinnock. From that point on, it seems to have been downhill. The Conservatives spent nearly all their advertising money in the final three days (at an annualized weight greater than the spending of Proctor & Gamble or Unilever on soap powder), levelling all their guns at the Liberal Democrats' voters 'letting Labour in'. The testimony of the Liberal Democrats' campaign manager was that this did great damage to their support in the final hours of the campaign. The Tory tabloids did all that they could on their front pages, as well as the leader columns, to ensure that the

WORCESTER

[Part 1,

Conservatives were returned to power. Right to the end of the election, the 'floating voters' were higher than ever before. 18. THE CLUES WERE THERE! In retrospect, it might have been possible to guess that a late swing to the Tories was on the cards. The polls themselves unearthed several clues. One clue was the image of the leaders. There is a tradition among political scientists in Great Britain to explain patiently to visiting American academics, psephologists, pollsters, politicians and political journalists that the main difference between British and American elections is that in British elections policies, not people, are the deciding factor. Indeed, MORI research over the past decade has shown a fairly steady 50%-30%-20% relationship between voting decision-making mainly conditioned by policies-leader image-party image. Yet during the 1992 election one striking finding stood out: throughout the election, from the beginning to the end, John Major led Neil Kinnock by from 9% to 13% as the 'most capable' Prime Ministerial candidate. With hindsight, I believe that this question and its replies consistently gave a guide to the outcome; w6 were just not sufficiently smart to see it clearly, although The Times's Robin Oakley focused an entire article on this early in the campaign (Oakley, 1992). Another clue was that when asked 'Which two or three issues will be most important to you in helping you to make up your mind on how to vote?',

the National Health Service (NHS), education, unemployment -all Labour issues -led the field. Yet the one time that a different question was asked, 'How much will you be influenced, if at all, in the way you vote at the General Election by the policies of the political parties on taxation?',

no fewer than 39% of the electorate said that their vote would be influenced 'a great deal': no lies, no prevarication from this four in 10 of the electorate. They told us plainly: 'a great deal', they said. Did we hear sufficiently clearly? Another key finding was in the post-election wave of the MORI panel survey for the British Broadcasting Corporation's (BBC's) 'On the record', a telephone recall on people interviewed throughout the campaign, by the MORI telephone survey subsidiary, On-Line Telephone Surveys. The panel consisted of floating voters, and comprised those who after expressing a voting intention said that they might change their minds between the time interviewed and polling day, plus the full category usually described as the 'don't knows', who are the 10% who said that they would not vote, the 4% who were 'undecided' and the fewer than 1% who refused to say how they intended to vote. In all, the 'floaters' represented about 38% of the baseline survey from which the panellists were recruited. On-Line reinterviewed 1090 of the panel on the Friday and Saturday following the election. When voters were asked 'Thinking about the way you voted, which was stronger, your liking the party you voted for or your dislike of the other parties?',

a majority, 55%, said that it was antipathy and only 37% liking the party that they

19961

POLITICAL POLLING

voted for, among the floating voters in the panel. Panellists also voted by more than 3:1, 56%: 18%, that the Conservative Party rather than Labour could best handle the economy generally. And by 4: 1, 48%: 12%, they regarded John Major, rather than Neil Kinnock, as the most capable Prime Minister. Perhaps we should have taken these factors into account before the election, in which case we would have been less surprised by the result. Of course, the economy is also vital. During the 'autumn election boom' in 1991, Independent Television News had rung us to find out what the economic optimism index (EOI) was for September. 'Why?', a colleague asked them. 'Because Central Office [Conservative Party headquarters] says they'll call the election if it goes over 15, as they did in '83 and '87', was the reply. It was checked, and they had! The 1992 election was finally called with the index negative. But it was little noticed, except possibly by Central Office and in an article by Ivor Crewe in The Times (Crewe, 1992b), that in our survey taken on March 16th a sharp reversal had taken place in the national mood, and the -2% EOI recorded in February had been replaced by +15%, with 36% expressing optimism and only 21% gloom. Another clue came late in the campaign, but replicated findings of a year and more earlier. When asked whether they thought a hung Parliament would be good or bad for Britain, a majority, 56%, said 'bad', and they included an astonishing 44% of Liberal Democrats. It was certainly this group who were worried by the Tories' attack on the Liberal Democrats and the prospect of a hung Parliament in which Mr Ashdown held the balance of power. Certainly captains of industry were in no doubt: MORI fieldwork for The Financial Times at the beginning of the campaign and certainly at the end proved that. In the first wave of the 'Captains' panel, six in 10 main board directors of the nation's largest companies said that a hung Parliament would be bad for their business; by April 6th-7th, when just such an outcome seemed the most likely, three-quarters said that it would be bad. A majority, 52%, of electors worried about their belief that the Tories would privatize the NHS, including 26% of their own supporters. But, at the same time, seven in 10 of the public, including nearly half, 47%, of Labour supporters, believed that most people would pay more taxes under a Labour government. Another clue was the degree of volatility in the electorate. One thing that the snapshot polls do not show is the movement of the electorate between parties, which largely cancelled itself out, but which reveals the remarkable changeability of the electorate's political allegiance. The panel surveys provided this information. The most instructive analysis should come from the MORI panel study conducted for The Sunday Times. The panel was interviewed face to face four times during the campaign, and then reinterviewed after the election by telephone by FDS, an independent market research firm who were subcontracted to undertake the recalls (as the MORI telephone survey subsidiary, On-Line Telephone Surveys, was fully stretched with recalls on the panel for the BBC). FDS, on MORI's behalf, contacted 934 panellists on Friday, April loth, between 10 a.m. and 9 p.m. This represented a 60% recall of the original panel, which is quite good for a one-day recall, especially when you bear in mind that about 10% of the original 1544 panel were unavailable because they were not on the telephone. The data were of course weighted to both the demographic and the political profile of the original panel. The MORI-Sunday Times panel recall found that only 63% said that they had made up their minds before the election had been called, down nearly 20% from the

[Part 1,

WORCESTER TABLE 8

Changes of mind during election campaigns 1979-92t

Year

% of electorate switching between main parties during campaign

% of electorate switching to or from others or 'don't knows'

Total % of electorate changing answers during campaign

1979 1983 1987 1992

5.6 7.8 8.4 9.4

6.9 7.1 10.1 11.6

12.5

14.9

18.5

21.O

?Source: MORI-Sunday Times panels.

more usual 80% that was measured in previous elections. And, as noted below, 8% said that they had only made their mind up in the last 24 hours, and 21 % during the last week of the campaign. As The Sunday Times reported week after week, the amount of movement in the electorate-people who switched from one party to another ('switchers') or in and out of 'don't know' ('churners') -was higher t h a ~ e v e before, r and as reported in the final article about 11.1 million electors changed their minds during the campaign out of the 42.58 million in the electorate (Table 8). The week before polling day, the panellists indicated their voting intention, at that time as a Labour lead of one point; when reinterviewed after the election, their actual votes indicated a 24% swing. (The other national panel, conducted by NOP for the Independent on Sunday, found an even bigger swing-4% over the same period.) Perhaps more relevant, however, is the proportion of the panel that had already switched allegiance during the campaign -21 % even before the final week, far more than ever before. These figures suggest that 1992 saw an electorate who made up their minds later, and shifted their ground in greater numbers, than measured before. After an extensive examination the MRS inquiry concluded: 'After the final interviews there was a . . . swing to the Tories. It seems likely this was the cause of a significant part of the final error. . . . We estimate that late swing . . . probably accounted for between a fifth and a third of the total error'

(Market Research Society Working Party, 1994). On the evidence of the panel, the proportion may have been even higher, perhaps as much as half. What does this imply for future elections? It will be dangerous to try to doubleguess the electorate and to correct the figures to reflect changes of mind before they happen. However, it is probably possible to construct models of behaviour to allow for those 'don't knows' who vote, and certainly all possible methods need to be used to correct for differential turn-out by using certainty of voting questions. (MORI did use such a question in 1992, with those not 'certain to vote' excluded from the final poll figures, which reduced but probably did not entirely eliminate error from differential turn-out.) 19. 'SPIRAL OF SILENCE'

One of the more simplistic letter-writers explained that the unreliability of the public opinion surveys (and of the media comment on them) is down to ignoring the

19961

POLITICAL POLLING

'don't knows'. He must have missed the front page of The Times on the day before the election when Oakley's article, the splash that day, was headlined 'Parties make final push to capture floating voters': 'However, 24 per cent of Liberal Democrat supporters said they might yet change their vote. So did a fifth of Conservative supporters and 16% of Labour backers', Oakley reported. We certainly did not ignore the 'don't knows'. Nevertheless, those who told us that they did not know may have contributed to the problem. Investigation has made it plain that there was differential refusal: Conservative supporters were less likely to reveal their loyalties than Labour supporters were. This certainly operated through the reluctance of some of those interviewed to reveal their voting intentions, both by outright refusal and by answering 'don't know'. A similar and probably numerically more significant effect probably operated through a refusal by some to be interviewed at all, although there is no solid evidence to support this -consequently the samples interviewed were tilted to Labour and Conservative support was underestimated. This probably arose through the operation of what has been described as 'the Spiral of Silence' (Noelle-Neumann, 1984): Conservatives, feeling that their party was unfashionable and that they were outnumbered were more reluctant to disclose their loyalties. This effect seems to have persisted in the period since the election. Techniques need to be developed to try to maximize the number of people who are willing to state their voting intention. Use of information on past voting and other attitudinal questions to correct for any differential refusal that remains is one technique to be explored further. The use of secret ballots can also be considered, although MOR17sexperiments in this field have not yet convinced us of the value of introducing secret ballots as a routine methodology. We must also explore ways in which refusal to participate in surveys could be reduced; we need to enhance public awareness of the importance of survey research. 20. SAMPLE DESIGN AND QUOTAS

In theory, differential refusal to participate by Conservatives ought not to have distorted the polls, because if the quota and weighting systems had been operating perfectly this would have compensated, with other similar voters who were prepared to disclose their opinions being interviewed instead. This brings us to the third cause that appears to have contributed to the error: inadequacies in the sampling. This arose partly because the quotas set for interviewers and the weights applied after interviewing did not reflect sufficiently accurately the social profile of the electorate at the time of the election; this should be easy to correct, with care, in future elections, using accurate, up-to-date sources for setting quotas and weighting. It also arose partly because the variables used in setting quotas and weights were not correlated sufficiently closely with voting behaviour to ensure that the polls reflected the distribution of political support among the electorate. We need to try to identify other variables more closely related to voting behaviour, to ensure that samples are as representative as possible. This is more challenging. One solution being advocated is to weight surveys by declared past vote, as is routine in some other countries. However, we would be very wary of adopting a technique that in all previous elections -probably even including 1992-would

WORCESTER

[Part 1,

have made predictions worse rather than better. British voters have a strong tradition of failing to recall that in deciding to vote against their usual party they voted for the Liberal Democrats or their predecessors. Consequently polls usually underestimate the number of ex-Liberals in their samples and the distortion that this would cause on weighting by past vote would far outweigh any corrective effects between the two main parties. Although for a few months following the 1992 election the recalled Liberal vote, unusually, held up fairly well at a realistic level, it soon thereafter lapsed into its usual underestimate of the actual third-party share. A less contentious approach is to expand the number and variety of demographic variables used in quotas and weighting, particularly by the use of economic variables such as car ownership. So long as reliable base-line data are available, such measures can be implemented without much difficulty and may improve the political representativeness of samples. 21. WHAT WE LEARNED

The effectiveness of new polling techniques in the 1994 European elections offers encouraging evidence that the problems encountered in 1992 can be overcome. The MRS report expresses hope that the media will encourage this work -as, indeed, some media clients already do. They must accept that, inconvenient though it may be, research cannot always be responsibly reported in a single sentence or a succinct headline; special care is needed in secondary reporting of research, which is at present too frequently slapdash and can frustrate our efforts to ensure that the real meaning of our work is understood. Our task is difficult enough without our being judged on the basis of inaccurate or misleading and simplistic reports, or our being confounded with the 'voodoo' polls and straw polls which are still far too often reported as if they were serious surveys, by BBC and other journalists who ought to know better. With the media's help and co-operation, we believe that the lessons of 1992 can be learned and that polling in Britain can be stronger as a result. 22. BANNING OPINION POLLS

We occasionally face the question from radio and television interviewers 'Isn't it time polls were banned, as in France?'. My answer is usually that as 'economic' man I would favour it, because the parties would continue to conduct their private polls at an even higher level, and the City would fall all over themselves to commission, use, leak and even to manufacture polls for their own benefit. As in France now, the pollsters, the psephologists, the pundits, the political writers, the politicians and their friends would all know the results of the latest private polls; the only ones who would be in the dark are those for whose benefit elections are supposed to exist, the voters. To ban polls would be an illiberal act, removing from the electorate the one objective and systematic form of information about the view of the electorate they are provided during an election. Yet calls still come, from letter-writers (a few), pundits (one or two), one or two politicians and some members of the public. We need only to look to the USA where, on the Saturday night before the Tuesday primary elections, the late Senator William Fulbright announced that his private polls showed him ahead by 2:l.A few days later Dale Bumpers, his opponent, won

19961

POLITICAL POLLING

the election by a margin of 55%:45%. Later Fulbright's aides admitted that no private polls had been taken. And that was where polls are not and, thanks to the protection of the First Amendment, will never be banned. Across the Channel to France, where they are banned in the final days before polling (see below), in the Giscard election of 1981, as soon as the seven-day banning of the publication of the polls began the Chirac forces began an orchestrated effort to suggest that their candidate's campaign was taking off, and that he would be the one to go through with Giscard into the final round, thus ensuring two conservatives in the run-off. This frightened the Communist candidate's (Marchais) supporters into switching over to Mitterand, the Socialist candidate, to ensure that the left had a challenger in the final contest. The politicians, pundits and psephologists in Paris all knew what the private polls are finding; the only people who were kept from the truth of the French public's views were the French public themselves. More recently we have seen the ludicrous spectacle of polls being commissioned daily by banks and stockbrokers during the period of the ban in the week before the 1992 referendum on the Maastricht treaty, and by German and British media, published in, for example, the Daily Mail on the Friday before polling and in The Sunday Times on polling day itself, although not in the French edition, of course. The Head of Policy of the BBC said to me that she saw no reason why the BBC should not report on the findings of the poll, despite being heard by some French listeners. In France for seven days preceding an election, in Spain for five, in Portugal during elections, in Belgium for 30 days and in Luxembourg there are national bans on the publication of public opinion polls. In each country polls continue to be taken and influence the pronouncements of politicians, the pundits and leader writers and political commentators, the financial market makers and money manipulators; the only people who are kept in the dark are the public-denied access by politicians and legislators from the only objective measure of what the public thinks. (For a detailed analysis of what the regulations are for the publication of opinion poll results across the globe, see Rohme (1992).) In 1985 a Council of Europe Parliamentary Assembly Committee held hearings in several European cities to investigate proposals for the 'harmonization' of laws to regulate the publication of polls. The Chairman of the French Commission on Opinions Polls, Pierre Huet, in evidence to the Committee's hearings in Strasbourg said that, if it were not already enacted, it would not and should not be, saying that 'restrictions of freedom of information were objectionable on principle' (Huet, 1984). After extensive investigation, a review of the practices of its member governments, considerable testimony taken from academics, journalists and editors, legislators and government officials the Parliamentary Assembly concluded '. . . therefore, having heard all the evidence, the Committee are not of the opinion that strong controls are shown to be desirable or necessary and there is no need or value in attempting international harmonisation'

(Council of Europe, 1985). The Belgian ban, enacted early in the 1980s, was tested in 1985 by the news magazine Knack whose editor Frans Verleyen commissioned a poll and, four days before the general election, published it in deliberate violation of the law; the law was also tested by De Morgen and De Standard. Knack quoted article 18 of the 1831

WORCESTER

[Part 1,

Belgian constitution in its defence, which states: 'The printed press is free; censorship can never be introduced . . .'. In a commissioned article by a distinguished Belgian jurist Knack went further, arguing that the law banning polls was not only in violation of the constitution but also of article 10 of the European Convention on Human Rights (Leonard, 1985). 23. CONCLUSION

Let me conclude with two brief quotes, one from Tony Benn, 'If you deny people knowledge of what the community is thinking, then those who think those thoughts believe they are out of line',

and, finally, John Major quoting in turn Randolph Churchill, 'Trust the People'. REFERENCES Clifford, P. and Heath, A. (1994) The election campaign. In Labour's Last Chance? (eds A. Heath, R. Jowell, J. Curtice and B. Taylor). Aldershot: Dartmouth Press. Council of Europe (1985) Oficial Report, 37th Ordinary Session, Parliamentary Assembly, Council of Europe, Sept. 30th, 1985. Crewe, I. (1992a) A nation of liars? The opinion polls and the 1992 election. Purl. A f . , 45, no. 4, 475495. (1992b) One poll victory does not make Kinnock's summer. The Times, Mar. 18th. Hoffman, P. and Ross, J. (1970) (eds) Burke's Politics: Selected Writings and Speeches of Edmund Burke on Reform, Revolution and War, p. 106. New York: Knopf. Huet, P. (1984) Testimony to the Council of Europe Parliamentary Assembly hearings, Strasbourg, Oct. 8th. Leonard, D. (1985) Belgian leaders should read 'Areopagitica'. Wall Street Journal, Oct. 18th. Lippmann, W. (1922) Public Opinion. New York: Macmillan. Market Research Society Working Party (1994) The Opinion Polls and the 1992 General Election. London: Market Research Society. Noelle-Neumann, E. (1984) The Spiral of Silence. Chicago: University of Chicago Press. Oakley, R. (1992) Leadership gap still troubles Labour despite lead in polls. The Times, Mar. 18th. Payne, S. (1980) The Art of Asking Questions. Princeton: Princeton University Press. Price, V. (1992) Public Opinion. Newbury Park: Sage. Rohrne, N. (1992) The state of the art of public opinion polling worldwide: an international study based on information collected from national market and opinion research institutes in April 1992. Marktng Res. Today, 264-273. Worcester, R. (1981) Whose government is it, anyway?: polls, surveys and public opinion. Royal Society for Public Administration Con$ Public Znfuence and Public Policy, Brighton, Apr. (1991) British Public Opinion: Guide to the History and Methodology of Opinion Polling. Oxford: Basil Blackwell.