Combination of Survey Methods
STANLEY L. PAYNE*
> Survey researchers in the
past have tended to look upon personal interviews, telephone interviews, and mail questionnaires as alternative methods. Reported here is a home-use product test which utilizes all three methods in combination, each obtaining appropriate information from the same households.
Not many years ago, the mail questionnaire was the amateur's way of collecting data while the telephone interview was given hardly a thought by either expert or novice. Notable long-time exceptions were the National Consumer Panel of Market Research Corporation of America which depended upon mailed diaries, and those radio and television rating services which utilized coincidental telephone calls. The approved, professional method of conducting most surveys was the personal interview. Answers might be recorded by pencil or on tape, interviews might be made with individuals or with groups, questioning might be direct or indirect, but interviewing in person was customary. Then, surveys by mail or by telephone began to be accorded more general respectability. Rapid increases in applications of both of these methods are indicated by the many articles [I, 4, 51 devoted to their pros and cons which have appeared in recent years. Many limitations earlier placed on mail and telephone approaches have been relaxed. It begins to appear that almost everything that can reasonably be ascertained in person can also be determined by these other methods. Among the few remaining restrictions are the facts that not all households have telephones, that visual aids cannot yet be transmitted by telephone, and that awareness and knowledge can be overstated in answers to mail inquiries. The three approaches-personal, telephone, and mail -have now come to be considered legitimate alternatives. Survey practitioners today give careful attention to selecting the most appropriate one of the three for each particular investigation. Their judgments are fortified by experiments, such as that reported by Dr. Hochstim 131, in which he compared the feasibility of mail, telephone, and personal strategies for obtaining information on health. Still, it is only recently that our eyes have been opened to the fruitful idea of using the basic survey
* Stanley Payne is president of Interview Research Institute, division of Market Research Corporation of America. Collaborating with him in the last stage of the evolution of the method reported here were Raymond J. Lucivansky of Pet Milk Company and Richard J. Tucker of Gardner Advertising Company.
methods in combination. We may have been too blinded from looking upon them as exclusive alternatives to observe that they might be applied as complementary parts of a single investigation. Not that we have been overlooking occasional needs for reaching different respondents through diierent approaches-such as, mail for one universe and telephone for another--or for reaching non-responders to original mailings through telephone or personal interviews [2]. Nor have we been neglecting the need for pretesting and the other phases of questionnaire development which often employ methods different from that of the h a 1 survey. The aim of this short article is to illustrate that sometimes a combination of all three methods may be used with the same respondents to produce results more efficiently than one method alone could do. The following product testing example is based on actual surveys employing the three basic methods. Like many other research developments, this combining of methods was not an overnight discovery but instead evolved from accumulated experiences. Traditionally, most tests of products or packages in the home have taken one or the other of two formseither the products were mailed to testing panels with mail questionnaires or the products were placed in person with independent samples followed by interviews also made in person. The first break with tradition came with a realization that personal delivery of test items did not necessarily imply subsequent personal interviews. Indeed, it was soon observed that placements were easier to make when interviewers could promise that they would not be back but would simply telephone to obtain the homemakers' reactions. Looking back now, telephone follow-up seems most obvious, but hide-binding tradition had long operated to obscure its advantages. One situation that still seemed to require personal follow-up of personal placements was the occasional need for reports from different individual members of the testing family. That is, certain placement studies required not only placing products but also placing selfadministered questionnaire forms to be filled in by each member of the family immediately after his first or second use of the product. The follow-up interview was
JOURNAL OF MARKETING RESEARCH, MAY 1964
still made in person rather than by telephone, mainly because the individual questionnaires had then to be picked up. In the context of this article the reader may already see the next step, although again it took a little time and a few such home-use tests before its virtues became apparent. One of the present combinations of methods for home-use tests consists of (1) personal placement of products, self-administered questionnaires, and return envelopes; (2) mail returns of the individual forms; and ( 3 ) telephone follow-up interviews-all three methods used for each participating household. In a recent product test of this type, mailed returns were received from 82 percent and telephone follow-up interviews were completed with 97 percent of the testing families. The former approach, entirely in person, would have produced a somewhat higher pick-up of individual forms but a greater loss of completed interviews, and at substantially greater costs. To elaborate, each of the three methods is used to good advantage in the combination. The placement interview, besides screening out unqualified households and placing products, forms, and return envelopes, obtains family background information as well as answers to questions about current and past usage of the type of product-buying frequency, brand, size, uses, reasons, members using, storage, etc. The questionnaires left for individuals are made short and easy to answer and ask only for reactions to basic product characteristics. The telephone follow-up interview goes into considerable product detail with the homemaker but still is not overly long since so much of the information has already been ascertained. This combination of methods is not only efficient but is also well received by respondents. Under other circumstances of home-use testing the three basic methods might even be used in reverse
order. If the incidence of qualified users were very low and if visual aids were needed in the follow-up interview, the original screening calls might more economically be made by telephone, the product might be sent by mail, and the follow-up interviews would have to be made in person-unless the visual aids could also be mailed, making telephone interviewing feasible again. Two-method combinations have long been in use for other types of surveys, as when telephone screening and/or appointments made by telephone have preceded personal interviews or questio~airemailings. The U. S. Census procedure in 1960 used a combination of mail delivery and personal pick-up of the basic schedule, and personal delivery and mail return for the 25 percent sample. A closer approach to the present method is personal placement of test products, interim letters to encourage continued participation, and telephone interviews at the end of the testing period. So far as the writer is aware, however, the home-use tests described above are the first application of all three methods, each one obtaining information from the same respondents. Flexibility in survey design may have advanced a step with this combination of methods.
REFERENCES 1. L. E. Benson, "Mail Surveys Can Be Valuable," Public Opinion Quarterly, 10 (Summer 1946), 234-241. 2. M. H. Hansen and W. N . Hurwitz, 'The Problem of Nonresponse in Sample Surveys," Journal o f the American Statistical Association, 41 (1946), 517-529. 3. J. R. Hochstim, "Alternatives to Personal Interviewing," Public Opinion Quarterly, 27 (Winter 1963), 629-630. 4. S. L. Payne, "Some Advantages of Telephone Surveys," Journal o f Marketing, 20 (January 1956), 278-280. 5. H. G . Wales and Robert Ferber, A Basic Bibliography on Marketing Research, Chicago: American Marketing Association, 1963, 18-24.