On the Lack of Real Consequences in Consumer Choice Research: And Its Consequences Sina A. Klein and Benjamin E. Hilbig

On the Lack of Real Consequences in Consumer Choice Research

And Its Consequences

Sina A. Klein and Benjamin E. Hilbig

Cognitive Psychology Lab, Department of Psychology, University of Koblenz-Landau, Germany

Abstract: Experimental tasks measure actual behavior when the consequences that follow actions and choices mirror those of real-life behavior. Consequently, choice tasks in consumer research would need to include both costs (losing a previously earned endowment) and gains (actually receiving what was chosen) to structurally resemble real-life consumer choices. A literature review of studies (k = 446) in consumer research confirms that full implementation of consequences is rare. The extent to which presence versus absence of these consequences systematically affects observable behavior is tested in an experiment (N = 669) comparing a fully consequential (cost and gain consequences), a partially consequential (gain consequence only), and a hypothetical (no consequences) consumer choice task. Results show that consequences, once real, affect both the general willingness to purchase and the relative preferences for different products. Hence, it would seem advisable to more carefully consider the role of consequences in future consumer research.

Keywords: consumer choice, research practices, literature review, food choice

Many subfields of psychology ultimately aim to explain and predict behavior. That is, they intend to draw conclusions about what people might actually do in “real life” (and why they would do so) from different kinds of observations such as participants’ responses on a self-report questionnaire or responses in some laboratory task. As has been repeatedly argued (Baumeister, Vohs, & Funder, 2007; Funder, 2009a, 2009b; Furr, 2009), many of the observations psychologists predominantly rely on are more or less strongly removed from the to-be-explained behavior. For example, several groups of authors (Baumeister et al., 2007; Furr, 2009; Meredith, Dicks, Noel, & Wagstaff, 2017; Patterson, 2008; Patterson, Giles, & Teske, 2011) argue –and demonstrate in literature reviews –that vast portions of recent psychological research rely on observations that cannot be considered “actual behavior.” Thus, Baumeister et al. (2007) provocatively state that much of psychology has become “the science of self-reports and finger movements” (p. 396). Importantly, the core argument is not that questionnaire responses or button presses are, per se, poor examples of behavior. Anyone who ever filled out an immigration or tax form (a questionnaire) or clicked a Website’s “buy”

button for a hugely expensive product will indubitably agree that these actions –while essentially being self-reports and finger movements –entail a lot of behavior. What then sets apart these examples from the omnipresent self-report personality questionnaires, hypothetical scenarios, or reaction time tasks that Baumeister et al. (2007) and others have convincingly argued do not represent observations of actual behavior? We argue that the core distinguishing aspect is whether the consequences a research participant faces (conditional on her and potentially others’ actions) match or at least approximate the consequences faced by agents in the corresponding real-life situations in a structurally comparable way. If the tasks given to participants “carry some form of consequence (e.g., social, financial, effort, time, self-efficacy)”, these will typically be “substantially more informative of real [...] behavior” (Morales, Amir, & Lee, 2017). Correspondingly, Lewandowski and Strohmetz (2009) have argued that consequences for the self or others are one defining element of behavioral choice: “Rather than ask participants to self-report what they believe they would choose, behavioral choice focuses on wh at participants actually select as the dependent variable” (p. 998). Similarly, Diederich (2003a, 2003b) argues that real consequences ought to be implemented to induce choice conflict in multi-attribute decision tasks. Indeed, this very principle –that well-specified consequences help transform researchers’ observations from some artificial task into truly behavioral observations –dates

back to the very origins of modern empirical psychology (one might say that Behaviorism was marked by an almost obsessive focus on consequences) and is now one of the cornerstone practices of experimental economics (Camerer & Hogarth, 1999; Hertwig & Ortmann, 2001). In early discussions about economics becoming an experimental science, Smith (1982) specified certain conditions or “precepts” that make an economic experiment valid although it does not fully represent the natural setting. In line with this, Plott (1991) argued that economic experiments do not have to mirror real-life settings exactly but the features essential to test a theory. As a consequence, experimental economists predominantly study fully specified games with well-defined structures of consequences (most commonly monetary gains or losses) that mirror the essential features of corresponding real-life situations. For example, the essence of real-life behaviors such as giving to charity is mirrored in the structure of consequences built into the Dictator Game (e.g., Forsythe, Horowitz, Savin, & Sefton, 1994) in which an individual in the role of the dictator is free to allocate an actual valued resource between herself and a passive recipient. In this simple economic game, the consequences mirror those of the real-life behavior targeted: If a dictator does not give anything, she will face the consequence of having/keeping the entire endowment to herself; to the extent that she allocates part of the resource to another, she bears the consequence of having/keeping less of the valued endowment. Thus, whereas the specific actions performed by the participant (e.g., typing a number into a box on the computer screen) do not necessarily match those involved in the real-life situation (e.g., placing money in a collecting tin of some charity on the street), the consequences attached to her actions do. Hence, participants in economic games “make real decisions with potent consequences” (Murnighan & Wang, 2016, p. 81) which is exactly why these games have long been considered to yield relatively objective observations of actual behavior (e.g., Pruitt & Kimmel, 1977). Actual decision behavior occurs daily, and one prominent and frequent example is consumer choice. In light of the above arguments, one may well ask whether and to what extent the field of consumer choice research actually studies consumer behavior or, in Lewandowski and Strohmetz (2009) terminology, behavioral choice. In other words, do the paradigms most heavily relied on in consumer choice research involve the types of consequences that define real-life consumer behavior? Arguably, real-life consumer choice behavior most commonly involves choosing from a selection of goods/products/services and spending a valuable resource (usually money) to then receive what was chosen. More specifically, actual consumer choices typically involve some endowment (money) which was previously earned in some form and which must be invested so as to purchase a product. Thus, as in the above example of clicking a Website’s “buy” button, real-life consumer choices bear two relevant sets of consequences: First, these choices are almost always costly in that deciding to buy a product is accompanied by a reduction or loss of one’s endowment. Second, they involve positive consequences or gains in that one actually receives the good/product/service and may consume or in some other way profit from it.

Correspondingly, one can broadly classify the consumer choice scenarios studied in consumer choice research into three different categories –depending on which of the consequences as defined above are actually present. First, a fully consequential consumer choice situation is characterized by both consequences, namely costs (e.g., losing something that was previously earned) and gains, meaning actually receiving the good/product/service. Second, a partially consequential consumer choice situation involves only one of the consequences, most typically 1

actually receiving the product but without bearing actual costs; in other words, whereas the positive consequence of consuming the chosen product is real, the negative consequence of losing (part of) an endowment that was obtained through effort is absent. Strictly speaking, choice situations in which a participant is simply given an endowment as a windfall without investing time or effort are also classified as only partially consequential –given that whatever is potentially spent was not previously earned. Third, hypothetical consumer choice situations involve neither of the consequences, that is, there are no costs associated with the choices made, but neither does one receive the chosen product.

Literature Review

To gain more firm insight on common research practices, we conducted a literature review including all studies on consumer choice published between February 2012 and April 2013 in the Journal of Consumer Research. Raters classified all studies within each published paper into four categories: (1) fully consequential choice, for example, buying a product with money that had to be earned in a previous task, (2) partially consequential choice, for example, selecting one out of several products and actually receiving/consuming it but without having to pay for it, (3) hypothetical consumer choice (including willingness to purchase), and (4) other non-choice tasks such as product evaluations, for example, on some scales with product features. Out of all 446 studies that were coded, 281 (63.0%)

used measures falling at least into one of these categories and were hence included in the analyses below. 2

The detailed coding sheet is available online (https://osf.io/ z5tn6/). Out of all studies, about two-thirds used choice tasks (63.5%) and about one-third used product evaluations (36.5%). Within the studies that used choice tasks, more than two-thirds (69.1%) of studies used hypothetical choices and only a fourth (24.9%) used partially consequential choices. No more than 6.0% of studies used fully consequential choices involving both cost and gain consequences. In conclusion, real consequences are not common in the study of consumer choice and fully consequential choice studies remain a rare exception.

Whereas the above evidence clearly indicates that (full) consequences are rarely implemented in consumer choice research, previous research suggests that the absence of both cost and gain consequences might indeed pose a problem. First, with regard to the cost consequence, Moser, Raffaelli, and Notaro (2013) point out that it is important to let participants use their own money instead of an endowment given by the experimenter as money received as a windfall might increase participants’ willingness to purchase. This idea is based on findings showing that, under some circumstances, people are more attached to their own, previously earned money than to money that was gained before the decision (e.g., Carlsson, He, & Martinsson, 2013; Cherry, Frykblom, & Shogren, 2002; Smith, 2010; Thaler & Johnson, 1990) which would also be predicted by effort justification, a mechanism to reduce cognitive dissonance (Festinger, 1957). However, while one group of participants in Moser et al.’s (2013) study actually had to use their own money, the only comparison made was to a hypothetical choice task. Therefore, evidence on consumer choices differing in whether money had to be previously earned or not (and thus, whether the choice is actually costly or not) is still missing (Moser et al., 2013). Second, with regard to the gain consequence, several meta-analyses (e.g., Foster & Burrows, 2017; Harrison & Rutström, 2008; Murphy, Allen, Stevens, & Weatherhead, 2005) comparing hypothetical and actual choice behavior to investigate hypothetical bias showed that the use of hypothetical versus actual preference or choice variables does indeed yield different results. Specifically, hypothetical measures lead to an overstatement of preferences and values for products and goods (e.g., in terms of how much money one would pay for a product) in comparison with actual measures (Murphy et al., 2005). Hypothetical bias typically refers to a choice between a certain product or good and money. Taking this one step further, we investigate whether there is also some form of hypothetical bias in a choice between different products.

Taken together, previous research indicates that the presence versus absence of consequences in choice tasks may alter the observable behavior systematically. This implies that tasks that do not model the real-life behavior one is intending to draw conclusions on will often have to be treated with caution (Morales, Amir, & Lee, 2017). However, to the best of our knowledge, studies have not directly compared the effects of cost and gain consequences and their interplay on these systematic behavioral changes. Therefore, it remains unclear to what degree cost and gain consequences contribute to the systematic change of behavior and hence which specific aspects of choice tasks alter which specific aspect of behavior. Ultimately, it remains difficult to judge whether certain consumer choice tasks will provide reliable estimates of people’s willingness to purchase and their relative preferences for specific options. To close this gap, we conducted an experiment that directly compares a fully consequential, a partially consequential, and a hypothetical choice task in the same setting.

Experiment

Specifically, two comparisons between these different choice tasks can reveal whether and, if so, to which degree the two types of consequences alter choice behavior. First, the fully and partially consequential choice tasks only differ with respect to the presence of the cost consequence, namely the fact that one has to invest effort and earn the money before being able to spend it. Thus, a comparison between these two conditions will provide insight into whether the presence of costs alters the willingness to spend the money and purchase the product and hence choice behavior. Second, the two consequential tasks differ from the hypothetical task only with respect to the presence of the gain consequence, namely actually receiving the product after making a choice. Thus, a comparison between the hypothetical condition and each of the two consequential conditions will test the presence of a hypothetical bias and hence provide insight into whether the possibility to actually consume the product alters the probability of selecting certain options.

2

Studies/articles that could not be assigned to any of the four categories were, for example, theoretical articles or studies with dependent variables such as creativity, perceived time, or mood. In addition to the analysis reported in the main text, we used a more liberal criterion for the inclusion of such studies. For example, we excluded evaluations of similarity of products from the conservative categorization but included it as product evaluation in the more liberal categorization. Also, studies from two retracted papers were also included in the more liberal categorization. In total, 314 studies were included in this second analysis. Results did not differ more than 3% from the conservative analysis.

Methods

Design and Procedure

An experiment with a one-factorial between-subjects design was conducted. Participants were assigned to one of three conditions in which they were faced with a fully consequential, a partially consequential, or a hypothetical consumer choice. The real-life consumer choice modeled in this study resembled a grocery shopping situation which involved what may be termed a “considerate” (i.e., organic and fair trade) and a “non-considerate” product option (see Reese & Kohlmann, 2015, for a similar choice task). Participants decided whether to invest a monetary endowment to buy one of two chocolate options or whether to keep the money. The two chocolate options were one organic, fair trade chocolate bar of 100 g, or two nonorganic, non-fair trade chocolate bars of 100 g each. The specific chocolate bars were highly similar on all attributes except for organic and fair trade produce. The monetary equivalent of the chocolate options was roughly the same. Compared to keeping the money, the chocolate options involved a 10–25% discount so as to make choosing either chocolate option more attractive.

In the fully consequential condition, participants first had to earn the monetary endowment for the choice task. Specifically, they briefly worked on unrelated, mildly effortful cognitive tasks for 3 min, such as disentangling alphabetic strings, identifying words within a list of non-words, or solving simple rule of three calculations (see Figure A1 in the Appendix for an example). If they solved at least four out of nine tasks (83.1% of the participants assigned to this condition were successful), they earned the 3€ endowment. Then, they were asked to choose whether they wanted to keep the money or spend it to buy chocolate, that is, either of the two chocolate options. In the partially consequential condition, participants were given 3€ without having to work for it and –exactly parallel to participants in the fully consequential condition –then chose whether they wanted to keep the endowment or spend it to buy one of the two chocolate options. In both the fully and the partially consequential conditions, participants actually received their chosen option, whether it was their monetary endowment or whichever chocolate option they had chosen. In the hypothetical condition, participants indicated whether they would keep their 3€ endowment or spend it to buy one of the two chocolate options; they were asked to decide as if the choice was fully consequential, but fully aware in advance that neither consequence would actually be materialized.

The study was conducted in different places on campus, for example, next to the cafeteria or the main entrance. Passing individuals were asked whether they wanted to participate in the study. After providing informed consent, participants were seated in separate booths, ensuring that their choice was confidential. In the fully consequential condition, participants first worked on the separate cognitive tasks. If they passed, the remaining procedure was the same as for participants in the other conditions: The experimenter placed a questionnaire, the three chocolate bars, and 3€ in cash in front of the participant. The questionnaire consisted of demographics, the consumer choice task, and the following screening checks (the complete questionnaires and the cognitive tasks for the fully consequential condition are available at https://osf.io/z5tn6/): Participants indicated whether they had food intolerance to chocolate and whether they consumed chocolate in general. If they indicated food intolerance or indicated that they did not eat chocolate in general, they were excluded from the sample (see below). After completing the questionnaire, participants were given their chosen chocolate or money (only in the two consequential conditions), debriefed and dismissed.

Analytical Strategy, Power Analysis, and Sample

Since the task structure of the present experiment is conditional in nature (choosing either chocolate option is conditional on deciding to spend the endowment), the appropriate analysis takes the dependence of observations into account. A straightforward approach is to model observed choices with a two-stage multinomial processing tree (MPT) model 3

(Batchelder & Riefer, 1999; Erdfelder et al., 2009) with separate probabilities for (i) keeping versus spending the monetary endowment, and, conditionally on the latter, (ii) making a considerate or a non-considerate product choice. Thus, the MPT model as depicted in Figure 1 comprises two parameters: First, parameter k denotes the probability to keep the endowment and thus distinguishes between the willingness to keep the money (probability k) and spending the money to buy a product (probability 1
k). Second, and conditionally on the willingness to buy a product, parameter c describes the probability of a considerate product choice and distinguishes between

3

Alternatively, ordinary chi-square tests could be used; however, they do not directly account for the conditional structure, which is why the MPT model is to be preferred. Nonetheless, we additionally conducted all analyses using standard chi-square tests. An overall test, a test without the monetary option, and a test without the monetary option and both consequential conditions analyzed together all revealed significant differences between conditions (p < .05) and thus corroborate the analyses in the MPT framework. Results of these analyses are available at https://osf.io/z5tn6/.

Figure 1. Graphical representation of the baseline multinomial processing tree model (for one condition).

choosing one organic and fair trade chocolate bar (probability c) and choosing two normal chocolate bars (probability 1
c). Models of this type have been used in other contexts in which the choice structure is a conditional one (e.g., Klein, Hilbig, & Heck, 2017). The complete MPT model across all three conditions correspondingly consists of three trees as depicted in Figure 1, one for each condition. Each tree, and hence each condition, has distinct parameters (k h & c h , k pc & c pc , k fc & c fc , with subscripts denoting conditions, that is, h for hypothetical, pc for partially consequential, and fc for fully consequential). The full model equations are available at https://osf.io/z5tn6/. To estimate parameters and test whether there are differences in choice behavior across conditions (i.e., likelihood ratio tests), we used the software multiTree (Moshagen, 2010). To compare conditions, we first implemented the parameter restriction k h = k pc = k fc to test for overall differences in parameter k (and hence the choice between keeping the 3€ endowment vs. “buying” either chocolate) and the parameter restriction c h = c pc = c fc to test for overall differences in parameter c (and hence the choice between considerate and non-considerate chocolate consumption). Second, whenever an overall test indicated significant differences between conditions, we conducted pairwise comparisons between all specific parameters (k h = k pc , k h = k fc , k pc = k fc and c h = c pc , c h = c fc , c pc = c fc ). In order to set a lower-bound sample size, we computed an approximate 4

a priori power analysis in the MPT framework using multiTree (Moshagen, 2010). Specifically, we aimed to detect a difference in c-parameters of c h = . 60, c pc = .50, and c fc = .40 (conservatively assuming that the k-parameter is .40 in each condition) which corresponds to a small effect of Cohen’s ω = .13 with a power of 1 –β = .80 (with α = .05). The required overall sample size was N = 599 which we thus set as our lower bound. In total, 850 participants completed the study. We excluded all participants who indicated a food intolerance against chocolate (n = 117), indicated they did not consume chocolate in general (n = 31), misunderstood the instructions and clearly indicated this toward the experimenter (n = 29), and/or had missing values in either one of the screening variables (n = 16) or the choice task (n = 3). Additionally, we excluded participants who noted on the questionnaire that they were vegan (n = 3). The final sample consisted of N = 669 participants aged between 18 and 54 years (M = 22.35, SD = 3.97). Slightly more than half of participants were female (n = 387), and most were students (n = 641).

Results

Across conditions, choices were relatively evenly distributed across options with keeping one’s monetary endowment turning out to be the most frequently chosen option (38%), followed by organic, fair trade chocolate (33%) and normal chocolate (29%). The raw data and choice proportions per condition are reported in Table A1 in the Appendix. Figure 2 summarizes parameter estimates from the MPT model. 5

As can be seen, there were noteworthy differences between conditions: Once the choice was fully consequential, 55% of participants decided to keep their money, whereas only about 30% of participants made the same decision when the choice was either hypothetical or only partially consequential (i.e., the money was given as windfalls and not previously earned through effort). Correspondingly, a test for overall differences between k-parameters turned out significant (ΔG 2

(df = 2) = 38.85, p < .001, Cohen’s ω = .24). To follow up, we conducted

4

Within the MPT framework, power analysis requires the full specification of model parameters and thus assumptions about the parameter values in the population. Correspondingly, without strong a priori knowledge on the to-be-expected parameter values, such a power analysis necessary remains approximate. 5

Note that, by necessity, parameter estimates are directly implied by the choice proportions (and vice versa). For example, the observed proportion for choices for the considerate option is, by definition, (1
k)  c.

Figure 2. Parameter estimates for parameter k (keeping money vs. spending money) and parameter c (considerate vs. non-considerate product choice) across conditions. Error bars represent one standard error of the parameter estimate.

pairwise comparisons to investigate which specific differences drive the overall test result. The parameters k h and k pc did not differ significantly (ΔG 2

(df = 1) = 0.13, p < .718, Cohen’s ω = .01), indicating that the probability of keeping one’s money was comparable in the hypothetical and partially consequential conditions. By contrast, both k h and k fc (ΔG 2

(df= 1) = 30.87, p < .001, Cohen’s ω = .21) and k pc and k fc (ΔG 2

(df= 1) = 26.74, p < .001, Cohen’s ω = .20) differed significantly. Thus, the fully consequential condition differed significantly from the other two in that more people chose to keep their monetary endowment as compared to the hypothetical and partially consequential conditions.

As can also be seen in Figure 2, once choosing to buy chocolate (i.e., conditional on 1
k), 46% of participants made a considerate choice in the hypothetical condition, whereas roughly 60% of participants made a considerate choice in each of the consequential conditions. An overall test confirmed that parameter c differed across conditions (ΔG 2

(df= 2) = 6.55, p < .038, Cohen’s ω = .10). More specifically, parameters c pc = c fc did not differ significantly (ΔG 2 (df= 1) = 0.02, p < .897, Cohen’s ω = .01), indicating that considerate choices (for the organic, fair trade chocolate) were comparably likely in the two consequential conditions. By contrast, parameters c h and c pc (ΔG 2

(df= 1) = 5.48, p < .019, Cohen’s ω = .09) and c h and c fc differed significantly (ΔG 2

(df= 1) = 3.78, p < .052, Cohen’s ω = .08), although the latter comparison only borders the conventional level of significance. In summary, in the hypothetical condition,

participants were significantly less likely to select the organic, fair trade chocolate than those in either of the consequential conditions (which, in turn, did not differ).

Discussion

All too often, conclusions about behavior are drawn from research tasks and paradigms that do not (or only incompletely) match the to-be-modeled real-life situations in terms of a correspondence in the structure of possible consequences (Morales et al., 2017). The upshot that research often does not provide observations of “actual behavior” (Baumeister et al., 2007) appears to hold for consumer choice research as evidenced by our literature review: The vast majority of choice tasks used in recent consumer choice research were hypothetical or only partially consequential in nature, whereas fully consequential tasks –involving both costs and gains and thus mirroring real-life consumer choice situations –are exceptionally rare. At the same time, several meta-analyses imply that hypothetical and actual choice tasks lead to differences in choice behavior (Foster & Burrows, 2017; Harrison & Rutström, 2008; Murphy et al., 2005). However, until now, there was insufficient evidence on how and to which degree consequences in terms of both costs (losing at least part of an endowment that was previously earned) and gains (actually receiving the chosen option) drive such differences.

Hence, the experiment reported on herein compared a fully consequential (cost and gain consequences), a partially consequential (gain consequence only), and a hypothetical choice task (no consequences) in the same setting. Results revealed that costs –that is, having to work for the endowment –and indeed only costs reduced the willingness to spend the endowment. This is in line with previous research regarding earned endowments versus endowments received as a windfall (Carlsson et al., 2013; Cherry et al., 2002; Festinger, 1957; Smith, 2010, Thaler & Johnson, 1990) and provides direct evidence for corresponding assumptions about consumer choice behavior (Moser et al., 2013). Furthermore, and arguably more problematically, the possibility to actually consume the products –that is, the reality of gain consequences –increased the probability of selecting a considerate (fair trade, organic) product over a non-considerate one. In other words, participants’ preferences given the exact same options depended on whether the corresponding consequences were going to be materialized. Taken together and in the extreme, the absence of both (cost and gain) consequences leads to twice as many participants choosing a non-considerate product (39%) in comparison with when both consequences were present (19%). This supports the findings of several metaanalyses showing the existence of hypothetical bias in choice tasks (Foster & Burrows, 2017; Harrison & Rutström, 2008; Murphy et al., 2005) and extends these findings to choices between different products.

In summary, our experiment demonstrates that both costs and gains are important consequences that can alter choice behavior. Whereas cost consequences affect the general willingness to purchase and thus whether a product is bought or not, gain consequences actually affect specific preferences, that is, which options are chosen. In turn, these results provide some guidelines for the implementation of consequences in consumer choice tasks: First, as the presence of cost consequences affects whether a product is purchased or not, it also determines whether the choice between different products is actually relevant or not. If the willingness to purchase a product approaches zero, it is neither relevant which out of different products is chosen nor how this choice can be influenced (e.g., through some experimental manipulation). Therefore, before implementing only gain consequences in experiments, it is important to demonstrate that one might reasonably expect at least some willingness to purchase (at cost). Second, whenever the preferences for certain options –and how these may be influenced –are of interest, it would seem highly advisable to implement the corresponding gain consequences or, at a minimum, demonstrate that these preferences are independent of whether or not the consequences are hypothetical versus real. Of course, it must also be acknowledged that there are practical limitations to the consequences that can realistically be implemented in consumer choice research. In particular, it will simply be unrealistic in studies specifically focusing on expensive or otherwise impractical goods/ products/services (we can neither actually give a car or holiday trip to participants nor have them pay for it). However, we would argue that research is certainly not exclusively concerned with goods/products/services of this nature; rather, the focus is most commonly on underlying mechanisms and principles of consumer behavior. The latter can be studied using those types of goods/products/services that actually lend themselves to implementing fully consequential choice tasks. Thus, rather than calling for universal or even compulsory implementation of cost and gain consequences, we merely emphasize that the issue of consequences should not simply be ignored away.

To conclude, the prevalent use of hypothetical choices in consumer choice research can and will likely lead to systematically inaccurate predictions for both the willingness to buy a product and the relative proportion of choices among different products. Although using fully consequential choice tasks which include both cost and gain consequences might be more complicated and costly to the researcher than hypothetical tasks, the more accurate estimation of choice behavior should outweigh these additional expenses. In the long run, a shift toward more commonly implementing both cost and gain consequences will help foster exchange with researchers from other, related fields for whom “actual behavior” is the imperative criterion (most notably, behavioral economics) and help counteract reducin g psychology to “the science of self-reports and finger movements” (Baumeister et al., 2007).

References

Batchelder, W. H., & Riefer, D. M. (1999). Theoretical and empirical review of multinomial process tree modeling. Psychonomic Bulletin & Review, 6, 57–86. https://doi.org/ 10.3758/BF03210812 Baumeister, R. F., Vohs, K. D., & Funder, D. C. (2007). Psychology as the science of self-reports and finger movements: Whatever happened to actual behavior? Perspectives on Psychological Science, 2, 396–403. https://doi.org/10.1111/j.1745- 6916.2007.00051.x Camerer, C. F., & Hogarth, R. M. (1999). The effects of financial incentives in experiments: A review and capital-laborproduction framework. Journal of Risk and Uncertainty, 19, 7–42. https://doi.org/10.1023/A:1007850605129 Carlsson, F., He, H., & Martinsson, P. (2013). Easy come, easy go. The role of windfall money in lab and field experiments. Experimental Economics, 16(2), 190–207. https://doi.org/ 10.1007/s10683-012-9326-8 Cherry, T. L., Frykblom, P., & Shogren, J. F. (2002). Hardnose the dictator. The American Economic Review, 192, 1218–1221. https://doi.org/10.1257/00028280260344740

Diederich, A. (2003a). Decision making under conflict: Decision time as a measure of conflict strength. Psychonomic Bulletin & Review, 10, 167–176. https://doi.org/10.3758/BF03196481 Diederich, A. (2003b). MDFT account of decision making under time pressure. Psychonomic Bulletin & Review, 10, 157–166. https://doi.org/10.3758/BF03196480 Erdfelder, E., Auer, T.-S., Hilbig, B. E., Aßfalg, A., Moshagen, M., & Nadarevic, L. (2009). Multinomial processing tree models. A review of the literature. Zeitschrift für Psychologie/Journal of Psychology, 217, 108–124. https://doi.org/10.1027/0044-3409.217.3.149 Festinger, L. (1957). A theory of cognitive dissonance. Stanford, CA: Stanford University Press. Forsythe, R., Horowitz, J. L., Savin, N. E., & Sefton, M. (1994). Fairness in simple bargaining experiments. Games and Economic Behavior, 6, 347–369. https://doi.org/10.1006/game. 1994.1021 Foster, H., & Burrows, J. (2017). Hypothetical bias: A new metaanalysis. In D. McFadden & K. Train (Eds.), Contingent valuation of environmental goods (pp. 270–291). Cheltenham, UK: Edward Elgar Publishing. Funder, D. C. (2009a). Naive and obvious questions. Perspectives on Psychological Science, 4, 340–344. https://doi.org/10.1111/ j.1745-6924.2009.01135.x Funder, D. C. (2009b). Persons, behaviors and situations: An agenda for personality psychology in the postwar era. Journal of Research in Personality, 43, 120–126. https://doi.org/ 10.1016/j.jrp.2008.12.041 Furr, R. M. (2009). Personality psychology as a truly behavioral science. European Journal of Personality, 23, 369–401. https:// doi.org/10.1002/per.724 Harrison, G. W., & Rutström, E. E. (2008). Experimental Evidence on the existence of hypothetical bias in value elicitation methods. In C. R. Plott & V. L. Smith (Eds.), Handbook of experimental economics results (pp. 752–767). Amsterdam, The Netherlands: Elsevier. Hertwig, R., & Ortmann, A. (2001). Experimental practices in economics: A methodological challenge for psychologists? Behavioral and Brain Sciences, 24, 383–451. Klein, S. A., Hilbig, B. E., & Heck, D. W. (2017). Which is the greater good? A social dilemma paradigm disentangling environmentalism and cooperation. Journal of Environmental Psychology, 53, 40–49. https://doi.org/10.1016/j.jenvp.2017.06.001 Lewandowski, G. W Jr., & Strohmetz, D. B. (2009). Actions can speak as loud as words: Measuring behavior in psychological science. Social and Personality Psychology Compass, 3, 992–1002. https://doi.org/10.1111/j.1751-9004.2009.00229.x Meredith, S. J., Dicks, M., Noel, B., & Wagstaff, C. R. D. (2017). A review of behavioural measures and research methodology in sport and exercise psychology. International Review of Sport and Exercise Psychology, 11 , 25–46. https://doi.org/10.1080/ 1750984X.2017.1286513 Morales, A. C., Amir, O., & Lee, L. (2017). Keeping it real in experimental research –Understanding when, where, and how to enhance realism and measure consumer behavior. Journal of Consumer Research, 44, 465–476. https://doi.org/ 10.1093/jcr/ucx048 Moser, R., Raffaelli, R., & Notaro, S. (2013). Testing hypothetical bias with a real choice experiment using respondents’ own money. European Review of Agricultural Economics, 41 , 25–46. https://doi.org/10.1093/erae/jbt016 Moshagen, M. (2010). multiTree: A computer program for the analysis of multinomial processing tree models. Behavior Research Methods, 42, 42–54. https://doi.org/10.3758/BRM.42.1.42 Murnighan, J. K., & Wang, L. (2016). The social world as an experimental game. Organizational Behavior and Human Decision Processes, 136, 80–94. https://doi.org/10.1016/j.obhdp. 2016.02.003

2019 Hogrefe Publishing Murphy, J. J., Allen, P. G., Stevens, T. H., & Weatherhead, D. (2005). A meta-analysis of hypothetical bias in stated preference valuation. Environmental and Resource Economics, 30, 313–325. https://doi.org/10.1007/s10640-004-3332-z Patterson, M. L. (2008). Back to social behavior: Mining the mundane. Basic and Applied Social Psychology, 30, 93–101. https://doi.org/10.1080/01973530802208816 Patterson, M. L., Giles, H., & Teske, M. (2011). The decline of behavioral research? Examining language and communication journals. Journal of Language and Social Psychology, 30, 326–340. https://doi.org/10.1177/0261927X11407174 Plott, C. R. (1991). Will economics become an experimental science? Southern Economic Journal, 57, 901 –919. https:// doi.org/10.2307/1060322 Pruitt, D. G., & Kimmel, M. J. (1977). Twenty years of experimental gaming: Critique, synthesis, and suggestions for the future. Annual Review of Psychology, 28, 363–392. Reese, G., & Kohlmann, F. (2015). Feeling global, acting ethically: Global identification and fairtrade consumption. The Journal of Social Psychology, 155, 98–106. https://doi.org/10.1080/ 00224545.2014.992850 Smith, V. L. (1982). Microeconomic systems as an experimental science. The American Economic Review, 72, 923–955. Smith, V. L. (2010). Theory and experiment: What are the questions? Journal of Economic Behavior and Organization, 73, 3–15. https://doi.org/10.1016/j.jebo.2009.02.008 Thaler, R. H., & Johnson, E. J. (1990). Gambling with the house money and trying to break even: The effects of prior outcomes on risky choice. Management Science, 36, 643–660. https://doi. org/10.1287/mnsc.36.6.643

History Received January 30, 2018 Revision received April 5, 2018 Accepted May 22, 2018 Published online February 19, 2019

Acknowledgments The authors thank Anja Humbs, Theresa Behringer, and Janine Rispler for their help with coding the studies for the literature review.

Conflict of Interest None.

Open Data The detailed coding sheet is available online (https://osf.io/z5tn6/). Complete questionnaires and the cognitive tasks for the fully consequential condition are available at https://osf.io/z5tn6/. Full model equations are available at https://osf.io/z5tn6/.

Funding This work was supported by the German Research Foundation [grant number HI-1600/6-1].

ORCID Sina A. Klein https://orcid.org/0000-0002-8154-5429

Sina A. Klein Cognitive Psychology Lab Department of Psychology University of Koblenz-Landau Fortstraße 7 76826 Landau Germany klein@uni-landau.de

Appendix

Table A1. Frequency and proportion of choices across conditions

Condition n frequency (row-wise proportion) of choices money considerate chocolate option non-considerate chocolate option

Fully consequential 222 122 (.55) 58 (.26) 42 (.19) Partially consequential 221 68 (.31) 90 (.41) 63 (.29) Hypothetical 226 66 (.29) 73 (.32) 87 (.39)

Figure A1. Examples for tasks in the fully consequential condition.