Discrete Choice Modelling Workshop – Warsaw 2016

 

 

 

Agenda

 

Session 1: Spatially explicit DCE

 

  • Danny Campbell, Wiktor Budziński, Mikolaj Czajkowski, Nick Hanley – "Using spatial latent class models to identify willingness to pay hot and cold spots"

    Notwithstanding the ability to include individual characteristics in the latent class membership function, there is a possibility that the unobserved factors that explain membership to latent classes may be spatially related. If so, the errors are spatially arranged, meaning that the assumption that the error terms are independent of one another is violated. Not addressing this means the model is mis-specified in the systematic component of the latent class membership function - in particular, the omission of variables that are spatially clustered. Overlooking this will, therefore, lead to bias, poor prediction and missed opportunities for insight. In this paper, we develop a latent class modelling framework, whereby spatial dependence in class membership is addressed.

 

  • Julian Sagebiel, Klaus Glenk, Robert Johnston, Jürgen Meyerhoff – "Comparing methods to account for spatial heterogeneity in discrete choice experiments"

    As willingness to pay values for environmental goods often vary by location, researchers have increasingly incorporated geo-statistical methods in the analysis of discrete choice experiments. It is challenging to test the validity of these approaches and, in many cases, different methods lead to different results. In this presentation, we briefly outline two approaches to model spatial preference heterogeneity and use the results to predict willingness to pay values on a map. The first approach relies on predicting willingness to pay based on spatial variables which are interacted with attributes. The second approach makes use of spatial interpolation techniques (kriging, inverse distance weighting). Here, geocoded individual willingness to pay values are used as observed points to predict willingness to pay in unobserved regions. We then propose ideas to compare the approaches and to test the validity of results.

     

  • Danny Campbell – "Spatial sampling strategies: should we be using them"

    The main disadvantage of a classical random sampling approach is that it ignores any spatial dependence. If spatial dependence exists, random sampling may lead to data redundancy. For example, many observations may be clustered in one area when perhaps one or two observations might suffice. Thus, in the presence of spatial dependence, random sampling is inefficient. The aim of spatial sampling methods is to get results of a higher quality at a lower cost. In this presentation, I discuss whether or not spatial sampling matters for stated choice experiments.

 

Session 2: Technical and methodological issues I

  • David Hoyos – "Dealing with endoneity in DCEs"

    Although dealing with endogeneity in classical regression models is well established in the econometric literature, how to deal with this problem in the framework of non-linear models like DCEs is a field that only recently has received some attention. The aim of this presentation is to discuss this issue by presenting an exploratory analysis of the performance of the Multiple Indicator Solution (MIS) method to dealing with endogeneity in DCEs. This method will be tested using three environmental valuation datasets conducted in recent years.

 

  • Wiktor Budziński, Mikołaj Czajkowski – "Hybrid Choice Models and accounting for the endogeneity of indicator variables: a Monte Carlo investigation"

    We dispel the common misconception that the hybrid choice models address endogeneity of indicator variables. Through a Monte Carlo analysis, we demonstrate that similarly to directly interacting indicator variables with the choice attributes, including them in the measurement or structural component of the hybrid choice models does not automatically account for possible correlations. We propose a few solutions, although their practical usefulness is yet to be confirmed.

 

 

  • Marco Boeri, Alberto Longo – "The impact on welfare analysis of not modelling scale heterogeneity: a Monte Carlo experiment"

    This note investigates, by mean of Monte Carlo simulation, the bias caused by the presence of a difference scale parameter across groups and the presence of individual scale parameter not incorporated in the model.  The Monte Carlo study is conducted generating 1000 samples assuming the presence of 3 groups with different scale parameters – specifically equal to 1, 0.5 and 2. Main Findings: Estimating MNL models on datasets with a DGP that includes scale heterogeneity (both continuous and discrete) does not have a strong impact on parameter estimations different from cost, however the presence of scale impacts strongly on the cost coefficient. This has an important effect on Welfare analysis. In fact the WTPs are biased much more than the corresponding parameters. This finding is in line with what has been assumed so far in the literature and highlights the importance of including scale heterogeneity in modeling people preferences.

 

Session 3: Random regret

  • Caspar G. Chorus, Sander van Cranenburgh, Cristian Angelo Guevara – "New insights on random regret minimization models"

    This paper develops new methodological insights on Random Regret Minimization (RRM) models. It starts by showing that the classical RRM model is not scale-invariant, and that – as a result – the degree of regret minimization behavior imposed by the classical RRM model depends crucially on the sizes of the estimated taste parameters in combination with the distribution of attribute-values in the data. Motivated by this insight, this paper makes three methodological contributions: (1) it clarifies how the estimated taste parameters and the decision rule are related to one another; (2) it introduces the notion of “profundity of regret”, and presents a formal measure of this concept; and (3) it proposes two new family members of random regret minimization models: the µRRM model, and the Pure-RRM model. These new methodological insights are illustrated by re-analyzing 10 datasets which have been used to compare linear-additive RUM and classical RRM models in recently published papers. Our re-analyses reveal that the degree of regret minimizing behavior imposed by the classical RRM model is generally very limited. This insight explains the small differences in model fit that have previously been reported in the literature between the classical RRM model and the linear-additive RUM model. Furthermore, we find that on 4 out of 10 datasets the µRRM model improves model fit very substantially as compared to the RUM and the classical RRM model.

 

  • Marco Boeri, Alberto Longo – "Information and choice paradigms in the preferences for renewable energy"

    This study aims to explore the impact of using two choice paradigms in deriving preferences for a stated discrete choice experiment on renewable energy programmes: the Random Utility Maximization (RUM) and the Random Regret minimization (RRM). In general RRM described better respondents’ choices. When considering both choice paradigms in a hybrid model Hybrid we included 3 classes RRM completely free, we get one class with scale = 0 (pure RR) and one with scale very high (RU) plus a third class with scale not significantly different from zero (but very low probability). Furthermore these models are not identifiable (numerical problem when scale = 0). So given Van Cranenburgh et al, (2015), we suggest to change the class with scale = 0 with a pure RR model and the class with very high scale with a RUM model. We do so and we get a significant model using LC without heterogeneity (RRM has higher probability and members of environmental org have higher prob of being RUM). When accounting for preference heterogeneity, hybrid models are not identifiable. Indeed, when considering preference heterogeneity the best model is to estimate P-RR, with truncated distribution to avoid to cross zero. To test the robustness of the survey instrument, we assessed whether additional information can affect variance of the utility function, profundity of regret or impact on results. When considering RUM we find that varying the level of information has no impact on either preferences or variance, while on regret having less information results in higher profundity of regret (PRR).

 

  • Romain Crastes – "What do we gain from introducing different decision rules in non-market valuation?"

    This paper introduces the μRandom Regret Minimization (μRRM) to the field of non-market valuation. The μRRM approach has been recently developed by Cranenburgh et al. (2015). It allows to model the degree of regret minimization behaviour imposed by a RRM model. In this paper we introduce the mixed  μRRM where the degree of regret is allowed to vary across respondents and across attributes and we compare the performances of this model to linear-additive RUM and classical RRM.

 

  • Marco Boeri, Alberto Longo – "The regret of not modelling regret in choice experiments: a Monte Carlo investigation"

    The Random Regret Minimization (RRM) approach to discrete choice analysis has been developed in the context of modelling the demand for travel, and, since then, it has been used in other fields including the demand for outdoor recreation and health. It presents a tractable, regret-based alternative to the dominant choice-modelling paradigm based on Random Utility Maximization (RUM). The idea that regret is an important determinant of choice behaviour is acknowledged, theoretically and empirically, in many fields including marketing, microeconomics, psychology, the management sciences, transportation and health. However,  no previous study has yet measured the bias that the presence of a RRM behaviour in a dataset can create to estimations and welfare analysis based only on the RUM assumption. This paper explores and measures, by means of Monte Carlo simulations, the bias caused by estimating a multinomial logit model assuming that the data conforms to the RUM choice behaviour only, whilst the data presents a mixture of the two choice paradigms, both the RUM and the RRM. This bias is investigated with a gradually higher presence of the RRM choice behaviour in the Data Generation Process (DGP). We simulated 13 different datasets generating 1,000 samples each. The DGP is based on estimates from data of a real study in health economics aimed at testing the trade-off that people are willing to make between life style choices, in terms of diet and physical activity, and the risk of dying from cardiovascular disease in the next 10 years. We find that MNL models based on the RUM paradigm on datasets with a DGP that includes choices generated by a RRM approach results in biased parameters estimation. As expected, this bias is intensified by the increased proportion of choices that follow a RRM paradigm in the DGP, with the bias been very important when the proportion of choices driven by the RRM is about 50%. A further finding is interesting and counterintuitive: the bias is not as strong on willingness to pay estimates as on parameter estimates. We finally conclude supporting the idea of developing methods that allow for the presence of both paradigms – such as latent class models – and explore new experimental designs allowing for the presence of both choice paradigms.

 

Special session 1: Basic discrete choice models in R

 

  • Petr Mariel – "Basic discrete choice models in R"

    R is a language and environment for statistical computing and graphics. It is available as Free Software under the terms of the Free Software Foundation’s GNU General Public License in source code form. It compiles and runs on a wide variety of UNIX platforms and similar systems, Windows and MacOS. This presentation is a short introduction to the discrete choice modelling using R. There are many different models applied in the literature, but we focus on the core ones: multinomial logit, latent class and mixed logit models. Detailed worked examples based on simulation exercises will be used to show in a didactic way the estimation procedures as well as the post-estimation analysis.

 

Session 4: SP elicitation theory and methodology

  • Fredrik Carlsson, Mitesh Kataria, Elina Lampi – "Old and new aspects of respondent behavior in stated preference surveys"

    We discuss the various aspects of why we sometimes observe a difference in behavior between a real situation and the corresponding survey situation. In particular, we investigate the role of the budget, available substitutes and experimenter demand effects. We find that an explicit budget exercise decreases the willingness to pay to improved water, primarily as a shift to an opt-out. An explicit mentioning of substitute uses of the money (on other environmental projects) has little effect on respondent behavior). Finally we developed a so-called experimenter demand script, which we find reduce willingness to pay for some of the attributes of the experiment.

 

  • Ulf Liebe, Klaus Glenk – "Are the Effects of Real Incentives in Stated Choice Experiments Context Dependent? A Comparison of Choice Behavior in Online and Field Environments"

    We compare the results of hypothetical and real choice experiments on ethical consumption – the purchase of organic and fair trade tea – carried out in an online survey and field setting (i.e. “research station” in the supermarket). We use propensity score matching to make the online and field data comparable (circa n=100 per group, online vs. field as well as hypothetical vs. real). Our findings indicate that the social context matters. First, willingness to pay is higher in the field setting than in the online setting. Second, we find a hypothetical bias which tends to be larger in the field setting. Third, men and women seem to react differently to real incentives and social contexts. This study contributes to the research on the hypothetical bias in stated choice data by showing that the social context is relevant for the effects of real incentives on individual’s choices.

 

  • Romain Crastes, Pierre-Alexandre Mahieu, Jordan Louviere, Ewa Zawojska – "Rewarding truthful-telling in stated preference studies"

    Stated preference surveys rarely provide respondents with such conditions in which a respondent’s optimal strategy is to answer truthfully. As a result, reliability of stated preference data is often questioned. We consider a new method, grounded in economic theory, to incentivize respondents to answer truthfully. Our method is based on a lie detector coupled with a reward. We discuss theoretical predictions of the method, and test them empirically in a split sample choice experiment dealing with a tree planting program. We find that the lie detection (i) increases the time spent to complete the valuation tasks and (ii) decreases the variance of the error term by using a hybrid choice model that accounts for possible endogeneity. Our results are encouraging but more research is needed to assess the validity of this new approach.

 

  • Katarzyna Zagórska, Mikołaj Czajkowski, Nick Hanley, Jacob LaRiviere, Natalia Letki – "Do social norms matter for environmental preferences?"

    Do social norms matter? We investigate by experimentally varying the information about the social norm communicated to respondents in two empirical studies dealing with household recycling and GMO foods.

 

  • Søren Bøye Olsen, Kennet Uggeldahl – "Show me the money"

    We test whether illustrating the cost attribute with pictures of real money can help reduce the welfare estimates derived from hypothetical discrete choice experiments, arguing that this method might help mitigate hypothetical bias. In a between sample design, we vary the presentation of the cost attribute, finding that pictures of real money significantly reduce willingness to pay estimates. The effect cannot be attributed to the visual presentation alone, as estimates do not differ between the control treatment and a treatment with a generic illustration of money, but only appear when real money is used in the illustration of the cost. These results are in line with previous findings in the behavior economics literature, and could improve the design of stated preference surveys.

 

  • Morten Raun Mørkbak – "Time preferences and DCE"

    Time preferences are important determinants of health related behavior – since such behavior often implies an intertemporal choice. An often used approach in eliciting time preferences is a choice experiment in the sense of a multiple price list, where respondents have to trade-off a smaller-sooner reward over a larger-later reward – the switching point is then used as the interval of the discount rate. Alternatively – one a more regular choice experiment can be used, but were we run into usual scaling issue when comparing discount rates across groups/segments. Finally, one can elicit time preferences using a more simple time-trade-off method, and then use this as e.g. class specific variable in a latent class model – segmenting individuals according to time preferences and specific preferences in a giving DCE.

 

  • Ulf Liebe – "A different approach to stated choice experiments: new developments in political science"

    Stated choice experiments are increasingly used in political science research. Here, researchers focus on causal inference, use fully randomized designs and non-parametric models. I would like to discuss how this perspective might complement choice experiment research in environmental economics and vice versa.

 

Special session 2: Good practice recommendations for DCE beginners

 

  • Jürgen Meyerhoff, Klaus Glenk, David Hoyos, Jette Jacobsen, Petr Mariel, Søren Olsen, … – "Good practice recommendations for DCE beginners – but not only"

    The good practice recommendations are meant to be a list of issues to consider when designing a choice experiment, subsequently analysing data and reporting results. They were (and will be) chosen because a) we found them to be relevant when doing choice modelling, b) students raised them, or c) we stumbeld over them as reviewers. As our background is within environmental economics, the issues raised are with inspiration from applications in this field. The main objective is to provide info on good practise to help PhD students and practitioners improving at the same time hopefully the quality of studies and published papers especially in interdisciplinary contexts.

 

Session 5: Technical and methodological issues II

  • Jürgen Meyerhoff, Katrin Rehdanz, Christine Bertram – "Preference matching effects - it's always good to have more choice options, isn't it?"

    Present studies investigating the effects of the number of alternatives presented on a choice set have mainly found that more alternatives seem to be beneficial. However, mostly only choice sets with two and three alternatives, including a status quo alternative, have been compared. Thus, we use five split samples (300 respondents each) to vary the number of alternatives from two to six alternatives on a choice set, keeping all other design dimensions equal. Overall, respondents were presented 12 sets, first eight from the experimental design, and afterwards four sets randomly drawn out of the first eight sets. In addition, individual decision making styles a captured through various items batteries concerning, for example, maximization, regret minimization, or impulsivity. One of the study objectives is to examine whether more is always good or only applies to the move from two to three alternatives.

 

  • Søren Bøye Olsen, Kennet Uggeldahl – "DCE, eyetracking and “gaze-contingency”"

    A new eyetracking experiment that includes a CE study. We hope to be able to incorporate some “gaze-contingency” stuff, where the an alternative/attribute in a choice sets will only be visible if the respondent actually looks at it. The aim is to be able to learn more about true non-attendance.

 

  • Danny Campbell – "Speedy Gonzales! Some thoughts on speeders and what to do with them"

    There are a growing number of papers looking response time. But there are still some unresolved issues. In this presentation, I will give my tuppence worth on these issues.

 

  • Mikołaj Czajkowski, Wiktor Budziński – "The effects of different specifications of standard deviations in the MXL model"

    We present a Monte Carlo simulation results demonstrating the effects of different specifications for the standard deviations in the MXL model. It turns out that operationalizing them as linear (and ignoring sign), absolute value, exp etc. substantially impacts model results. The conclusions for the field are discussed.

 

  • Mikołaj Czajkowski, Wiktor Budziński – "Choice task blocking and design efficiency"

    We share some insights from the investigation of how blocking function works in NGENE. We suggest practical ways to improve design efficiency.

 

  • Marco Boeri, Juan Marcos Gonzalez – "Functional forms considerations in Maximum Acceptable Risk calculations"

    Maximum acceptable risk (MAR) is commonly used to incorporate patients’ preferences into the evaluation of benefit-risk profiles for medications. Often, relative preferences are elicited for more than two levels of risk for a particular treatment adverse event. The researcher must define a specification for the inclusion of the risk in the choice model, which implies making assumptions about respondents’ relative preferences for levels that were excluded from the experimental design, but fall within the range of the risks considered. We explore the impact of these assumptions on the calculation of MARs and evaluate how differences in MARs induced by specification choices could affect benefit-risk evaluations of treatments.

 

  • Morten Raun Mørkbak, Line Bjørnskov Pedersen, Riccardo Scarpa – "Handling resolvable uncertainty from incomplete choice set scenarios – choice probabilities versus discrete choices"

    Forecasting choice behavior for new health care, environmental or transportation programs and services is challenging because actual data is often unavailable. In order to derive estimates of the demand for such programs and services researchers often must resort to data derived from hypothetical market scenarios. An increasing popular way of doing this is by means of hypothetical (Discrete) Choice Experiments (DCE). Respondents participating in a hypothetical discrete choice experiment are likely to be provided with only a subset of the information deemed relevant or even necessary for conducting a real life choice. Manski (1990) argues that even under best case hypothesis, intentions stated during DCE survey will not be good predictors of future behavior, since scenario descriptors will always be at least in part “incomplete”. Such unavoidable incompleteness will be at least in part resolved in a real choice context, which gives rise to a component of uncertainty referred to as “resolvable” because once faced with a real choice scenario subjects will have some uncertainty resolved. Cognizant of this fact analysts are faced by an extrapolation problem in which assumptions are likely to be crucial and hence matter. However, eliciting choice probabilities (ECP) instead of stated choices could potentially overcome this issue, by allowing respondents to explicitly state uncertainty about their stated choice. It turns out that this approach might afford the additional advantage of being less econometrically demanding. In the present paper we compare the elicited subjective choice probabilities approach with the standard DCE approach using a split sample design in a health care context. The very preliminary results show large differences with respect to willingness-to-pay estimates, but remarkable similarities with respect to forecasting abilities, suggesting the validity of the far less econometrically demanding ECP approach, which would seem to be at least as good as the usual more demanding DCE approach. Furthermore, we extend the model of the ECP approach by distinguishing between those with at least some resolvable uncertainty and those with only unresolved uncertainty by using separate simultaneous equations related to the choice attributes. This is done by fitting a logit distribution to the two extreme probability processes (zero and one) and a Beta distribution to the intermediate process.