S.1. Supplemental Information
The online supplement provides additional reading and background considerations for readers interested in delving deeply into the conceptual and empirical issues that remain
unresolved and heavily investigated. These issues have been moved to the supplement to avoid diverting the primary discussion with issues of interest primarily only to experts.
Major Issue One: The characteristics of top down control in relation to consciousness and resource capacity.
Several descriptors of top down control are debated. Consciousness is a particularly problematic descriptor. The psychological literature admits of a range of seemingly “higher order” mental operations that can be carried out without awareness (Dijksterhuis & Strick, 2016; Marien, Custers, Hassin, & Aarts, 2012; Papies & Aarts, 2011). Models of cognitive control retain Type II features but not consciousness (Botvinick & Cohen, 2014). It may be that Type II processes include operations always available to consciousness, without being an immediate focus of attention and “non-conscious.”
A second major debate concerns the limits on the phenomenon of limited resource. Top-down processing clearly has limited capacity in that only so many items can be processed at one time. A separate issue is whether there is a limited resource, or whether there is a fatigue factor over time (vigilance effect) or both. Baumeister and colleagues (Baumeister &
Heatherton, 1996; de Ridder, Lensvelt-Mulders, Finkenauer, Stok, & Baumeister, 2012), have proposed an account of limited resource supported by dual task experiments in which
completion of a task demanding cognitive control is followed by reduced performance on the next task demanding control. The first major meta-analysis of this literature covered almost 200 tests and concluded a robust effect size of about 0.7 (Hagger, Wood, Stiff, & Chatzisarantis, 2016). However, subsequent re-analysis using different methods argued that this estimate was inflated and the effect might not exist (Carter, Kofler, Forster, & McCullough, 2015). Supporting the latter view, Hagger, Chatzisarantis, et al (2016) reported a multi-lab, pre-registered
replication effort on one particularly experimental test that yielded a null effect. They propose that a fatigue effect (wearing out of capacity after extended use) may exist but be different than a resource depletion effect after acute effort. All of this does not address the fact that capacity is limited (only a finite amount of information can be processed by controlled processes). A side-debate exists as to what is depleted if something is, and whether it is energy, resource, motivation. One particular proposal, glucose depletion, appears to be an unlikely explanation (Botvinick & Braver, 2015; Vadillo, Gold, & Osman, 2016). However, Killeen and colleagues (2016) speculated a different mechanism, a cellular basis in neuronal energy supply for this capacity that supports top-down control, related to the astrocyte-neuron lactate shuttle. This could account for a fatigue effect or, if it exists, an ego-depletion effect. It has yet to face empirical investigation. In short, under some conditions control may behaves as if there is a limited resource, but the extent of this phenomenon remains unclear and should be decoupled from capacity. Even a limited capacity lacks an agreed its psychological or biological basis (Evans & Stanovich, 2013; Sergeant et al., 1999). Connectionist models can account for an apparent capacity limit by requiring serial processing of competing information, which leads to an apparent capacity limit (i.e., a “bottleneck”)—but does not require a posited resource pool (Botvinick & Cohen, 2014).
Major Issue Two. What is the correct way to represent and study motivated decision making? Decision making paradigms constrain to a two-choice decision between options that require a trade-off, then map the shape of the decision function mathematically. When faced
with an immediate versus a later payoff, a purely “logical” discounting function is exponential— how a computer makes investment decisions. However, humans and animals discount
according to a hyperbolic discounting function, implying “excess” discounting of future or uncertain rewards. All the equations that capture the discounting process are related to
psychophysical laws, i.e., the Law of Effect (Thorndike, 1933), Weber-Fechner-Stevens power law (Namboodiri, Mihalas, & Hussain Shuler, 2016; Namboodiri, Mihalas, Marton, & Hussain Shuler, 2014) or the related matching law. The hyperbolic model is a general model that in practice has been subject to sophisticated computational refinements, such as a
quasi-hyperbolic function (L. Green & Myerson, 2004; Mitchell et al., 2015), that account for both an immediacy preference and a calculated consideration of delayed payoffs (McClure, Laibson, Loewenstein, & Cohen, 2004), and are thus attempt to be more mechanistic rather than merely descriptive (Mitchell et al., 2015). Weber’s law observes that the just noticeable difference in two magnitudes is not constant, but rather is small when absolute magnitudes are small and larger when magnitudes are larger, and the change is proportional.
Fechner’s modification was that perception varies linearly with the log of actual intensity. Stevens added that for some modalities the function is not strictly logarithmic but nonetheless obeys a particular power function, so the general formula became S=kIa, where S is perceived intensity, k is an arbitrary constant denoting the units of measurement (and preventing intensity reaching infinity when delay is 0), I = objective intensity, and a is a multiplier (power exponent) that varies with modality (vision, hearing, time, probability). Herrnstein’s (1961) matching “law” addresses not perception but ratios of choice to operant reinforcement. The initial form was R1/R2 = Rf1/Rf2, that is, the ratio of response 1 to response 2 is proportional to their ratio of reinforcement. The equation can be modified to situations that vary in magnitude or delay rather than rate, resulting in the generalized matching law (Baum, 1974). An updated version that takes into account response bias is log(R1/R2)=log(b)+s*log(Rf1/Rf2), where, borrowing from signal detection theory, log(b) is bias and s is sensitivity (referred to as d or d-prime in signal detection theory).
In a simplified two-choice situation where reward varies in delay but not frequency, the equations yield the hyperbolic discounting function. Herrnstein’s proposal seen as a special case of logistic regression by Killeen (2015), who argues that it is not actually a law but a descriptive method, and contends that the “law of effect” from Thorndike was superior. Active work continues regarding the proper mathematical representation of these decision properties in relation to different moderating conditions (Killeen, 2009; Mitchell et al., 2015; Redish & Kurth-Nelson, 2010). However, hyperbolic discounting by living organisms is not “irrational” (an unfortunate view sometimes espoused) because outside of the monetary or investment sphere, it is often adaptive. Indeed, a major problem with psychopathology experiments has been excessive reliance on monetary rewards. Monetary discounting behaves differently than discounting of consumables like food (Killeen, Russell, & Tannock, 2016).
Crucially, the models are not one-size fits all. In particular, interpretation of the literature and design of future studies requires the recognition that temporal discounting curves vary by domain (e.g., academic versus social) (Tsukayama, Duckworth, & Kim, 2013), as well as by target or reward (e.g., monetary versus consumable) (L. Green & Myerson, 2013). Time
perception is a critical under-explored parameter in SR. Although it is unclear whether time
perception operates by psychophysical principles, it can be described by the same equations (Killeen, 2015). A key future direction is to better relate discounting research to the literature on time perception and its alterations in ADHD and other conditions related to poor SR and/or to impulsivity in particular (Hwang-Gu & Gau, 2015; Namboodiri et al., 2014).
Second, just as mathematical and psychophysical approaches consider the components of impulsivity or decision making, in the personality literature, an early componential model proposed that impulsivity can be based on the following components: “negative urgency”
seeking (perhaps related to temporal discounting of future reward), and lack of perseverance (Whiteside & Lynam, 2001). It remains unclear whether these proposed factors represent
distinct influences on psychopathology or can be simplified (Berg, Latzman, Bliwise, & Lilienfeld, 2015) as the same impulsivity in different contexts (e.g., social vs cognitive), perhaps in parallel to the experimental discounting findings.
Major issue three: What exactly are the qualities that define impulsivity and risk taking and how are they best studied?
Each adjective associated with impulsivity can be and has been debated. Is it non-reflective? Disadvantageous? Here are a few elaborations as well as a comment on a related issue related to risk-taking.
Is it non-reflective? First, a non-reflective specifier has ample precedence (Stevens & Stephens, 2010). DeYoung (2011) suggested that impulsivity is simply the decision to act on immediate urge (or, by implication, for short-term gain) despite or regardless of consideration of long-term outcomes (his pg. 486-487). My view is that “reflective” or deliberate impulsivity is a sufficiently different construct as not to belong in the impulsivity rubric here. Clearly, one can choose an immediate reward for strategic rather than simply stimulus driven reasons, but this is not impulsive (see Scholer and Higgins (2010).
Is it regrettable? Second, a regrettable specifier has ample precedent in the literature (Ainslie, 2001). However, I decline to require it here because it is almost unspecifiable at a general level (adaptive from one point of view may be maladaptive from another)—therefore psychopathology requires operational finding of adaptiveness but the construct of impulsivity does not. First, hasty choices for immediate reward are not inevitably regrettable. If impulsive is merely hasty choice for immediate reward, then adaptive impulsivity can be conceptualized and operationalized as distinct from maladaptive impulsivity (Dickman, 1990). Second, the
subjective goal of the actor may be quite different than what the experimenter thinks it is, because in actual decision-making humans have many goals simultaneously (Scholer & Higgins, 2010). Third, that “adaptation” has an evolutionary meaning as well. For example, developmental origins theory posits that the fetus develops in response to biological signals from the mother to predict the post-birth environment. A “thrifty phenotype” hypothesis suggests that an impulsive style could emerge on expectation of an environment in which that style is most adaptive (Reis et al., 2016). If the prediction is wrong, and the environment demands a reflective style for maximum fitness, then that individual will have “maladaptive impulsivity” as the result of a biological process aimed at adaptation. Parallel arguments apply in regard to a tendency for risk-taking. However, flexibility is a feature needing more study. A future integration can relate to the fact that flexibility of control style is the hallmark of Block’s idea of ego
resiliency (J. H. Block & Block, 1980) (Table 1).
How is it best modeled? Third, however, whether impulsivity is adaptive is poorly assessed in typical experimental designs in which one must choose the larger, later reward to “win.” In real life, the best choice depends on context. In an evolutionary framework, the longer an animal (or a hunting human group) waits to consume an immediately catch, the more chances of something going wrong and losing what it has. Yet, stopping to consume what’s available has an opportunity cost: continuing to hunt means exerting more energy and risking getting weaker, yet it may yield a greater reward than stopping to eat, particularly if the hunting or fishing is good and we can hold on to the immediate catch (Stevens & Stephens, 2010). This problem is addressed with a foraging model and “patch” experiment. Both humans (E. C. Carter, Pedersen, & McCullough, 2015) and non-human primates (Blanchard & Hayden, 2015) learn to maximize long-term gain by picking smaller, immediate rewards when context allows, or picking later, larger rewards otherwise. Thus, simple preference for immediate reward is not necessarily maladaptive.
Is risk taking the choice for unpredictability? Fourth, in terms of definitions, risk-taking has been defined as the selection of the option with the greatest variability in outcome (greatest potential loss and gain) (Crone, van Duijvenvoorde, & Peper, 2016). While this is defensible, it is clearer to isolate the fact that risk-taking emanates from a general style that is punishment insensitive relative to reward attraction. If negative and positive consequences are correlated, then punishment insensitivity leads to gambling on the largest reward, and the largest loss (i.e., the most variable outcome). But risk-taking more generally is action taken with disregard for uncertain but potential loss (i.e., reward-seeking is risk-taking in the context of uncertain outcome and potential loss).
Literature Cited in Supplement
Ainslie, G. (2001). Breakdown of will. New York, NY, US: Cambridge University Press. Baumeister, R. F., & Heatherton, T. F. (1996). Self-regulation failure: An overview. Psychological Inquiry, 7(1), 1-15.
Berg, J. M., Latzman, R. D., Bliwise, N. G., & Lilienfeld, S. O. (2015). Parsing the
heterogeneity of impulsivity: A meta-analytic review of the behavioral implications of the upps for psychopathology. Psychological Assessment, 27(4), 1129-1146. doi:10.1037/pas0000111
Blanchard, T. C., & Hayden, B. Y. (2015). Monkeys are more patient in a foraging task than in a standard intertemporal choice task. PLoS One, 10(2), e0117057.
Block, J. H., & Block, J. (1980). The role of ego-control and ego-resiliency in the origination of behavior. In w. A. Collings (ed.), The Minnesota Symposia on Child Psychology (Vol. 13). Hillsdale, NJ: Erlbaum.
Botvinick, M., & Braver, T. (2015). Motivation and cognitive control: From behavior to neural mechanism. Annual Review of Psychology, 66, 83-113. doi:10.1146/annurev-psych-010814-015044
Botvinick, M., & Cohen, J. D. (2014). The computational and neural basis of cognitive control: Charted territory and new frontiers. Cognitive Science, 38(6), 1249-1285.
doi:10.1111/cogs.12126
Carter, E. C., Kofler, L.M., Forster, D.E., & McCullough, M.E. (2015). A series of meta-analytic tests of the depletion effect: Self-control does not seem to rely on a limited resource. Journal of Experimental Psychology: General, 144, 796-815.
Carter, E. C., Pedersen, E. J., & McCullough, M. E. (2015). Reassessing intertemporal choice: Human decision-making is more optimal in a foraging task than in a self-control task. Frontiers in Psychology, 6. doi:10.3389/fpsyg.2015.00095
Crone, E. A., van Duijvenvoorde, A. C., & Peper, J. S. (2016). Annual research review: Neural contributions to risk-taking in adolescence - developmental changes and individual differences. Journal of Child Psychology and Psychiatry, 57(3), 353-368.
doi:10.1111/jcpp.12502
de Ridder, D. T., Lensvelt-Mulders, G., Finkenauer, C., Stok, F. M., & Baumeister, R. F. (2012). Taking stock of self-control: A meta-analysis of how trait self-control relates to a wide range of behaviors. Personality and Social Psychology Review, 16(1), 76-99.
doi:10.1177/1088868311418749
Dickman, S. J. (1990). Functional and dysfunctional impulsivity: Personality and cognitive correlates. Personality and Social Psychology Review, 58(1), 95-102.
Dijksterhuis, A., & Strick, M. (2016). A case for thinking without consciousness. Perspectives on Psychological Science, 11(1), 117-132. doi:10.1177/1745691615615317
Evans, J. S., & Stanovich, K. E. (2013). Dual-process theories of higher cognition: Advancing the debate. Perspectives on Psychological Science, 8(3), 223-241.
doi:10.1177/1745691612460685
probabilistic rewards. Psychological Bulletin, 130(5), 769-792. doi:10.1037/0033-2909.130.5.769
Green, L., & Myerson, J. (2013). How many impulsivities? A discounting perspective. Journal of the Experimental Analysis of Behavior, 99(1), 3-13. doi:10.1002/jeab.1
Hagger, M.S., & Chatzisarantis, N.L.D. (2016). A multi-lab preregistered replication of the ego-depletion effect. Perspectives on Psychological Science, 11, 546-573.
Hagger, M.S., Wood, C, Stiff, C., & Chatzisarantis, N.L.D.(2016). Ego depletion and the strength model of self-control: A meta-analysis. Psychological Bulletin, 136, 495-525.
Hwang-Gu, S. L., & Gau, S. S. (2015). Interval timing deficits assessed by time
reproduction dual tasks as cognitive endophenotypes for attention-deficit/hyperactivity disorder. PLoS One, 10(5), e0127157. doi:10.1371/journal.pone.0127157
Killeen, P. R. (2009). An additive-utility model of delay discounting. Psychological Review, 116(3), 602-619. doi:10.1037/a0016414
Killeen, P. R. (2015). The logistics of choice. Journal of the Experimental Analysis of Behavior, 104(1), 74-92. doi:10.1002/jeab.156
Killeen, P. R., Russell, V. A., & Tannock, R. (2016). Neuroenergetics. Current Directions in Psychological Science, 25(2), 124-129. doi:10.1177/0963721416628530
Marien, H., Custers, R., Hassin, R. R., & Aarts, H. (2012). Unconscious goal activation and the hijacking of the executive function. Journal of Personality and Social Psychology, 103(3), 399-415. doi:10.1037/a0028955
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004). Separate neural systems value immediate and delayed monetary rewards. Science, 306(5695), 503-507. doi:10.1126/science.1100907
Mitchell, S. H., Wilson, V. B., & Karalunas, S. L. (2015). Comparing hyperbolic, delay-amount sensitivity and present-bias models of delay discounting. Behavioral Processes, 114, 52-62. doi:10.1016/j.beproc.2015.03.006
Namboodiri, V. M., Mihalas, S., & Hussain Shuler, M. G. (2016). Analytical calculation of errors in time and value perception due to a subjective time accumulator: A mechanistic model and the generation of weber's law. Neural Computation, 28(1), 89-117.
doi:10.1162/NECO_a_00792
Namboodiri, V. M., Mihalas, S., Marton, T. M., & Hussain Shuler, M. G. (2014). A general theory of intertemporal decision-making and the perception of time. Frontiers in Behavioral Neuroscience, 8, 61. doi:10.3389/fnbeh.2014.00061
Papies, E. K., & Aarts, H. (2011). Nonconscious self-regulation or the automatic pilot of human behavior. In K. D. Vohs & R. F. Baumeister (Eds.), Handbook of self-regulation:
Research, theory, and applications (2 ed., pp. 125-142). New York: Guilford Press.
Redish, A. D., & Kurth-Nelson, Z. (2010). Neural models of delay discounting. In G. J. M. W. K. Bickel (Ed.), Impulsivity: The behavioral and neurological science of discounting (pp. 123-158). Washington, DC, US: American Psychological Association.
Reis, R. S., Dalle Molle, R., Machado, T. D., Mucellini, A. B., Rodrigues, D. M., Bortoluzzi, A., . . . Silveira, P. P. (2016). Impulsivity-based thrifty eating phenotype and the protective role of n-3 pufas intake in adolescents. Translational Psychiatry, 6, e755. doi:10.1038/tp.2016.16
Scholer, A., & Higgins, E. T. (2010). Conflict and control at different levels of
self-regulation. In K. O. Ran Hassin, Yaacov Trope (Ed.), Self control in society, mind, and brain (pp. 312-355). Oxford Scholarship Online: Oxford University Press.
Stevens, J. R., & Stephens, D. W. (2010). The adaptive nature of impulsivity. In
Impulsivity: The Behavioral and Neurological Science of Discounting (GJ Madden & WK Bickel, eds.) pp. 361-388. Washington DC: American Psychological Association.
Thorndike, E. L. (1933). A proof of the law of effect. Science, 77(1989), 173-175. doi:10.1126/science.77.1989.173-a
Tsukayama, E., Duckworth, A. L., & Kim, B. (2013). Domain-specific impulsivity in school-age children. Developmental Science, 16(6), 879-893. doi:10.1111/desc.12067
Vadillo, M.A., Gold, N., & Osman, M. (2016). The bitter truth about sugar and willpower: The limited evidential value of the glucose model of ego-depletion. Psychological Science. 2016 Sep;27(9):1207-14. doi: 10.1177/0956797616654911. PubMed PMID: 27485134.
Whiteside, S. P., & Lynam, D. R. (2001). The five factor model and impulsivity: Using a structural model of personality to understand impulsivity. Personality and Individual Differences, 30(4), 669-689. doi: http://dx.doi.org/10.1016/S0191-8869(00)00064-7
