Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Jan 8;14(1):10.
doi: 10.1038/s41398-023-02717-7.

Exploring the steps of learning: computational modeling of initiatory-actions among individuals with attention-deficit/hyperactivity disorder

Gili Katabi  1 , Nitzan Shahar  2   3
Affiliations

Exploring the steps of learning: computational modeling of initiatory-actions among individuals with attention-deficit/hyperactivity disorder

Gili Katabi et al. Transl Psychiatry. .

Abstract

Attention-deficit/hyperactivity disorder (ADHD) is characterized by difficulty in acting in a goal-directed manner. While most environments require a sequence of actions for goal attainment, ADHD was never studied in the context of value-based sequence learning. Here, we made use of current advancements in hierarchical reinforcement-learning algorithms to track the internal value and choice policy of individuals with ADHD performing a three-stage sequence learning task. Specifically, 54 participants (28 ADHD, 26 controls) completed a value-based reinforcement-learning task that allowed us to estimate internal action values for each trial and stage using computational modeling. We found attenuated sensitivity to action values in ADHD compared to controls, both in choice and reaction-time variability estimates. Remarkably, this was found only for first-stage actions (i.e., initiatory actions), while for actions performed just before outcome delivery the two groups were strikingly indistinguishable. These results suggest a difficulty in following value estimation for initiatory actions in ADHD.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

Fig. 1
Fig. 1. Trial sequences for the multiple-stage task.
Participants were told that a puppy was hiding behind a chest and that their task was to locate the puppy. A Each trial individuals made choices across three stages (houses, doors, and chests) to try and locate the puppy. B State-action transition structure shows a deterministic transition from stage one (houses), stage two (doors), and stage three (chests). C We used a typical reinforcement learning design where the true expected value (probability of finding the puppy for each chest) for each chest drifted slowly across the block, thus requiring participants to keep learning across the task, similar to conventional reinforcement-learning paradigms [, –97].
Fig. 2
Fig. 2. Effect of choice difficulty on accuracy rates and reaction-time variability.
A The influence of choice difficulty, group (HC vs. ADHD), and stage (1, 2, or 3) on participants’ accuracy rates. First, results demonstrate that across stages and groups, accuracy rates improve as a function of choice difficulty. Importantly, ADHD individuals showed an attenuated sensitivity to choice difficulty changes only in the first stage but not in the second and third stages. B The influence of choice difficulty, group, and stage on participants’ RT variability. Results demonstrate that across stages and groups, RT variability improved when choices were easier. Importantly, ADHD individuals showed an attenuated sensitivity to choice difficulty changes only in the first stage (y-axis represents log(τ) estimates for an ex-gaussian distribution fitted to empirical data).
Fig. 3
Fig. 3. Effect of latent internal values difference for each action on accuracy rates and reaction-time variability.
A Choice accuracy estimates as a function of group, stage, and absolute difference between the internal values estimated using a reinforcement learning model (i.e., |ΔQ|). B RT variability as a function of group, stage and absolute difference between internal action values (i.e., |ΔQ|). (y-axis represents log(τ) estimates for an ex-gaussian distribution fitted to empirical data).
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. American Psychiatric Association (ed.). Diagnostic and statistical manual of mental disorders: DSM-5. 5th ed. American Psychiatric Association: Washington, D.C; 2013.
    1. Faraone SV, Banaschewski T, Coghill D, Zheng Y, Biederman J, Bellgrove MA, et al. The World Federation of ADHD International Consensus Statement: 208 Evidence-based conclusions about the disorder. Neurosci Biobehav Rev. 2021;128:789–818. doi: 10.1016/j.neubiorev.2021.01.022. - DOI - PMC - PubMed
    1. Thomas R, Sanders S, Doust J, Beller E, Glasziou P. Prevalence of attention-deficit/hyperactivity disorder: a systematic review and meta-analysis. Pediatrics. 2015;135:e994–e1001. doi: 10.1542/peds.2014-3482. - DOI - PubMed
    1. Lara C, Fayyad J, de Graaf R, Kessler RC, Aguilar-Gaxiola S, Angermeyer M, et al. Childhood predictors of adult attention-deficit/hyperactivity disorder: results from the World Health Organization World Mental Health Survey Initiative. Biol Psychiatry. 2009;65:46–54. doi: 10.1016/j.biopsych.2008.10.005. - DOI - PMC - PubMed
    1. Langberg JM, Smith ZR, Dvorsky MR, Molitor SJ, Bourchtein E, Eddy LD, et al. Factor structure and predictive validity of a homework motivation measure for use with middle school students with attention-deficit/hyperactivity disorder (ADHD) Sch Psychol Q. 2018;33:390–8. doi: 10.1037/spq0000219. - DOI - PubMed