Origins of metacognition in evolution and development by smfleming

Origins of metacognition in evolution and development

Steve Fleming Wellcome Centre for Human Neuroimaging, UCL stephen.fleming@ucl.ac.uk

metacoglab.org

Outline of the course 1. Introduction to metacognition: theory and measurement

2. Origins of metacognition in evolution and development

3. Neural architecture of metacognition

4. Functional roles of metacognition in behavioural control (Rouault)

Origins of metacognition?

Formal approach to the study of reflective awareness

How do we measure metacognition?

BEHAVIOUR E.g. answer to exam question

SECOND-ORDER REPORT E.g. confidence in getting the answer right

Types of second-order report

Judgment of learning Feeling of knowing Confidence

Error monitoring Post-decision wager

Explanation Timing Will you be able to Prospective recall this item in the future? Will you be able to Prospective recognise the right answer?

Domain Memory

How confident are Retrospective you in your answer? Did you make an Retrospective error? How much would Retrospective you bet that your decision is correct?

Decision-making/ memory

Memory

Decision-making/ memory Decision-making/ memory

Smith’s Paradigm in the The opt-out response

Smith, i h J. D., Schull, h ll J., Strote, J., McGee, K., Egnor, R., bottlenosed dolphin (Tursiops truncatus). J Exp Psychol

• • • •

Judgment of pitch height: 2100 Hz or a lowe L H The animal can refuse the test and is then ass Discrimination is excellent, but the animal sp or canEalso show signs of hesitation The animal precisely in the same frequency range.

Smith et al. (1995) JEP:General

• •

Judgment of the density of screen Smith’s Paradigm: Th « star The » key k provides id escape from f the the h « uncertain » response primary task and prompts butD invariant Smith Smith, JJ. D D., Shields Shields, W. W E., E Schull J., Schull, Ja modest & Washburn, Washburn D. A A. (1997) (1997). The uncertain response in humans and animals animals. Cognition, 62(1), 75-97. response The opt-out reward. macaques The animal candensity then manifest its « uncertainty » • • Judgment of the of screen « lack self-confidence • The Thor its « star » key k of provides id escape »from f the h

•

primary task and prompts a modest but invariant reward. The animal can then manifest its « uncertainty » or its « lack of self-confidence »

humans

Smith et al. (1997) Cognition

A cautionary note

Smith’s Paradigm in the bottlenosed D

Smith, i h J. D., Schull, h ll J., Strote, J., McGee, K., Egnor, R., & Erb, b L. ((1995). ) The h uncertain i bottlenosed dolphin (Tursiops truncatus). J Exp Psychol Gen, 124(4), 391-408.

• • • •

Judgment of pitch height: 2100 Hz or a lower frequency? The animal can refuse the test and is then assigned an easier one. Discrimination is excellent, but the animal specifically refuses difficu The animal can also show signs of hesitation (swims slowly, slowly shakes it precisely in the same frequency range.

Capuchins as a contrast case Beran et al.

Page 17

NIH-PA Author Manuscript NIH-PA Author Manuscript Figure 2.

Beran et al. (2009) J Exp Psychol Anim Behav Proc

Critique of opt-out studies Peter Carruthers: “…the animals will attend to, and notice, something about themselves (such as their own bodily feelings) in order to learn the cue-based rule in question. So it can appropriately be described as a form of uncertainty monitoring, even if the monitoring involved is not metarepresentational.” “Clever Hans”

Hampton criteria To meet the criteria for “private” (introspective) metacognition, we need to rule out that behaviour is driven by*:

• Response competition - opting out “by default” due to too much competition between response alternatives • Environmental cue associations - e.g. learning that the middle of the stimulus space maps to the opt-out response • Behavioural cue associations - e.g. monitoring one’s own response latencies as a cue

*Note Hampton allows for “public” metacognition in these scenarios - what Carruthers would label monitoring Hampton (2009) Comp Con Behav Rev

mals’ metacognitive abilities have almost all focused on

Post-decision wagering 990s David Smith and colleagues tested whether animals (dolphins

eir own uncertainty (Smith et al the 1995) during decision-making. Post-decision wagering presents metacognitive response after completion of the test trial, ruling out a “response competition” account

tainty monitoring”, “decline”, or “escape” tasks, we will refer to

Also allows computing of metacognitive ability (link between performance asks (Fig 1a). Animals are required and confidence on a trial-by-trial basis)to perform a primary decision

g tasks. (a) Schematic of opt-out task paradigms. Opt-out tasks eptual discrimination. On some proportion of trials, a third opt-out -out target results in a small but ensured reward. Participants utilizing ect the opt-out target more often on more difficult trials, and make Kepecs & Mainen (2012) Phil Trans B

Metacognition in monkeys

Anthe authentic demonstration of metacognition in animal demonstration of metacognition in the animal

rnell Son & Terrace, rnell, Terrace Psychological Science 2007

essment: k. aking

ct choice, propriate i t with 3

ect on the ibbles are

eft-hand ts choice, a single

Kornell Son & Terrace, Kornell, Terrace Psychological Science 2007

Primary Task NÂ° 2 = Shortterm memory : After viewing a series of 6 images, the animal must recognize in the subsequent series of 9 images, the one which also appeared in the preceding di series. i In a first stage, the animal must learn the task. task Then, the judgment of risktaking is added (over 20 seconds after the stimuli).

onse is val of the .

Area

Memory Kornell et al. (2007) Psych Science

Kornell et al. (2007) Psych Science

Comparat

Retrospective metacognition in monkeys Figure 2

Monkeys

evidence that the orb metacognition in rats d tory discrimination. T experiment were two tained more of compo inserting its head into p Once that rat entered sequence of its res responses or a TO fol brief interval whose du the anticipation inte neurons in the orbita on difficult than on ea the opposite pattern.

It was also shown that, rates of single cells to on trials on which s incorrectly. That diffe cells fired differentiall disparity was interpre w-Correlations. The value of w was calculated for the block of 10 days of subjectsâ&#x20AC;&#x2122; confidence ab training on the numerical and physical area perceptual tasks and for the showserial retrospective metacognitive accuracy it occurred before rew working memory task. Each value is significantly greater than zero tered. It does not follo (P < .05). Kornell et al. (2007) Psych Science

Prospective metacognition in monkeys

Monkeys were more accurate when they chose to accept the test than when they were forced to take the test - they adaptively decline the test when memory is poor Prospective response rules out â&#x20AC;&#x153;behavioural cue associationâ&#x20AC;? account Hamtpon (2001) PNAS

Neuron A

Decision Confidence and Orbitofrontal Cortex

Retrospective metacognition in rodents

Number of trials

Odor sampling (s)

(A) Fitting the computation ioral data. Two parameter First, the SD of the sens which Ta w Figure 1.distribution Postdecision(s), Wagering A correct reward Temporalmodel’s Wagerstrial by trial choic (A) Schematic the behavioral paradigm ond,of the opportunity cos trial, rats entered the central odor port confidence and reward da pseudorandom delay of 0.2–0.5 s, a m tion 6) was used to calcula odors was delivered. Rats responded by choice Estimating m “A” the left or Middle) right choice port, wherethe a dro psychometric curve; s was was delivered after a 0.5–8 s waiting correct decision (exponentially distribut A minimize the difference be 0.5-8s decay constant of 1.5 after a 0.5 sThe offse B reward omission psychometric curves. D C (catch trial) maximum). In a small fraction of correct choice estimate the confidence a als, water rewards were omitted. Trials o “B” in each The o odor-mixture ratiostrial. were (Right) randomly inter by minimizing dependentestimated of rats’ performance in the pr rat’s and animals the mod als. While the waiting for reward, wer to keep their snouts inside the choice Following the fitting, the pm error no reward was continuously using rat’s infrar closely monitored overlapped beams. Failure to break the photo-beam Experimental Procedures error. (B–D) Predictions thepsy mo (B) Behavioral performance of and STIMULUS PERCEPTUAL WAITING LEAVING DECISION example rat. thin Theline mr function offrom an example rat. Each SAMPLING DECISION PERIOD OR REWARD logistic fitpredictions (see Experimental about Procedure the rela behavioraldence, data collected in a single tes perceptual accura Dots represent behavioral performance B C D F G and trial outcome. The pr across all trials of all test sessions. T 100 0.5 120 closely match behavio line represents logistic fit the to the avera Waiting time thick lines represent the pE mance data shown with black dots. Water delivery represent (with ±SEM parameters across trials. optimize 50 0.3 60 (C) Odor curve sampling duration duratio and overall(the WT dist were sampling the odor before moving to data are shown as mean port) as a function of odor mixture con The model predicts that d example rat. Thin lines represent odor 0 0 0.1 accuracy (th duration inhave each higher of test sessions. Thick 0 5 10 15 20 50 80 20 50 80 model’s psychometric cur sents the data averaged across all trials Time (s) Odor mixture (% A) Odor mixture (% A) sessions. WT. Dark gray thick lin (D) The timing of reward delivery (blue, see Experimental Procedures) and the distribution of waiting times at the reward(defined ports of allas test sessions forpe on above 70th rat (black). Waiting times were measured for all the error trials and fraction of correct trials (i.e., reward omission trials). short W Lak etlineal. represents (2014) Neuron

Left choice (%)

Figure 3. Postdecision Follows Decision Confi

trials or across rats. Cannulae implantation (A) Schema had no effect on the decision accuracy (Figuro locations (C) Mean waiting times for control and mus See Figure conditions for the example rat (top) and ave (B) Decisio across Figure rats (bottom). 4. OFC Inactivation contrast Disrupts C fo (D) Psychometric functionsWaiting separated based dence-Dependent Time buto bined) and Accuracy in the Decision control and muscimol conditions fo (top) and a remains example intact (A) rat Schematic for cannulae implants and anato (top) and averaged across rats logistic fit locations of confirmed inactivation sites acros tom). Black and gray dots represent long WT (a See Figure S4 for examples of histology sectiI cedures). 70th percentile) and short WT (shorter than (B) Decision accuracy as a function of odor m trials or ac percentile) control trials, respectively. Red and contrast for control (saline and no injection had no effe bined) andlong muscimol conditions forpercentile the examp dots represent WT (above 70th (C) Mean th rats (bottom). (top) and averaged across Line short WT (shorter than 70 percentile) mus conditions logistic fits to the data (see Experimental trials, respectively. Lines represent logistic fit o cedures). In all images, error barsacross are ÂąSEM rata accuracy data (see Experimental Procedures trials or across rats. Cannulae implantation (D) Psycho (E) Mean normalized a function o had no effect onWT the plotted decisionas accuracy (Figur in the con mixture(C) contrast and trial forand contro Mean waiting timesoutcome for control mus example conditions the example ratTo (top)combine and aver averaged acrossforrats (bottom). tom). Black rats (bottom). across across different sessions of each rat and acros (D) Psychometric functions separated 70th based perceo normalized were used. For this normaliz is reduced actua B in theWTs control and muscimol conditions fo percentile) F the WTexample in eachrat trial was divided by mean WT (top) and averaged across rats dots acter repre trials oftom). the session Experimental Proced Black and(see gray dots represent long WT (a short2010 WT 70th linear percentile) WTAsterisks (shorter than Lines are fit toand theshort data. ind Proc trials, resp percentile) control trials, respectively. Red and significant differences (p < 0.05) between indi th dots represent long WT (above 70 percentile accuracy cally data points. Cannulae implantation itself hadd short WT (shorter than 70th percentile) mus Mean no 5D; p fect on trials, the WT pattern Lines (Figure S6). (E) See Figure respectively. represent logistic fit o mixture co and effect of muscimol onExperimental WT patterns in ratsw accuracy data (see Procedures averaged (E) Mean normalized WT plotted cannulae positioned outside OFC.as a function 5D, oa across diffe mixture accuracy contrast andastrial forofcontro (F) Decision a outcome function z-sp and averaged across rats (bottom). normalized To combine waiting time (see Experimental Procedure Fin across different sessions of each rat and acros the WT in controlnormalized and muscimol conditions for the exa WTs were used. For this normaliz ventr of the rat (top)theand ratstrials (bottom). WT averaged in each trial across was divided by mean WT rats, are session Proced D trials of the Lak et(see al. Experimental (2014)Lines Neuron

Retrospective metacognition in rodents A B C First-order performance E

Metacognitive accuracy A

D D

Muscimol inactivation of OFC

Higher-order process (but still implicit?) C

alsocietypublishing.org on April 16, 2012

Summary - nonverbal tests of metacognition

nal framework for confidence

dered (a) ted Opt-out in f can ithms e supas no comysical (b) w that Decline rvous option 2]. ect a ractience, cit or

A. Kepecs & Z. F. Mainen

1323

Postdecision wager confidence

choice choice

decline

Kepecs & Mainen (2012) Phil Trans B Figure 1. Behavioural tasks for studying confidence in ani-

Formal approach to the study of reflective awareness

Thinking about other minds

Until age of 4-5 years, children do not reliably pass this test

Wimmer & Perner (1983); Baron-Cohen et al. (1985)

Development of self-directed metacognition in infants

Decision

Confidence

â&#x20AC;&#x153;Put answer into smiley box when you have done well and want the experimenter to look at itâ&#x20AC;?

Fig. 1. Structure of the encoding and retrieval tasks. During the encoding task (a), children viewed a sequence of stimuli, half of which were presented once and half twice. During retrieval, children decided which of two drawings they had seen before (b), indicated how confident they were in their answer using a 3-point scale (c), and sorted the answers between two boxes (d) to indicate whether they wanted the experimenter to look at their answer (open-eyes box) or whether they did not want the experimenter to look at their answer (closed-eyes box).

size was selected to be comparable with previous investigations of metacognition in this age group (e.g., Lyons & Ghetti, 2011). Seventy-eight percent of children were European American, 6% were Asian American, 5% were Hispanic, 3% were African American, and 8% reported another ethnic background or did not report their eth-

Encoding task. At the beginning of each visit, children completed the encoding task, in which 30 black-andwhite line drawings were presented sequentially (Cycowicz, Friedman, Rothstein, & Snodgrass, 1997) on a touch-screen monitor (Fig. 1a). Stimuli were paired according to perceptual similarity (e.g.,Psych paint brush with Hembacher & Ghetti (2014) Science

Metacognition in infants 1772

Monitoring

Preschoolers’ Uncertainty and Decision Making Hembacher, Ghe

Control

Repeated and Accurately Identified Items Items Presented Once and Accurately Identified

2 Age group and item type 3-year-olds (n = 27) Accurately identified Inaccurately identifieda Overall 4-year-olds (n = 28) 1 Accurately identified Inaccurately identified Overall 5-year-olds (n = 26) Accurately identified Inaccurately identified 0 Overall

Mean Confidence

Table 1. Mean Proportion of Responses at Each Level of Confidence Open-Eyes Box

Closed-Eyes Box

Inaccurately Identified Items

3-Year-Olds

4-Year-Olds

5-Year-Olds

Age Group Fig. 2. Mean confidence score as a function of age group and item

3-Year-Olds

Confidence Low

Medium

High

.18 (.20) .16 (.22) .17 (.20)

.11 (.18) .15 (.22) .11 (.18)

.72 (.2 .69 (.3 .71 (.2

.07 (.09) .17 (.23) .08 (.10)

.10 (.14) .07 (.10) .10 (.12)

.83 (.2 .76 (.2 .82 (.2

.12 (.14) .40 (.35) .15 (.14)

.13 (.14) .21 (.27) .13 (.13)

.76 (.2 .40 (.3 .72 (.2

4-Year-Olds

5-Year-Olds

Note: Standard deviations are given in parentheses. a Age Group Incorrect responses were produced by only 23 of the 27 three-year olds.

NB no differences in first-order performance b with age type. Error bars represent standard errors. 2

group: 3-year-olds vs. 4-year-olds vs. 5-year-olds) × 3 (item

Hembacher & Ghetti (2014) Psych Scienceth SD = 0.39), F(1, 26) = 4,17, p = .05. Thus, although

Metacognition in infants N=80, 20 month old infants A

PSYCHOLOGICAL AND COGNITIVE SCIENCES

Fig. 1. (A) Experimental procedure. Infants watched as a toy was conspicuously hidden under one of two opaque boxes in full view (possible trials) or behind a curtain (impossible trials). For possible trials, the two boxes were then occluded behind the curtain for a variable delay (3, 6, 9, or 12 s). Then, infants were presented with the two boxes again and taught to indicate where they remembered the toy to be by pointing toward its location. The chosen box was then pushed forward for the infant to recover the toy in the case of a correct response, or discover that there was no toy in the case of an incorrect response.

Goupil et al. (2016) PNAS

Metacognition inEvidence infants A Toy location

Evidence

Delay

Toy location

Task difficutly

Delay

Pointing

Task difficutly

First-order decision

Decision co

Pointing

Searching

First-order decision

Decision confidence

3sec

12sec

6 5 4 3

Correct

6 5 4 3 2

Correct 0.9 0.8 0.7 0.6

Incorrect 0.5

0.9

Short 0.8 PTs Long PTs

Mean Accuracy

C Mean Persistence Time (seconds)

p(persistence | correct)

B p(persistence | correct)

C Mean Persistence Time (seconds)

0.7

18-month0.6 old infants 0.5 0.4

Incorrect 0.3 1 3 6 1 3 6 0.4 9 12 2 p(persistence | incorrect) Memorization delay (seconds) Memorizatio 0.3 0 1 3 6 9 12 0 1 3 6 9 12 Goupil & Kouider (2016) Curr Biol p(persistence | incorrect) Memorization delay (seconds)

3000 ms

Waiting Period 2500 ms

Cue C

Alternating Masks 2500 ms

Amplitude V

Fixation

y (1st Look)

Invisible 0.7 0.6

***

Correct

-8

-10

Incorrect

-12 -14

Visible Invisible Masked Face Visibility 0.9

Incorrect 0.8

Visible Invisible Masked Face Visibility

0 -5 -10 -15

Visible *

-6

Gaze Contingent

Visi

-4

ccuracy (1st Look) ean Accuracy

50 - 300 ms (6 levels)

-2

0.9

ShortPTs PTs Short Short PTs LongPTs PTs Long Long PTs

0.8 0 0.7

0.2

0.4 0.6 Seconds

0.8

Difference Correct - Incorrect

Reward

Correct

Amplitude V

Mean Persistence Time (seconds)

Mean Amplitude in Peak Window ( V)

Error monitoring in infants

5 0 -5 -10 -15

Figure 4. Experiment 3:0.6 Event-Related Potential Results

(A) Mean amplitude in peak window depending on first-order accuracy and visibility. Error bars sh (B) Scalp topographies showing 0.5 statistical significance maps (t values) of the difference between

Goupil & Kouider (2016) Curr Biol

Brain development over the lifespan

Myelination over the lifespan

The approximate myelination trajectory of the frontal lobe of normal human brains (there are large individual variations)

Yellow regions mature late during development

section (rostral). C there Can h be b correlation l i with i h more subtle b l changes h i the in h organization i i off the h brain b i in i normal subjects? Continued development of metacognition Here, subjects engage in a difficult psychophysical task (detecting a patch of a slightly hi h contrast higher t t between b t 2 screens ). ) This task is maintained close to the threshold to ensure an overall success rate of 71%. After each trial, participants report their degree of confidence in their first response .

Weil, Fleming et al. (2013) Consciousness & Cognition

Continued development of metacognition 268

L.G. Weil et al. / Consciousness and Cognition 22 (2013) 264–271

Fig. 2. Relationship between Aroc, age and sex. (A) Scatterplot illustrating the significant positive correlation between Aroc and age in adolescence (r = .38, p = .048). (B) Scatterplot illustrating the non-significant relationship between Aroc and age in adulthood (r = !.22, p = .25).

Meta-d’ (area under type 2 ROC) increases with age in adolescence Note no relationship between d’ and age - perceptual performance remains stable

There was no significant correlation between Aroc and task difficulty (mean stimulus contrast; Pearson’s r = !.09, p = .50) or Aroc and d0 (r = .21, p = .12). Broc,, a measure of the tendency to use higher or lower confidence ratings, was similarly analysed. Broc ranged between !5.68 and 1.94 (values < 0 corresponding to a bias towards lower confidence ratings, and values > 0 corresponding to a bias towards higher confidence ratings) and showed no correlation with task difficulty (mean contrast; r = !.05,&pCognition = .70), Weil, Fleming et al.stimulus (2013) Consciousness

Changes Predict Metamemory Monitoring Improvement over

Having provided evidence for cortical development over children, we examined whether these cortical changes ed change in metamemory monitoring. To this end, we ed whether initial thickness and change in thickness in OIs over time predicted improvements in metamemory ion, measured as the AUCROC2. In the left hemisphere, n with more pronounced insula thinning over time (b = , P < 0.05; Fig. 4A), as well as those children who showed ronounced increase in vmPFC thickness over time (b = P < 0.05; Fig. 4B), exhibited a higher rate of metamemory ring improvement over time. Change in the APFC (b = P = 0.63) or dACC (b = 0.017, P = 0.68) did not predict in metamemory resolution over time. Additional control s demonstrated that the rate of increase in metamemory ion was still related to change in the left vmPFC (b = 0.13, 02) and left anterior insula (b = −0.14, P = 0.01) after ling for recognition accuracy (SI Results). ionright andhemisphere, experimental design. (A) thinning Each unique parhe greater insular over time d as a predictor overall metamemory corresponding resolution at a a different row of with the participant’s evel (b = −0.090, P = 0.05) and was not related to the rate ircle) connected by a line. (B) During encoding, paramemory change over time. A model collapsing across nes divided into active or passive blocks. Active blocks heres revealed similar results as those results observed in modulate their hemisphere (SIattention: Results). Participants were instructed ether, demonstrate that cortical development theythese wereresults presented on a green background and to tal when subregions to improvements in children’s em theycontributes were presented on a red background. o introspect on their memory accuracy. However, the way participants were presented with a probe that was ch different cortical regions contribute to metamemory scenes or a novel scene, and were of asked to indicate ring depends on their unique pattern cortical change. sults suggest thatprevious the rate of increase in monitoring ability sented in the block. During passive blocks, ransition to adolescence is relatedon to ainsular and ted to view scenes presented blue thinning background ease in vmPFC thickness over time. e the direction of an arrow presented as a probe. viewed studied scenestofrom all attention nants of Metamemory Monitoring Intellectual Ability. condiMetay, passive; monitoring ability is thought to support the regulaattend scenes used as active probes were learning (1). Thus, we asked metamemory es, and indicated whether the whether scenes had been seen

lescents can effectively harness their metamemory judgments to terest (ROI control learning and memory retrieval actively, and whether these right anter findings extend to socially relevant contexts, which can influence adolescents’ risk taking and decision making (24). Critically, we been impli provide a demonstration of the cortical changes supporting metai memory improvements during development and definition their relation to intellectual development. sectional age differences in recognition inedaccuracy crossThe results of the present study shed light onto the neural point (T1) are reported elsewhere (17). Longitu ch mechanisms supporting the development ofyounger metamemory significant linearin(bthe = anterior 0.061, Pinsula < 0.05) and quad monitoring. Changes and vmPFC preable age di dicted over time. These brain P < improvements 0.05) effectsinofmetamemory time, demonstrating that rec that in all R regions have been implicated in metacognition by greater structural improved over time, but changes were and functional neuroimaging studies of young (25, 26) and and younger

Continued development of metacognition

with later in time (Fig. S1). eachtheother Furthermore, we assessed whether man cortical difficulty resulted in the expected differencesdei curacy. Across all children and timemonitoring points, accu Next, we condition was higher than in the ignore conditio chil 0.05, Bonferroni-corrected P (PBonfamong ) < 0.05], w higher than in the passive conditiontered (b = 0.050, at 9.6 0.05) (Fig. S1). Longitudinal improvements in and statisti tion were larger relative to the passive conditio Table 1 (in (b = 0.020, P < 0.05, PBonf = 0.09),left but hemisp not rela condition (P > 0.54). The passive and ignore c insula and differ from each other, (P > 0.13). No remaining Fig. 4. Results of models testing the relation betweenmultiple cortical thickness co significant at themonitoring P < 0.05 level (allPlots Psshow > 0.08). O change and metamemory improvement. interaction lower corti effects as predicted metamemory monitoring (y axis) over time (x axis) for accuracy improved across conditions over time three fixed values (lines) of change in cortical thickness in the left anterior sessment. T improvements condition. insula (A) and left vmPFCin (B).the Fixed attend values of cortical thickness change were

but not in analyses co For an initial assessment of age differences in m tical thinnin Fandakova et al. (2017) PNAS

chosen to demonstrate maximal increase in thickness (blue), no change in thickness (black), and maximal decrease in thickness (red) in the anterior Age Differences and Change in Metamemory Mon insula and vmPFC to visualize the interactions between time and cortical change. Error bars indicate maximum and minimum values.

ToM

Implicit/â&#x20AC;&#x153;coreâ&#x20AC;? mechanisms, early to develop

Mindreading