Acquisition of Complex Systemic Thinking: Mental Models of Evolution

Educational Research and Evaluation 20()4, Vol. 10. Nos. ^ . pp. 499-521

Acquisition of Complex Systemic Thinking: Mental Models of Evolution* Sylvia T. d'ApoIlonia', Elizabeth S. Charles^ and Gary M.

y

'Concordia University and Dawson College, Montreal. Que.. Canada, "Georgia histitute of Technology, College of Computing, and "*Concordia University, Monlreal, Que., Canada

ABSTRACT We investigated lhe impact of introducing college students to complex adaptive systems on their subsequent mental models of evolution compared to those of students taught in the same manner hut with no reference to complex systems. The students' mental models (derived from similarity ratings of 12 evolutionary terms using lhe pathfinder algorithm) were significantly similar to iheir teachers' mental models and were correlated to their performance on an essay on evolution. Introducing students lo complex systems facilitated their understanding of the mechanism of inheritance, the mechanism of evolution, and the role of chance in evolution.

MISCONCEPTIONS ABOUT EVOLUTION Researchers in science education have shown lhat, despite instruction, postsecondary students retain deep-rooted misconceptions in several scientific disciplines {Driver & Easley. 1978: Gardner, 1986; Griffiths & Grant, 1985; Hackling & Garnet, 1986; Halhoun & Hestenes, 1985). Although Darwin and Wallace's theory ot evolution by natural selection is a central theme within the discipline of Biology (American Association for the Advancement of Science.

*A draft of this paper was presented at the Annual Meeting of the American Educational Research Association. New Orleans. LA, April 20()2.

This research was supported in part by Social Sciences and Humanities Researeh Council of Canada and Programme d'aide a la recherche sur I'enseignement el I'apprentissage.

Address correspondence to; Sylvia T. d'Apoilonia, Concordia University and Dawson College, 3040 Sherbrooke Street W., Montreal, Que., Canada H3Z IA4. E-mail: sdapolloniaŽplace. dawsoncoUege.qc.ca l380-36n/04/104-6-499$16.00 Š Taylor & Francis Ltd.

500

SYLVIA T. D" APOLLONIA ET AL.

2000), many studies have shown that college and university students have persistent misconceptions about its characteristics (Brumby, 1984; DemastesSoutherland, Good, & Peebles, 1995).

Neo-Darwinism or the modem synthesis (of genetics and evolutionary biology) is traditionally taught in terms of four principles (Ohlsson & Blee, 1992). Firstly, there is pre-existing genetic variability iti the population which has been brought about by random mutations in individual genotypes (producing a qualitative change in an inheritable characteristic) and by the variation resulting from sexual reproduction. Secondly, environmental pressures lead to differential survival/reproduction of individuals. Thirdly, the combination of random mutations and a selection mechanism leads to changes in the population gene pool of successive generations (changes in allelic frequencies). Fourthly, over long periods of time, there is a change in the characteristics of the population such that it has become more adapted to the environment (more individuals will have the adaptive trait).

In traditional instruction, each postulant is explicitly taught and tested. In some cases, the role of these postulants in an evolutionary case study such as the evolution of DDT resistant mosquitoes or changes In the proportions of dark and light peppered moths is specifically described, ln addition, some biologists also teach that non-adaptive mechanisms such as genetic drift infiuence evolution (Gould, 1980).

Nevertheless, researchers have shown that students have persistent difficulties in connecting what they know of genetics to evolution (DemastesSoutherland, Good, Sundberg. & Dini, 1992; Hallden, 1988; Jensen & Finley.

1994). Bishop and Anderson (1990) have shown that students have difficulties attributing the origin of new traits to chance and their survival to natural selection. That is, they have difficulties distingui.shing qualitative changes to genotypes from quantitative changes in the proportion of individuals that carry the trait (Bishop & Anderson, 1990). That is, they have difficulties distinguishing what happens at the individual level from what happens at the population level (Hallden, 1988).'

Researchers have also suggested that non-domain-specific factors, such as religious and cultural beliefs (Matsumura, 1998), difficulties with the scientific meaning of common terms (e.g., evolutionary adaptation) (ZaimIdrissi, Desautels, & Larochelle, 1993), and inadequate textbooks (American

'This W;LS resolved through mathematical modeling by Godfrey Hardy, an English mathematician, and Wilhelm Weinberg, a German physician.

ACQUISITION OF COMPLEX SYSTEMIC THINKING

501

Association for Ehe Advancement of Science, 2000; McComas, 1997), may be a factor in students' persistent misconceptions.

This research is based on several theoretical perspectives, described below.

It is based on the assumptions that students use mental models or explanatory frameworks to organize knowledge and enhance its retrieval during problemsolving tasks. When these mental models are fragmented and/or inadequate, understanding is inhibited. Thus, one approach to enhancing students understanding of evolution is to help them acquire the appropriate conceptual framework as they construct their understanding of evolution. Thus, this research is based on ideas from the study of Complex Systems and from the misconception literature, specifically Michelene Chi's (1993) work.

COMPLEX SYSTEMS THINKING Many researchers (Anyang, 1998; Bar-Yam, 1997; Kaput, Bar-Yam, & Jacobson, 1999) have argued that the study of complex systems may provide learners with a unifying and cross-disciplinary framework that allows students to construct a betterunders tanding of many science phenomena. Complexsystemsarehierarchically organized collections of interacting components (called agents) operating under specified rules and resulting in the emergence of higher level associations which behave differently from what one could predict from knowledge of the agents (see also, Charles & d'ApoUonia, 2(X)3). For example, there are at least three levels (or associations) to contend with in understanding evolution: genes, individuals, and populations. Genes mutate, individuals survive and reproduce, and populations evolve. Students have difficulty in understanding how these three different levels interact and how a random event (mutation) at the gene level emerges as an evolutionary pattern at the population level.

Several researchers (Jacobson, 2000; Resnick, 1994, 1996; Resnick & Wilensky, 1997; Wilensky, 1999) have suggested that many of the difficulties students face in understanding such topics as ecology, evolution, predatorprey interactions may reflect confusion with such complex system properties as emergent levels, self organization, and the ro!e of chance on complex systems. Jacobson and his colleagues (Jacobson & Archodidou. 2000; Jacobson, Sugimoto, & Archididou, 1996) explored the relationship between students' complex systems thinking and their conceptual understanding of evolution. They identified four knowledge components^ (origin of new traits. correspond to the basic principles of evolution.

502

SYLVIA T. D" APOLLONIA ET AL.

mechanism of inheritance, mechanism of evolution, and final causes). For each knowledge component, they subsequently defined two stages in the learner's understanding (based on reductive "clock-work" mental models or based on "emergent" mental models). They were thus able to describe eight stages in learners' mental models of evolution from Naive to Neo-Darwinian based on their appropriate use of the knowledge components in their explanations of evolution. Although the results from Jacobson's study (Jacobson & Archodidou, 2000) were obtained from a small sample, they indicate that novice learners use "reductive" thinking in explaining evolution while experts use "emergent" thinking.

Thus, in this study, we introduced students in the experimental group to complex systems thinking as an organizing framework for the study of biology. We subsequently taught the aforementioned principles of evolution to both the experimental group and the control group. We hypothesize that, without prompting, students who have been exposed to complex systems thinking will subsequently construct better mental models of evolution than students taught biology along traditional lines and will therefore use emergent thinking in solving problems in evolution.

CHI'S CONCEPTUAL CHANGE THEORY Several researchers (Chi, 1993; Chi, Slotta, & deLeeuw, 1994; Ferrari & Chi, 1998; Slotta & Chi, 1999; Vosniadou, 1994) have proposed that learners use mental models or explanatory frameworks (general concepts used to interpret and explain the world) to "make sense" of many scientific phenomena. The basic assumption of their theory is that learners relate all new conceptions to ontological categories on the basis of the language used to describe the conceptions. These schema-like associations act as facilitators or inhibitors of future knowledge retrieval and problem solving. Chi and her coworkers (Chi el al.. 1994; Ferrari & Chi, 1998; Slotta & Chi, 1999) argue that learners use explanatory frameworks to organize and express their ontological commitments. When learners categorize new conceptions to inappropriate ontological categories, they cannot subsequently use it.

One such general ontological category is the category of ''processes".

Within this category there are both "event processes" and "emergent processes". Chi and her colleagues hypothesize that most misconceptions occur when concepts such as evolution are assigned to the "event process"

ACQUISITION OF COMPLEX SYSTEMIC THINKING

503

rather than to the "emergent process" category. Stotta and Chi (1999) argue that once a concept has been assigned to the wrong category, it is very difficult to change. Reassignment may entail the leaming of a new explanatory framework. This may require learning new terminology, acquiring new mental models, and developing different attitudes and values.

Representational Models of Learning and Memory Traditional constructivist theorists (Ausubel, 1963, 1968; Novak, 1988; Piaget, 1954) hold a representational model of memory in which domainspecific declarative knowledge is stored as a network in long-term memory (e.g., Anderson, 1983). These network models assume that concepts are stored as nodes that are interconnected to form a vast associative network. Most researchers believe that the nodes are organized in a hierarchical manner such that more general concepts iire superordinate and more specific concepts are subordinate. In associative models, the links are unlabelled and therefore the same. Propositional network models (Anderson, 1983; Anderson & Bower, 1973) hold that propositions (a combination of concepts having both a subject and a predicate that have a truth value, e.g., water is a molecule) rather than unitary concepts (e.g., water) form the nodes. The links are labelled and therefore are not identical.

However, understanding science involves not only knowing what (declarative knowledge) but also knowing how (procedural knowledge).

Therefore, representational models propose that procedural rules on how to manipulate the nodes are also stored (Anderson, 1983). Procedural rules for general situations are stored as schemata (Rumelhart, 1980) while procedural rules for specific subject-matter domains are stored as mental models (Johnson-Laird, 1983; cf. Gentner & Stevens, 1983). Thus, mental models are analogies of the events or situations along with procedural rules to mentally manipulate the event or situation. Individuals use these models lo predict future events, answer comprehension questions, or solve problems. Initially, individuals construct conceptual structures lhat include only declarative knowledge; however, under appropriate conditions, learners downplay these semantic features and construct mental models of the situation, by encoding procedures, goals, and relationships (McNamara, Miller, & Bransford, 1991). Whether a learner opts to encode text propositionally or construct a mental model appears to be a function of text features, task difficulty, expertise, and knowledge of subsequent testing procedures (McNamara et al., 1991).

504

SYLVIAT. D'APOLLONIA ETAL.

Assessment of Mental Models and Associated Conceptual Structures Methods of inferring the way in which people organize domain-specific information are very diverse (Adelson, 1981; Egan & Schwartz, 1979; Murphy & Wright, 1984; Shavelson, Ruiz-Primo, & Wiley, in press; Shoenfeld & Herrmann, 1982). However, all are introspective, beginning with individuals making metacognitive judgements of what they know. These methods can be categorized as verbal reports, clustering methodologies, and scaling techniques (Koubek & Mountjoy, 1991). Each method involves three stages: collecting the declarative knowledge that has been acquired, generating a representation of the conceptual structure, and quantifying the degree of conceptual organization.

Concepts and relationships can be elicited by essays, concept maps, ordered lists, and pair-wise similarity ratings such as described in Pathfinder (Schaneveldt, 1990). In this study, we elicited students' concepts about evolution by both asking them to write an essay and by asking them to rate the degree of relationship between 12 pairs of evolutionary terms.

Cognitive psychologists have developed several methods of generating representations of the way in which declarative knowledge is structured (Olson & Biolsi, 1991), Firstly, the subjects' verbal protocols can be parsed into propositions and the knowledge structure represented as a network of labeled interconnected nodes. (d'Apoilonia, De Simone, Dedic, Rosenfield, & Glashan, 1993; Frederiksen & Breaueux, 1990; Mosenthal & Kirsch, 1992). Secondly, written or spoken text can be converted into concept maps.

For example, Novak and Musonada (1991) interviewed students on their understanding of chemistry, converted their protocols into concept maps, and subsequently assessed the maps. Thirdly, subjects' performances on multiple choice tests can be coded into historical or developmental stages in the theoretical development of the domain (Jacobson & Archodidou, 2000; Vosniadou & Brewer, 1987). Finally, concept maps, similarity ratings, and other such data can be transformed into general weighted networks such as Pfnets by Pathfinder (Schaneveldt, 1990) or by multidimensional scaling.

The Pathfinder algorithm generates networks in which the links may be either directed or non-directed. The Pathfinder algorithm also generates several measures of coherence and network similarity: the coherence or consistency of the generated network, the similarity between two networks, a test for the probability that the number of links in common between two links could arise by chance, and the "information" that two networks have in common.

ACQUISITION OF COMPLEX SYSTEMIC THTNKING

505

Thus, several methods of generating and analyzing conceptual structures have been developed. They are all based on representational models of memory. They all maintain that conceptual structures are symbolic internal representations of extemal reality stored in long-term memory. They are based on strong assumptions that conceptual structures are relatively stable (once leamed) and meaningful (have semantic properties), and can be inferred from an individual's overt behaviour.

GOALS OF REPORTED STUDY The basic objectives of our research described in this article are threefold: 1. To demonstrate that rating scales capture experts' mental representations of evolution and that these measures have face validity.

2. To demonstrate that teaching students about complex adaptive systems influences their mental representations of evolution.

3. To demonstrate that students who were first taught about complex adaptive systems would subsequently have a better understanding of evolution.

RESEARCH DESIGN AND METHODOLOGY Participants and Intervention One class of 37 college students enrolled in a lst-year biology course was taught complex adaptive systems during the 1st week of the course. We emphasized the hierarchical nature of biological systems, the emergent properties of life, and the role of chance and probability. The concepts were reinforced periodically by asking students to relate a biology topic (e.g., cell structure, cell division, genetics) to complex adaptive systems (see Appendix A for summary of instruction and selected students' assignments). Six weeks later they were taught principles of evolution using a traditional pedagogy.

The same teacher taught principles of evolution using only the traditional pedagogy to another group of 36 students. Thus, the only difference in instruction between the two groups was that students in the experimental group were taught about complex systems and encouraged to develop this as an explanatory framework for biological processes. Students subsequently reviewed evolution in light of genetics and were asked to complete two tasks.

They were asked to rate the degree of relatedness among all pairs of 12

506

SYLVIA T. D'APOLLONIA ET AL.

evolutionary terms (adaptive (natural) selection, chance, change in allelic frequency, differential survival, environmental pressure, genetic drift, genetic variation, genotype, individual, mutation, non-adaptive selection, populalion).

We selected these terms on the basis of the major headings in the students' text book (Campbell, Reece, Mitchell, & Taylor, 2003) and the results of prior pilot tests of the rating scale. They were subsequently asked to write an inclass essay in which they explained how whales may have evolved 60 million years ago from the ancestors of ungulates (sheep, cows, etc.), a previously unseen evolutionary case study.''

Deriving Conceptnal Strnctures From Similarity Ratings We used the Pathfinder software, MacKnot (Interlinks) to (a) generate Pfnets for each student. These Pfnets represent the students' overall structural organization of evolutionary concepts; (b) generate graphs depicting the multidimensional distances among the terms; and (c) generate composite'* Pfnets for four domain experts, 30 students in the experimental class, and 20 students in the control class. See Schaneveldt (1990) for a discussion of the mathematics that underlies the Pathfinder algorithms.

Deriving Conceptual Structures From Essays The students' essays were segmented into propositions. The investigators prepared a template that was used to classify the idea units onto the 12 evolutionary terms. For example, when students indicated that sotne organisms would survive, while others would not, we substituted diffetential survival. Each proposition was subsequently "translated" into propositions containing the 12 evolutionary terms.

We subsequently coded each proposition (if relevant) on the basis of the student's understanding of evolutionary concepts using a coding schema (presented in Table 1), modified from Jacobson and Archodidou (2000). We coded the frequency of statements a student made, at each level (novice ( — 1) or expert (+1)), for each Knowledge Component. The students' understanding of evolutionary concepts is the sum of the four frequencies.

We subsequently coded each statement (if relevant) on the basis of the student's understanding of the "emergent" nature of evolutionary processes. ^See (http://www.nalLnr.corn/nsu/OI0920/OI0920-l l.html). "^The MacKnot software incltides a program which computes the average proximity matrix across several proximity matrices and thas generates a mental representation of the "group's understanding" of evolution.

ACQUISITION OF COMPLEX SYSTEMIC THTNKING

507

Table 1. Coding Schema Used to Score Students' Understanding of Knowledge Components of Evolution.

Levels of understanding and mental model (E is expert. N is novice)

Knowledge component Origin of New Traits

Prior mutations Environmental pressure

E

Mechanism of Inheritance

Transmission of genes from parent to offspring Confusion (acquired traits, or other transmission)

E N

Mechanism of Evolution

Differential survival —* change in allelic frequencies E Gradual developmental of individuals M

Role of Chance in Origin of New Traits

Probabilistic Teleoiogical/deterministic

M N

Table 2. Coding Schema for Students' Understanding of the Emergent Nature of Evolutionary Processes {E is expert. N is novice).

Processes

Genetic Transmission Mutation Differential Survival Selection Environment Change in gene pool Genetic variability Evolution

Hierarchical levels Ind

Pop

E

N

N

E

Processes

Genetic Transmission Mutation Genetic Drift

Differential Survival Natural Selection

Final Cause Prob

Dir.

E

N

N

E

That is, we coded the frequency of statements a student made, {novice ( —l)or expert ( + 1)), on the basis of the hierarchical level to which students situated evolutionary proeesses and on the role of chance {probabilistic causes) versus direction (teleological causes) (see Table 2). The students" understanding of the "emergent" nature of evolutionary processes is the sum of the two frequencies. Repeated statements were counted more than once, only if the contexts were different. For example, if the probabilistic nature of evolutionary proeesses was described for mutations and for genetic drift it

508

SYLVIA T D'APOLLONIA ET AL.

was counted twice. However, if the probabilistic nature of evolutionary processes was described twice for mutations., it was counted once.

The inter-rater reliability (^agreement) over 20% of the essays for each category was the following: Origin of New Traits {100%), Mechanism of Inheritance {100%), Mechanism of Evolution (75%), Role of Chance {75%), Emergent Levels {75%), Final Cause (75%). In all cases, the differences were in the "strength" of the coded relationship, not the direction. That is. students sometimes repeated a statement and coders sometimes coded it twice rather than once. See Appendix B, for the data generated by one student in the experimental group.

RESULTS AND DISCUSSION The results of this research are organized around three questions: Do rating scales (as analyzed by the Pathfinder algorithm) capture metital tnodels?

Are these measures setisitive to different instrtiction? Does expostire to the explanatoiy framework of complex systems facilitate the acquisition of appropriate mental models of evolution and enhance the understanding of evolution ?

Do Rating Scales Capture Experts^ Mental Representations of Evolution?

The collective experts' mental model of evolution derived from the composite Pfnets is presented in Figure 1. It clearly indicates that the domain experts have neo-Darwinian mental models on the Origin of New Traits {mutations affecting genotype of individuals), on the Mechanism of Evolution {environmental pressure on individual related to differential survival resulting in natural selection changing the population gene pool), and on Role of Chance {random forces affect both mutation and changes in gene pool). Unfortunately, the selected terms do not "capture" the Mechanisms of Inheritance. In future studies, we will add terms to capture this knowledge component.

The experts' composite Pfnet was analyzed using multidimensional scaling. The 12 ternis were clearly separated into four clusters on the basis of two dimensions: levels (population/individual) and role of chance {probabilistic/directional). Thus, it appears that the Pfnets determined from rating scales do capture the relevant dimensions of evolution. Thus, rating scales can be used to capture the mental models of evolution.

ACQUISITION OF COMPLEX SYSTEMIC THINKING

509

Fig. I. Compo.site Pfnets from four experts.

Does Teaching Students At>out Complex Systems Influence Their Subsequent Mental Representations of Evolution?

Figures 2a and 2b shows the composite mental models of 30 students in the experimental group and 20 students in the control group derived from the composite Pfnets.

The two mental models derived from the composite Pfnets are surprisingly similar. They differ in only 6 relationships. The two models differ in the role of chance, with the students in the experimental class putting more weight on its role in genetic drift and non-adaptive selection; while the students in the control class put more weight on its role in mutation. The two models also differ on the role of differential survival, natural selection, and genetic variability. The students in the experimental class may view differential survival affecting genetic variation through the mediation of natural selection; while the students in the control class may view genetic variation causing differential survival resulting in natural selection. These are tentative speculations on how the two models may differ.

What is striking is the degree to which they are alike. Both classes appear to have taken the same constructs from their teacher's explanation. This may reflect the powerful impact that a teacher's mental model has on student leaming. Both groups of students view the impact of the environmental

510

SYLVIA T. D'APOLLONIA HT AL.

Fig. 2a. Composite Pfnet from students taught Complex Adaptive Systems. The dotted lines are the links missing relative to the experLs' Pfnets. The bold dashed lines are the links added by the students.

Fig. 2b. Composite Pfnet from control students. The dotted lines are the link.s missing relative to the experts' Pfnets. The bold dashed lines are the links added by the students.

ACQUISITION OF COMPLEX SYSTEMIC THINKING

511

Table 3. Charactenstics of Ptnets From Experts and Students.

Group

Expert.s Students in control group Students in experimental group

Sample size

4 20 30

Coherence

Number of links

Mean

SD

Mean

SD

.38 .19 .32

.15 .30 .21

21 26 22

6.7 5.7 7.9

pressure to be at the population level rather than at the individual. Similarly, both groups of students relate the impact of mutations and genotype on genetic variability of the population to be direct rather than mediated by individuals.

That is, they do not appear to have the concept of the emergence of population characteristics from activities at the individual levels.

Table 3 compares characteristics of the mental models of experts, students introduced to complex adaptive systems before being taught evolution, and students taught evolution directly. The experts' mental models of evolution are more coherent and have fewer links than did those of students either in the experimental or control group. The mental models of students introduced to complex adapted systems appeared to be more similar to those of the experts.

However, there was a high degree of variability in individual mental models.

Therefore, we were nol able to show that tbese differences were significant at an a level of 0.05. However, there is a trend that as expertise develops, learners begin distinguishing clusters of related terms, rather than considering all the terms to be related. The students tbat produced the worse essays on evolution had mental models in which the link to node ratios were extremely high.

All tbe experts' mental models were significantly similar (t prob p < .05) to that of the researcher who taught the experimental and control classes.

Similarly. 63.3% (19/30) oftbe students in tbe experimental group had mental models of evolution significantly similar to that of the teacher's; but only 40% (8/20) of the students in the control class did.

Table 4 shows the degree of similarity (measured as tbe information content (â&#x20AC;&#x201D;log? pdue to chance)) between the teacher's mental model and those of the students in the control and experimental classes. The students who had been introduced to complex adaptive systems constructed mental models of evolution that were significantly (F = 6.66. df = 48,1. p = .Ol) more similar to

512

SYLVIA T. D'APOLLONIA ET AL.

Table 4. Similarity (Infonnation) Between Teacher's Mental Models, and Students'.

Sample size

Treatment

20 30

Control students Experimental students

Similarity Mean

SD

4.13 6.73

2.86 3.86

the teacher's mental model than were the students not introduced to complex adaptive systems. Thus, the introduction of complex adaptive systems affects students' mental models.

Does Teaching Students Complex Adaptive Systems Facilitate Their Subsequent Understanding of Evolution?

We first analyzed the students' essays in order to determine whether they "picked up" the same concepts captured by the derived mental models (i.e., their Pfnets). Subsequently, we analyzed the coded essays to determine whether students who had been introduced to complex adaptive systems have a better understanding of evolution than do the students not introduced to complex adaptive systems.

Table 5 shows the correlations between each students' mental model and their scores on the essay. The derived mental models are correlated to the students' level of understanding of three of the four evolutionary concepts (origin of new traits, mechanism of inheritance, mechanism of evolution) and one out of two of the emergent properties of evolution (hierarchical levels).

Students who were first introduced to complex systems had a significantly better (F^ â&#x20AC;&#x201D;5.02; df â&#x20AC;&#x201D; 2,47; p = .01) understanding of both evolutionary concepts and the emergent properties of evolutionary processes (Means = 3.57 Table 5, Correlation Between Students' Derived Mental Models (Similarity Index to the Teacher's Mental Model) and Their Scores on the Essays.

r

Origin of new traits

Mechanism of inheritance

Mechanism of evolution

Role of chance

Levels

Final cause

.31*

.32*

.45*

-.04

.31*

.03

Note. * Significant at the .05 alpha level.

Using multivariate tests of significance.

ACQUISITION OF COMPLEX SYSTEMIC THINKING

513

and 3.43, respectively) than did the control students (Means = .59 and 1.34, respectively). When the subcomponents were subsequently analyzed, significant differences were found for the origin of new traits, the mechanism of inheritance, the mechanism of evolution, and on the emergent levels of evolution. Thus, the intervention enhanced students' understanding of the mechatiisms of evolution but not of the role of chance in these mechanisms.

This may refiect either problems with coding the essays or the difficulties inherent in understanding the role of chance in the emergence of systems.

GENERAL DISCUSSION AND CONCLUSIONS Similarity ratings appear to be an effective and valid method of capturing the knowledge representations (mental models) held by both students and teachers.

The composite expert's derived mental models are interpretable, corresponding to a neo-Darwinian model of evolution. When the experts' similarity ratings were analyzed using multidimensional scaling, two dimensions (the role of chance, and emergent levels) separated the concepts into four clusters.

Students' mental models were similar to their teachers' mental models. Thus, the Pfnets captured the underlying organizational structures held by the teachers. Although the teachers' mental models were significantly similar, there were indications that the newer teachers' mental models more closely resembled the mental models imbedded in textbooks.

The knowledge representations were sensitive to an instructional strategy (the introduction to the students of complex adaptive systems as an organization framework in biology). Students who had been introduced to complex systems had derived mental models more similar to the teacher's than did students not introduced to complex systems. Moreover, their mental models predicted their performance on a writing task in which they were asked to explain the evolution of whales.

Thus, we concluded that the collection of similarity ratings of 12 evolutionary terms from students had elicited information on their conceptual structures and that the Pathfinder technique was an effective and valid method of representing and analyzing their mental models.

The selection of appropriate terms (both the number and the terms themselves) is crucial to obtaining reliable and valid mental models. We pilot tested several sets of terms before deciding on the present set. However, the knowledge representations lack certain ideas (and misconceptions). For

514

SYLVU T D'APOLLONIA ET AL.

example, they do not capture the misconception that tnany students stated that ungulates slowly develop traits that "adapt" them lo an aquatic environment.

Thus, in future studies we will add additional terms (to capture these misconceptions) and ask students to add their own terms.

Introducing students to complex adaptive systems prior to the traditional teaching of evolution, facilitated their subsequent understanding of evolution.

These students had a more profound understanding of the source of new traits, on the mechanism of inheritance, and on the mechanism of evolution.

However, they did not acquire a better understanding of the role of chance in evolution than did the control students. This tnay reflect the brevity of the intervention. We only introduced students to complex systems for one lecture and gave them one reinforcement assignment. In subsequent research we will increase the amount of time that students engage in interactive activities (simulations and discussion) that etnphasize probabilistic reasoning. We will also lest their understanding both of complex systems and of other phenomena that are based on complex systems, for example, predator-prey relationships.

Mental models of evolution are like mental maps Ihat students may use when they attempt to explain a phenomenon. There are several routes that they can take between many concepts. For example, they can make a direct link from mutation to genetic variability in a population, or they can link the two indirectly via the influence of mutations on an individual. Having been introduced to the ideas of complex adaptive systems rnay provide an "incentive" for them to take the longer and correct route rather than the shorter and incorrect route. In other words, the idea from complex system that stable characteristics of a collective could emerge from the "random" activities of agents can act as scaffolding for their evolutionary explanations.

Thus, the intervention may have helped students subsequently understand evolution, without their necessarily understanding fully (or even marginally) complex systems.

In conclusion, we found that the Pathfinder technique of eliciting students' and teachers' conceptual stmctures, representing them as pfnets, and analyzing their similarity was an effective method of deriving their mental models of evolution. The teachers' mental models strongly influenced their students' mental models. The similarity of the students' mental models to that of their teacher predicted their performance in writing an explanation of the evolution of whales. Introducing students to adaptive systems, stimulated their understanding of the mechanism of inheritance, the mechanism of evolution, and the role of chance in evolution.

ACQUISITION OF COMPLEX SYSTEMIC THINKING

515

REFERENCES Adeison, B. (1981). Problem solving and the developmenl of absiracl categories in programming languages. Memory and Cognition. 9(4), 422-433.

American Association for ihe Advancement of Science. (2000). http://www.project2061.org/ research/textbook/hsbio/derault.him Anderson, J.R. (I9K3). Problem solving and learning. American Psycholo^^isf, 48. 35-44.

Anderson, J.R., & Bower, G.H. (1973) Human associative memory. Washington, DC: Winston.

Ausubel, D.R (1963). The psychology of meaningful verbal leaming. New York: Grune & Stratton.

Ausubel. D.R (1968). Educational psychology: A cognitive view. New York: Rinehart and Winston.

Auyang. S. (1998). Foundations of complex-system theories in economics, evolutionary biology, und stalistical phy.sics. Cambridge. UK: Cambridge University Press.

Bar-Yam, Y. (1997). Dynamics of complex systems. Reading. MA: Addison-Wesley.

Bishop, B.A., & Anderson, C.W. (1990). Student conceptions of nalural selection and its role in evolution. Journal of Research in Science Teaching, 27, 415-427.

Brumby. M.N. (1984). Misconceptions about the concept of natural selection by medical biology students. Science Education. 68, 493-503.

Campbell, N.A.. Reece. J.B.. Milchell. L.G., & Taylor, M.R. (2003). Biology: Concepts ami connections (4th ed.). San Francisco: Benjamin Cummings.

Charles, E.S.. & d'Apoilonia, S.T. (2003). A systems approach to science education. Final Report submitted to Programme d"Aide a la recherche sur I'enseignement et I'apprentissage (PAREA). Gouvemement du Quebec. ISBN 1-55016-169-5.

Chi, M.T.H. (1993). Barriers to conceptual change in learning science conccpt.s: A theoretical conjecture, ln W. Kintsch (Ed.), Proceedings of the Fifteenth Annual Cognitive Science Society Conference (pp. 312-317). Hillsdale, NJ: Erlbaum.

Chi, M.T.H., Slotta, J.D., & deLeeuw, N. (1994). From things to proces.ses: A theory of conceptual change for leaming science concepts. Learning and Instruction. 4. 27-43. d'Apollonia, S.. De Simone. C . Dedic. H.. Rosenfieid, S.. & Glashan. A. (1993). Cooperative netH'orking: A method of promoting understanding in the .sciences. Final Report submitted to Programme d'Aide a la rescherche sur I'enseignement et I'apprentissage (PAREA). Gouvemement du Quebec. ISBN 0-969728-1-0.

Demastes-Southerland, S., Good. R.G.. & Peebles, P. (1995). Students' conceptual ecologies and the process of conceptual change in evolution. Science Education, 79. 637-666.

Demastes-Southerland, S.. Good, R.G., Sundberg. M.. & Dini, M. (1992. March). Students' conceptions of natural selection and its role in evolution: A replication study and more.

Presented at the Annual Meeting of the National Association for Research in Science Teaching. Boston, MA. Driver, R., & Easley, J. (1978). Pupils and paiadigms: A review of literature related to concept development in adolescent science students. Studies in Science Education, 5, 61-84.

Egan, D.E., & Schwartz. B.J. (1979). Chunking in recall of symbolic drawings. Memory and Cognition, 7. 149-158.

Ferrari. M.. & Chi, M.T.H. (1998). The nature of naive explanations of natural selection.

International Journal of Science Education, 20., 1231-1256.

516

SYLVIA T. D'APOLLONIA ET AL.

Frederiksen. C.H., & Breaueux. A. (1990), Applying cognitive task analysis and research methods to assessmeni. In N. Frederiksen. R. Glaser. A. Lesgold. & M.G. Shafto (Eds.), Diagnostic monitoring of skill and knowledge acquisition (pp. 143-165). Hillsdale. NJ: Lawrence Eribaum.

Gardner, P.L. (1986). Physics student's comprehension of motion with constant velocity.

The Austtalian Science Teachers Journal, 31. 27-32.

Gentner, D., & Stevens, A. (Eds.). (1983). Mental models. Hillsdale. NJ: Eribaum.

Gould. S.J. (1980). Is a new and general theory of evolution emerging? Paleohioiogy, 6, 119-130.

Griffiths. A.K.. & Grant, B.A.C. (1985). High school students' understanding of food webs: Identification of a learning hierarchy and related misconceptions. Journal of Research in Science Teaching. 22, 421 -436.

Hackling, M.W., & Garnet. P.G. (1986). Chemical equilibrium: Learning difficulties and teaching strategies. The Australian Science Teachers Journal, i7, 8-13.

Halhoun, I.A.. & Hestenes. D. (1985). The initial knowledge state of college physics students.

American Joumal of Physics, 5.?, 1056-1065.

Hallden, O. (1988). The evolution of the species: Pupil perspectives and school perspectives.

International Journal of Science Education, 10. 541-552.

Jacobson. MJ. (2000. April). Butterflies, traffic jams, & cheetahs: Probletn solving and complex systems. Paper presented at the Annual Meeting of lhe American Educational Research Association. Atlanta.

Jacobson. M.J.. & Archodidou. A. (2000). The design of hypermedia tools for leaming: Fostering conceptual change and transfer of complex scientific knowledge. The Journal ofthe Learning Scietices, 9(2). 149-199.

Jacobson, M.J.. Sugimoto. A.. & Archididou. A, (1996). Evolution, hypermedia leaming environments, and conceptual change: A preliminary report. In D.C. Edelson & E.A. Domeshek (Eds.). Intematiotial Conferetue on the Learning Sciences. 1996: Proceedings of ICLS 96 (pp. 151-158). Charlottesville. VA: Association for the Advancement of Computing in Education.

Jensen. M.S., & Pinley, F.N. (1994). Changes in students' understanding of evolution resulting from different curricular and in.^Hructional strategies. Paper presented at the Annual Meeting of the National Association of Research in Science Teaching. Anbcim, CA. Johnson-Laird, P.N. (1983). Mental models. Cambridge. MA: Harvard University Press.

Kaput, J.. Bar-Yam, Y. & Jacobson. M. (1999). Planning documents fnr a national itiitiative on complex systems in K-16 education (Report to the National Science Foundation).

Cambridge, MA: New England Complex Systems Institute. Online source: http:// necsci.org/events/cxedl6.html Koubek, R.J., & Mountjoy, D.N. (1991). Toward a model of knowledge structure atid a comparative analysis of knowledge structure tneasurement techniques. ERIC_NO: ED339719 Matsumura. M. (1998). Is it fair to teach evolution.' Reports of the National Center for Science Edtwation. iH. 19-21.

McComas. W.F. (1997). The discovery and nature of evolution by natural selection: Misconceptions atid lessons from the history of science. Ametican Biotogv Teacher.

59, 492-.^0().

McNamara. TP., Miller. D.L.. & Bransford. J.D. (1991). Mental models and reading comprehension. In R. BaiT. M. Kamil, P. Mosenthal. & PD. Pearson (Eds.). Handbook of research in reading (pp. 490-511). New York, NY: Longman.

ACQUISITION OP COMPLEX SYSTEMIC THINKING

5 17

Mosenthal, P.B.. & KJrsch, I.S. (1992). Leaming from exposition: Understanding knowledge acquisition from a knowledge model perspective. Jnttmal of Reading, 35, 356-370.

Murphy. G.L.. & Wright. J.C. (1984), Changes in conceptual stmcture with expertise: Differences between real-world experts and novices. Joumal of Experimental Psychology: Learning. Memoiy. and Cognition, 10, 144â&#x20AC;&#x201D;155.

Novak, J.D. (1988), Leaming science and the science of Ieaming. Studies in Science Education, /5. 77-101.

Novak, J.D., & Musonada, D. (1991). A twelve-year longitudinal study of science concept leaming. Atnerican Educational Research Joumai 28, 117-153.

Ohisson, S., & Blee, N.V. (1992). The effect of expository text on childrens' explanations of biological evolution. OERI Report. Santa Cmz, CA: Learning and Development Center.

Olson, J.R., & Bioisi. K.J. (1991). Technique for representing expert knowledge. In K.A. Ericsson & J. Smith (Eds.), Toward a general theory of expertise (pp. 240-285).

Cambridge: Cambridge University Press.

Piaget, J. (1954). The con.strtiction of reality in the child. New York: Basic Books.

Resnick. M. (1994). Turtles, termites and traffic jams: Explorations in massively parallel microworlds. Cambridge, MA: MIT Press.

Resnick, M. (1996). Beyond the centralized mindset. Joumal ofthe Leaming Sciences, 5(1), 1-22.

Resnick, M.. & Witensky, U. (1997). Diving into complexity: Developing probabilistic decentralized thinking through role-playing activities. Joumal ofthe Leaming Sciences, 7(2). 153-172.

Rumelhart, D.E. (1980). Schemata: Tbe building blocks of cognition. In RJ. Spiro. B.C. Bruce. & W.F. Brewer (Eds.), Theoretical issues in reading comprehension (pp. 33-58).

Hillsdale, NJ: Eribaum.

Schaneveldt. R.W. (1990). Pathfinder associative networks: Studies in knowledge organization.

Norwood, NJ: Ablex.

Shavelson, R.J.. Ruiz-Primo, M.A., & Wiley, E. (in press). Windows into the mind.

Intemational Joumal of Higher Education.

Shoenfeld, A.H.. & Herrmann. D.J. (1982). Problem perception and knowledge structure in expert and novice mathematical problem solvers. Joumal of Experitnental Psychology: Learning, Memory, and Cognition, H, 484â&#x20AC;&#x201D;494.

Slotta, J.D., & Chi, M.T.H. (1999). Overcoming robust misconceptions through ontological training. Paper presented at the Annual Meeting of the American Educational Research Association, Montreal.

Vosniadou. S. (1994). Capturing and modeling the process of conceptual change. Learning and In.struction, 4, 45-69.

Vosniadou, S.. & Brewer, W.F. (1987). Theories of knowledge restructuring in development.

Review of Educational Research. 57, 51-67.

Wilensky, U. (1999). GasLab: An extensible modeling toolkit for connecting micro- and macro-properties of gases. In W. Feurzeig& N. RoheritA (Bda.). Modeling and simulation in .science and mathematics education (pp. 151-178). New York: Springer-Veriag.

Zaim-Idrissi, K.. Desauteis. J., & Larochelle, M. (1993). The map is the territory! The viewpoints of biology students on the theory of evolution. The Alberta Journal of Educational Research, 39, 59-72.

518

SYLVIA T. D'APOLLONIA ET AL.

APPENDIX A The introduction to Complex Systems consisted of one, 75 min, class in which Complex Adaptive Systems was defined. The following characteristics of complex adaptive systems were described and then students were asked to come up with examples of these characteristics.

• Simple agents aggregate to make more complex components or systems which usually have a hierarchical structure.

• Agents can be reused as components of many different structures (modularity).

• There is variety among the components.

• The components maintain their identity.

• The more complex components emerge dynamically from the random actions of the simpler agents.

• The flow of information and feedback maintains the individuality of the components and the functionality of the system.

• There is a selection mechanism whereby the most "suitable" components and systems survive and contribute to future systems.

• Tbe relationships among the components are non-linear and change over time.

The students were then asked to discuss online why complex adaptive systems are relevant to biology. Below are some of their comments: The properties of Complex Adaptive Systems (CAS) are relevant to biology because it involves components interacting with other components that adapt to their environment. Like in all living systems, complexity refers to the presence of hierarchical level. That is, the components are formed by depending on those below them. When studying biology, we know that we have the hierarchy which involves: ecosystem-community-populationOrganism-Organ-System-Organ-Tissue-Cell-Molecule. I believe that if we understand this hierarchical system we should be able to understand Complex Adaptive Systems. Basically what I am trying to say is that CAS is relevant to Biology as it gives knowledge about components and their adaptation to the environment.

To understand how Complex Adaptive Systems are relevant to Biology we must first understand what they are composed of. A Complex Adaptive System is composed of various levels of hierarchy. Each level is made up of the components ofthe level below: however, the behaviour ofthe higher level can

ACQUISITION OF COMPLEX SYSTEMIC THINKING

5 19

not be determined by its' components' behaviours. At each level components interact witb the environment and other components according to a set of rules and procedures. Based on information received from other components or outer stimuli, the system adapts by changing the rules. Over a period of time, it will develop new procedures and behave in a different manner. Since, Biology is the study of living things and how they interact and Complex Adaptive Systems are living things that interact with their environments., therefore, the properties of Complex Adaptive Systems are not only relevant to biology but essential to it.

The properties of complex adaptive systems (CAS) are relevant to biology because most ofthe living organisms have many ofthe features of CAS. They involve adapting to a particular environment and having a certain hierarchy.

When a component of the system changes its environment, it changes its rules to better adapt to that new environment by producing complex temporal pattems of adapting components. At each level of the hierarchy, the components are created by the combination of the components of the level below. Similarly, in biology, all the organisms are part of a hierarchical organization and depend on the organisms from the lower levels from which they have evolved. In both, the CAS and biology, the components work and interact together to set new rules for their survival in a new environment.

520

SYLVIA T. D'APOLLONIA ETAL.

APPENDIX B Figure 3 represents the derived mental model (from similarity ratings) for one student. This is contrasted to the same students essay.

Student's Essay: Numbers Refer to Extracted Propositions Biologists have found fossils of sheep-like animals that lived on the banks of rivers in Africa that have the same ankle bones as whales. An explanation for this is that when the sheeps lived on the river banks some of them might have had a change in allelic frequencies (pi). The individuals had to develop certain genotypes to survive given the environmental pressures (p2, p3, p4). Those who had the adaptive features, suitable to tbis environment were selected by Natural Selection (p5). This selection mechanism results in members of the population being more suited (p6) to the environment. Individuals contribute their genes (p7) to the next generation resulting in individuals becoming more and more suited for the environment (p8).

Extracted Propositions pi. Some sheep might have had a change in allelic frequencies. p2. Individuals developed certain genotype (mutations).

Fig. 3. Student's mental model of evolution derived from Pfnet.

ACQUISITION OF COMPLEX SYSTEMIC THINKING

p3. p4. p5. p6. p7. p8.

521

Individuals had t o . . . in order to survive.

Individuals... given the environmental pressures.

Those (individuals) with adaptive features were selected.

Natural Selection leads to Differential Survival of individuals.

Individuals contribute their genes.

Individuals become more and more suited.

Coding of Essay: Knowledge Components Origin of New Traits: —.5 (>.5 for pi implied, —1 for p4).

Mechanism of Inheritance: -1-1 for p7.

Mechanism of Evolution: 0(1 for p5 and p6 (reverses causation). — 1 for p8.

Role of Chance: — 1 for p3.

Coding of Essay: Emergent Nature of Evolutionary Process Hierarchical Levels: 6 (6 for p2, p3, p4, p5, p6, p7 and —2 for pi and p8).

Final Causes: 1 (2 for pi, p5 and —1 for p3).

Thus, this students appears to have three misconceptions; namely, that environmental pressures cause certain mutations to arise, that the origin of adaptive traits is determinate (i.e., not a result of random forces), and that individuals gradually become more and more suited to the environment. This is apparent both in Figure 3 and in the essay. For example, there is an inappropriate link between mutation and environmental pressure in Eigure 3 that is reflected in p4. Although this student appears to have a good grasp of the appropriate levels at which evolution operates, she relates the impact of mutations and genotype on genetic variability of the population directly rather than indirectly mediated by individuals.