Altering the Modality of Instructions to Facilitate Imagination

Instructional Science (2006) 34: 343–365 DOI 10.1007/s11251-005-6075-5

Ó Springer 2006

Altering the modality of instructions to facilitate imagination: Interactions between the modality and imagination effects SHARON TINDALL-FORD & JOHN SWELLER* School of Education, University of New South Wales, Sydney, NSW, 2052, Australia (*Author for correspondence, e-mail: j.sweller@unsw.edu.au) Received: 31 August 2004; in final form: 8 December 2005; accepted: 21 December 2005 Abstract. Under some conditions, learning is improved by using a dual mode presentation involving for example, visual diagrams and auditory, rather than written text (modality effect). Under other conditions, learning is improved by asking learners to imagine rather than study instructional material (imagination effect). Both effects have been explained using cognitive load theory. This paper investigates interactions between the modality and imagination effects. It was hypothesized that the imagination effect would be facilitated when accompanied by audio/visual instructions compared to visual only instructions. Experiment 1 provided evidence to suggest that for the materials used, audio/visual instructions were required to obtain an imagination effect. Experiment 2 through verbal protocols aimed to investigate the cognitive mechanisms required when studying and imagining and found that learners who studied tended to engage in search while learners who imagined focused on entities and relations that needed to be learned.

Keywords: coginitive load theory, imagination effect, modality effect

This paper explores interactions between two instructional effects, the modality and imagination effects. The modality effect occurs when instructional material that is presented in dual, audio/visual form is superior to a visual only presentation (e.g. Tindall-Ford et al., 1997). The imagination effect occurs when learners who are asked to ‘‘imagine’’ a procedure or concept learn more than learners who are simply asked to ‘‘study’’ the same procedure or concept (e.g. Cooper et al., 2001). In this paper we test the hypothesis that the imagination effect is more likely to be obtained following dual mode instruction rather than visual only instruction. Both effects are based on cognitive load theory (Sweller et al., 1998; Sweller, 1999, 2005; Clark et al., 2006) and it is this theory that is used to predict the interaction between the modality and imagination effects.

344 Cognitive load theory is designed to apply some characteristics of human cognition to instructional design. Sweller (2003, 2004) provided a foundation for cognitive load theory by employing an evolutionary view of human cognition designed to indicate why our cognitive architecture has some of the characteristics particularly relevant to instruction. Human cognitive architecture includes a working memory that is very limited with respect to both capacity (Miller, 1956) and duration (Peterson & Peterson, 1959) when dealing with novel information, but with no known limits when dealing with familiar information that has been stored in long-term memory (Ericsson & Kintsch, 1995). Sweller (2003) has suggested that working memory has these characteristics because when dealing with new information, there is no plausible central executive available to determine how elements of information should be combined during learning. In contrast, as indicated below, when dealing with familiar information, schemas held in long-term memory can act as a central executive. If, as occurs when dealing with novel information, there is no central executive to organize elements, they must be organized randomly with the consequences of any random organization tested for effectiveness using a problem solving process. For example, failing knowledge, problem solving moves must be determined randomly and tested for effectiveness. If a person does not know how to get from A to B, they will have to randomly test particular steps to see if they move closer to the goal. Such a system can only operate when dealing with a very small number of elements to prevent combinatorial explosions and hence, when dealing with novel information, working memory must be limited.

In contrast, the characteristics of working memory are altered dramatically when dealing with familiar information organized schematically in long-term memory. Schemas indicate how elements should be organized, obviating the need for random organization followed by tests of effectiveness (Sweller, 2003). In effect, schemas act as a central executive. As a consequence, when dealing with familiar information, working memory limitations are no longer required and indeed, are substantially eliminated (Ericsson & Kintsch, 1995). Unlike when dealing with novel information, schemas, held in long-term memory, can act as a central executive organizing information and actions.

There need be no limits to the amount of familiar, schematically organized information transferred from long-term memory that can be processed by working memory. If a person knows how to get from A to B, the entire route, that can consist of a huge number of

345 potential choices, can be processed effortlessly in working memory because a schema for the route can be retrieved from long-term memory.

In effect, that schema acts as a learned central executive indicating what actions should be taken and when they should be taken. In the process, the limitations of working memory are eliminated.

These characteristics of human cognitive architecture have instructional implications (Sweller, 2003, 2004). One of the functions of instruction should be to provide a non-random organization of novel information. Instruction can act as a substitute for the missing central executive when dealing with novel information. As indicated above, once schemas have been constructed, they can take over the central executive function and instruction is no longer required. Until that point is reached, without suitable instructional material and procedures, problem solving is unavoidable and problem solving must include random combinations of elements with tests of effectiveness.

Cognitive load theory has provided a variety of instructional design procedures intended to reduce the need to randomly test element combinations while reducing working memory load and so assisting schema acquisition. Instructional design procedures based on cognitive load theory derive from a series of effects with each effect generated by the superiority of a particular instructional procedure using a controlled experimental design. The imagination effect is one example (Cooper et al., 2001; Ginns et al., 2003; Leahy & Sweller, 2005). The effect occurs when students demonstrate greater learning and understanding after they are encouraged to imagine or mentally practice a procedure that has been demonstrated in a worked example, compared to studying the same worked example. Imagining in this context requires learners to mentally work through the steps to solution of a worked example without looking at the example while studying means looking at the example and emphasizing understanding and learning the relevant procedures and concepts.

Evidence for the effectiveness of imagining (or synonymously, mental practice) on learning has been demonstrated for many years over a wide range of physical activities (Surburg, 1968; Schick, 1970; Rawlings & Rawlings, 1974; Romero & Silvestri, 1990; Etnier & Landers, 1996). Much less emphasis has been placed on cognitive tasks but Driskell et al. (1994), conducting a meta-analysis on the efficacy of mental practice, found that the greater the cognitive demands involved in a task, the more beneficial mental practice was in increasing performance. Cooper et al. (2001), Ginns et al. (2003), and Leahy & Sweller (2005) found that imagining a cognitive procedure was

346 superior to studying the same procedure but only if learners had sufficient schematic knowledge to process the material in working memory. For novices with insufficient knowledge, studying was superior to imagining, providing an example of the expertise reversal effect (Kalyuga et al., 2003).

If the imagination effect occurs only under conditions where learners have sufficient knowledge to be able to manipulate the relevant material in working memory, the effect also might be increased under other conditions designed to facilitate learning. Reducing working memory load or facilitating working memory functioning may enhance learning.

Another cognitive load effect, the modality effect, could provide such a set of circumstances (Low & Sweller, 2005; Mayer, 2005). Contemporary research suggests that working memory is not a simple, solitary structure but composed of multiple channels or processors (Penney, 1989; Baddeley, 1992; see also Paivio, 1986). These processors include a visual system for processing visual images and an auditory system for dealing with verbal information. The two systems appear to process their different forms of information with some degree of independence which means that usable working memory capacity may be increased if both systems are used (e.g. visual and auditory processors) rather than only one processor. Using Baddeley’s terminology, by presenting verbal material in auditory rather than written form, only the phonological loop is required while presenting the material in visual form will require the use of both the phonological loop and the visual-spatial sketchpad.

Thus, the use of spoken rather than written material should reduce the working memory load on the visual-spatial sketchpad increasing usable working memory.

The modality effect occurs under split-attention conditions (e.g. Sweller et al., 1990). Assume multiple visual sources of information such as a diagram and its associated, written text that, because they refer to each other, cannot be understood in isolation. In order to be understood, visual attention must be split between the multiple sources of information and they must be mentally integrated. Alternatively, they can be physically integrated which reduces cognitive load leading to the split-attention effect (e.g. Sweller et al., 1990). The use of dual mode presentation acts as a substitute for physical integration. Textual material, rather than being presented in written (visual) form can be presented in spoken (auditory) form. If, as suggested above, the use of both auditory and visual processors increases usable working memory capacity (see Penney, 1989 for a review), then switching from written to spoken text under split-attention conditions, should facilitate learning.

347 This modality effect has been obtained on many occasions. Research on the modality effect and instructional design has demonstrated that studying instructional materials which employ a dual format consisting of, for example, visual diagrams and auditory text may result in superior learning compared to studying an equivalent visual only presentation consisting of visual diagrams and visual text (e.g. Mousavi et al., 1995; Tindall-Ford et al., 1997; Moreno & Mayer, 1998; Kalyuga et al., 2000; Bru¨nken et al., 2002, 2004; Mayer & Moreno, 2003; Tabbers et al., 2004).

In summary, it is suggested the modality effect occurs for the following reasons: (a) Dual modality presentation expands usable working memory, permitting the same information presented in dual rather than single modality form to be processed more readily. (b) Expanding usable working memory reduces cognitive load which is determined by the difference between the working memory resources needed to process information and the working memory resources available. Cognitive load is high if almost all of the resources available are needed to process essential information but low if there are excess resources available. (c) If cognitive load is reduced by increasing the working memory resources available, more resources become available for schema construction, the ultimate aim of most instruction.

The current work tested the hypothesis of an interaction between the imagination and modality effects. Experiment 1 compared four distinct instructional formats: 1. Audio/visual instructions followed by an imagination component; 2. Audio/visual instructions followed with a conventional study based strategy; 3. Visual only instructions followed by an imagination component; 4. Visual only instructions followed by a conventional study based strategy.

It was hypothesized that the imagination effect would be more likely to be obtained following audio/visual than visual alone presentation techniques. The first experiment tested this hypothesis while the second used verbal protocols to obtain information concerning the cognitive processes occurring under the various conditions.

Experiment 1 The first experiment tested the hypothesis that the imagination effect was more likely to be obtained under audio/visual than visual only

348 conditions. The specific hypotheses were that the imagination effect was more likely to be obtained comparing the imagination and study groups under audio/visual conditions than visual only conditions because more working memory capacity should be available to learners to imagine the information under the audio/visual conditions. Under the visual only conditions, if insufficient working memory capacity is available to imagine the information, the imagination effect may be inhibited.

The curriculum area used was a sub-section of a junior high school mathematics curriculum. Learners were taught how to construct a frequency table, how to sum frequencies and how to calculate the mean, mode and range using the table.

Method Participants Forty-four male eighth grade students from a Sydney private secondary school participated in this experiment. Students were of similar socio-economic (middle-class) and ethnic (Caucasian) background and were turning 14 years of age during the school year. The school had six eighth grade classes. Students were organized into three levels of mathematics based on their end of 7th grade mathematics exam results. The top class consisted of students who were accelerated through the mathematics curriculum and the sixth class consisted of students who were receiving remedial instruction in mathematics. For the purpose of this experiment, only students from the four middle ability classes were tested. These students were chosen because pilot testing indicated that the materials used in the experiment were suitable for these students but too easy for many students in the top class and too difficult for students in remedial classes. All students had completed the algebra unit of the eighth grade syllabus but had not been introduced to basic statistics, which was the instructional material used in this experiment.

Materials and procedure Participants were randomly allocated to one of the four groups with the only restriction being that, as far as possible, equal numbers of participants from each class were allocated to each group. The visual only instructions presented to the two Visual-only groups consisted of a frequency distribution table and related textual statements. Figure 1 provides an example of visual-only study instructions. Neither the table nor the text was intelligible as separate entities. An understanding

349

Figure 1. Instructional material used in section 1 of both experiments.

of the instructions and how to perform a particular statistical operation could only be made by mentally integrating the text with the related diagrammatic entities. The audio/visual instructions provided the same frequency distribution table and textual information, however the textual instructions were provided in an auditory form via a tape recorder. The length of the audio instructions determined the time allocated to either reading the instructions or listening to the tape.

350 There were eleven participants in each group and each participant was tested individually. There were four instructional sections demonstrating basic statistical concepts and operations. Each section increased the complexity of information presented and relied on an understanding of the previous sections. Each section incorporated a learning phase consisting of reading or listening to the instructional material, a practice phase consisting of studying or imagining the instructions, and a test phase incorporating test questions based on what had been previously learned.

The section 1 learning phase had a time limit of 70 s to read or listen to the taped instructions. The instructions demonstrated how data could be organized into a frequency distribution table. These instructions were followed immediately by a practice (60 s) and test phase (120 s) for this section. The section 1 test phase required students to enter data into a frequency distribution table. The data was presented to students in raw form and they had to record each score value and the frequency of each score. Three test marks were allocated to this test section. Figure 1 demonstrates the instructional material used in section 1.

The section 2 learning phase instructions (110 s) demonstrated how, on a frequency distribution table, the score value (x) may be multiplied by the frequency value (f), to calculate the fx value. These instructions were immediately followed by a practice (90 s) and test (120 s) phase for this material. This test phase required students to ďŹ ll in the missing numbers on a frequency distribution table. Students had to calculate the fx value or if presented with the fx value calculate either the score or frequency value that was missing. Twelve marks were allocated to this test section.

Section 3 (110 s) showed how to calculate the mean from a frequency distribution table by calculating the sum of all scores, then dividing by the sum of the frequency value. A practice (90 s) and then a test (300 s) phase followed based on the instructional material presented in this section. The two test questions for section 3 required students to ďŹ ll in the missing data on the frequency distribution table, calculate the fx column, calculate the sum of the frequency column and the sum of the fx column and use this information to calculate the mean. Fourteen marks were allocated to this test section. Figure 2 shows the visual only instructional material used for section 3 for both the imagination and study groups.

The section 4 learning phase instructions (90 s) demonstrated how to calculate the mean, range and mode from a frequency distribution

351

Figure 2. Instructional material used in section 3 of both experiments.

table. These instructions too were immediately followed by a practice (120 s) and test (600 s) phase. The ďŹ rst two test questions for this section required students to calculate the mean, mode and range from

352 a frequency distribution table, while the last question required students to initially organize the data into a frequency distribution table and then calculate the range, mode and mean. Twenty nine marks were allocated to this test section.

As indicated above, a practice phase immediately followed each learning phase for each of the four sections. The practice phase was a continuation of the learning phase. Participants were asked to either study or imagine the procedures they had just been learning. The instructions provided to both the study and imagination groups were identical to the information shown during the learning phase except that the spoken information was no longer available to the audiovisual groups and the identical written information was eliminated for the visual only groups. Thus, only the tabular/diagrammatic information of a frequency table without explanatory information was available during the practice phases of each of the four sections.

Prior to students starting Experiment 1, each participant was given a verbal overview describing what was required during the practice phases when either studying or imagining. Students in the Study groups were instructed as follows: When studying, try and understand the information shown, try and make sense of the instructional material and what statistical procedure is being demonstrated. Attempt to understand the steps to solve the problem. Read through the information carefully and examine the diagram.

Students in the Imagination groups were told: When imagining, try to imagine yourself working out the problem, close your eyes or look away from the information and think about yourself actually carrying out the steps and solving the problem. Attempt to imagine the steps needed to solve the problem. If you forget how to do the problem look back at the instructional material.

Students were then provided with an example unrelated to the experimental materials to study or imagine. The example showed a rectangle with measurements for its length and breadth. Under the diagram was the mathematical solution for calculating the perimeter of the rectangle. Students in the Study group were told to ‘‘study the information, read through the text, study the diagram and try and understand the information’’. The Imagination groups were provided with exactly the same example of a rectangle with the corresponding

353 mathematical calculations. Students in the Imagination groups were told when imagining the information ‘‘turn away from the material and try and imagine yourself actually doing the problem, that is, performing the steps to solve the problem.’’ This introduction to the strategies of either studying or imagining was used to try to ensure that students had an understanding of and employed these strategies during each practice phase. After each set of learning materials, students were given a specified practice time to study or imagine.

As indicated above, test phases followed the learning and the practice phases for each of the four sections. In all test sections, once a student completed a question the student could not return to the question nor use it as a reference for future questions. All test questions were objective and aimed to assess the students’ ability to apply the statistical processes and procedures that had been introduced during the learning and practice phase. No test questions required replication of what was previously shown in the learning and acquisition phases. An example of the test material for section 2 is shown in Figure 3. It should be noted that while each test question had a maximum time, learners could finish earlier. The time to find a solution

Figure 3. Example of test materials used in Experiment 1, section 2 part 2.

354 was noted, with learners who failed to complete a question within the available time allocated the maximum time. While time to solution was not emphasised to participants, it should be noted that all participants were aware that their time to solution was being recorded.

Results and discussion The variables under analysis were scores on test items. Means and standard deviations are displayed in Table 1. A 2 (visual vs. audio-visual) 2 (imagine vs. study) analysis of variance was completed on overall test scores. There was a significant audio/visual effect, F(1, 40)=5.34, MSe=78.80, p=0.03, Cohen’s d=0.7, a significant imagination effect, F(1, 40)=7.95, MSe=78.80, p=0.01, d=0.85 and a significant audio/ visual by imagination interaction, F(1, 40)=5.82, MSe=78.80, p=0.02, d=0.73.

Following the significant interaction, simple effects testing indicated a significant difference between the audio/visual-imagine and audio/visual study groups, F(1, 20)=12.95, MSe=82.19, p=0.002, d=1.53, but no significant difference between the visual imagine and visual study groups, F(1, 20)=0.09, MSe=75.93, p=0.77, indicating the imagination effect was only obtainable under audio/visual conditions. Clearly the significant interaction is due primarily to the superiority of the audio/visual imagination group. To investigate if the audio/visual and imaginations effect were more robust on more complex information, remembering each section became more complex and introduced more concepts, follow up ANOVA’s of the four individual section tests were conducted. Analyses indicated an identical pattern of significance to the over all test.

The results demonstrate that the imagination effect was only obtainable under audio/visual but not under visual only conditions.

We argue that the audio/visual technique facilitated schema construction, with imagination then allowing schema automation. Students’ ability to imagine the information was enhanced when audio/visual

Table 1. Means and standard deviations for test scores in Experiment 1; n=11

Mean SD

Audio/visual Imagination

Audio/visual Study

Visual Imagination

Visual Study

50.82 5.90

36.82 11.34

38.18 7.03

37.09 10.12

355 instructions were used prior to imagining the information. To imagine a mathematical procedure, it must be possible for all elements of that procedure to be processed in working memory. A schema incorporating the relevant elements assists in this process and the audio/visual technique facilitated schema construction. Schema construction, in turn, permits multiple elements of information to be treated as a single element in working memory. (A schema for the word ‘‘cat’’, for example, permits competent readers to treat the word as a single element in working memory rather than as three separate letters.) Once schema construction freed sufficient working memory resources to allow the effective imagination of the statistical procedures required, automation could commence. In contrast, study techniques following audio/visual instruction did not make as much use of the newly acquired schemas while visual only instruction did not even permit sufficient schema construction to commence, rendering subsequent imagination instructions ineffective.

With respect to the modality effect, it might be noted that while audio/visual instructions were superior to visual only instructions under imagination conditions contributing to the significant interaction, there was no modality effect under study conditions. A possible explanation is that the tests failed to detect differences in schema acquisition until the schemas became automated under imagination conditions. Schemas may have been better acquired under audio/visual conditions but may not have been usable during the tests until they had become automated.

Experiment 1 indicated the efficacy of an imagination instructional strategy when combined with audio-visual instructions. The next experiment aimed to further investigate the effectiveness of audio/visual and imagination instructions via the use of verbal protocols in order to ascertain the procedures used by students when they were required to imagine or study.

Experiment 2 Experiment 1 provided quantitative evidence that students who engaged in imagining, rather that studying information may have a greater understanding of statistical concepts and procedures.

However, the superiority of imagining was only found following audio/ visual instructions. The aim of Experiment 2 was to further explore this phenomenon, by qualitative rather than quantitative analysis. The objective was to provide information concerning the cognitive

356 processes involved in imagination by collecting verbal protocols during the practice phase in Experiment 2. As was the case in Experiment 1, Experiment 2 included learning, practice, and test phases. During the learning phase, students were provided learning materials in audio-visual or visual only format, followed by a practice phase using the same materials without the auditory component or the equivalent written component for the visual only groups, where students either studied or imagined the statistical processes.

The verbal protocols of Experiment 2 were intended to provide evidence for two hypotheses. (1) It was hypothesized that learners asked to study the materials would engage in a search for what needed to be learned while learners asked to imagine would directly engage in the learning process. Search requires working memory resources and interferes with learning (e.g., Sweller & Cooper, 1985; Cooper & Sweller, 1987). (2) It was hypothesized that the verbal protocols would provide evidence that imagining is assisted when preceded by audio/visual instructions compared to visual only instructions because of enhanced learning under audio/visual instructions.

A number of techniques were employed in this experiment to maximize the validity of the verbal reports. Based on the work of Ericsson and Simon (1993) concerning procedures that increase the validity of verbal protocols, the following strategies were used. An unaided verbalization procedure was adopted where the students were asked to provide a verbal monologue while either imagining or studying. The verbal monologue was audio taped and later transcribed. Minimum intervention by the researcher occurred, but when a student paused for a period of time, the researcher would ask the student to ‘‘tell me what you are thinking about’’. Students were instructed not to try and explain their thoughts, but merely to narrate their thoughts.

Method Participants Sixteen female students from a Sydney girls’ secondary school, participated in Experiment 2. (It should be noted that past research on both the modality and imagination effects has not obtained any distinctions between males and females – see Mousavi et al., 1995; Cooper et al., 2001; Ginns et al., 2003.) Students were of similar socio-economic (middle class) and – ethnic (Caucasian) background.

All students were turning 13 years of age during the school year and were presently completing seventh grade. The school had five seventh grade classes. Only students from the top mathematics class took

357 part in the experiment. In terms of knowledge and ability, pilot studies indicated this class was most similar to the students used in Experiment 1. The top class was formed based on results obtained from a series of mathematics examinations that students completed at the beginning of the school year. The pilot studies indicated that students in the middle and lower ability classes may have had difficulty learning the materials, as they had not been taught sufficient algebra to understand the statistical concepts being introduced. Unlike Experiment 1, which was conducted at the beginning of the school year with middle ability students from Year 8, this experiment was conducted near the end of the year with higher ability students in Year 7.

Thus, there was about a 3 month difference in amount of schooling received by the participants between the two experiments.

Materials and procedure The instructions and procedures used in Experiment 2 were similar to those used in the previous experiment. As was the case for Experiment 1, a 2 2 experimental design was used. Because the intention was to collect verbal protocols rather than analyze quantitative data, there were only 4 participants per group.

Modifications were made to reduce the time required to test each participant because of school stipulations and the added time required to collect verbal protocols. For this reason, participants in this experiment did not complete section 1 or section 2 test questions. Students in Experiment 2 completed the following; 1.

2.

3.

4.

Section Section Section Section

1 2 3 4

Learning and Practice Phase, Learning and Practice Phase, Learning, Practice and Test Phase, Learning, Practice and Test Phase.

The description of instructional, practice material, times and marks allocated for each section can be found in the description of Experiment 1.

During the imagination/study component, students were told to try and keep up a running commentary of what they were doing, including their thought processes. Participants in the imagination groups were told that they should tell the experimenter ‘‘what you are imagining’’. Students from the study group were told they should indicate ‘‘what you are thinking about or looking at’’. During the study or imagination process, if students failed to provide a commentary, the experimenter prompted them with the above statements. A test phase followed the learning and practice phase. Students completed test

358 questions from Sections 3 and 4 of Experiment 1. All questions and marking were identical to Experiment 1 (see Table 1).

The following procedures were used to analyse the verbal protocols and provide evidence concerning the two hypotheses. We hypothesized that learners asked to study the materials would engage in a search for what needed to be learned while learners asked to imagine would directly engage in imagining the procedures. We looked for evidence of this hypothesis by comparing the extent to which search statements such as ‘‘looking for’’, ‘‘searching’’, ‘‘attempting to find out’’, ‘‘checking’’ or equivalent terms were characteristically used by the study but not imagination groups while direct procedural statements such as ‘‘I am doing’’, ‘‘I am calculating’’, ‘‘I am working through’’ were used by the imagination but not the study groups.

The hypothesis that imagining is assisted when preceded by audio/ visual instructions compared to visual only instructions because of enhanced learning under audio/visual instructions was assessed by considering the positive and negative imagery statements made by audio/visual and visual only groups. A positive imagery statement is one that indicates an ability to imagine while a negative imagery statement is one that indicates difficulty in imagining.

Results and discussion As stated previously, the purpose of this study was to obtain qualitative data through detailed verbal protocols that were collected during the practice phase when students had to either imagine or study the information. Participants were encouraged to provide a running commentary in order to give an insight into their cognitive processes.

This commentary was recorded and then later transcribed and analyzed.

Cognitive processes used by learners in the imagination and study groups Analysis of students’ comments from the two study groups, indicate that searching is a common element in many of the verbal protocols.

Search statements are defined as ones in which learners include terms such as ‘‘looking for’’, ‘‘searching’’, ‘‘attempting to find out’’, ‘‘checking’’ or equivalent terms. Search appears to be one of the key processes involved when asked to study as can be seen from the following examples: ‘‘I am looking at the answer to find information’’, ‘‘I am looking for all possible answers....just looking for number

359 patterns’’ and ‘‘I kind of figure out all the kinds of ways I could figure this out’’ and ‘‘I keep searching making up my own examples’’.

Research has repeatedly demonstrated the negative effect of the search process for learning and understanding (e.g., Sweller & Cooper, 1985; Cooper & Sweller, 1987). While these studies compared problem solving search with studying worked examples, the present work suggests that search occurs during studying as well. A search process places heavy demands on limited working memory resources, leaving few for the task of learning. Differential search may provide a valid explanation why study groups performed relatively poorly on test questions compared to imagination groups in Experiment 1. The process of searching reduces focus. Statements like, ‘‘I am mostly looking, I’m searching, I am checking’’ and ‘‘I am looking over things’’ confirm this suggestion. Every study student, regardless whether they were in the audio/visual study or visual study group made some reference to engaging in some form of search process when studying.

In contrast, the verbal protocols collected from the imagination groups suggested when a person effectively imagines a procedure a more focused cognitive engagement appears to occur. Statements like, ‘‘I am doing’’, ‘‘I am calculating’’, ‘‘I am working through’’, ‘‘I’m visualizing’’ or the equivalent were common phrases used by both imagination groups that indicated focused cognitive processing.

Examples suggesting a successful imagination process included; ‘‘I go through each number in the score column and multiply it by its frequency so one times three is three, then two times its frequency of two is four, I am doing each one now and to do the sum of the frequency I am going through and adding each number together’’.

When the imagination process is used correctly, it appears to provide some framework and structure for learning and encourages deliberate, organized practice. The imagination group provided no statements concerning search.

Imagination is assisted by the use of audio/visual instructions Verbal protocols indicated that the imagination process appeared to be assisted when coupled with audio/visual instructions. Positive imagery statements were determined by students’ verbal protocols indicating active cognitive processing. Some examples are; ‘‘I can see myself doing each one of these steps’’; ‘‘I multiply each one, that is what I am doing now’’ and ‘‘what I am doing now is dividing the fx by the f ’’. This can be contrasted to comments from students in the visual-imagine group who indicated on occasions an inability to imagine a process. This difficulty in imagining was defined by negative

360 statements and is demonstrated by the following verbal protocols; ‘‘I don’t know how to imagine it’’; ‘‘I’m not sure, I don’t quite understand’’ and ‘‘I’m trying to find out ...um... it is not clear really what I should do’’. There was only one statement from the audio/visual-imagination group suggesting an inability to imagine.

The visual imagination group tended to make tentative statements defined by words such as; ‘‘I might’’, ‘‘I guess’’ or equivalent. Examples are: ‘‘I imagine how it all works out ... I imagine I would go through all the numbers from 1 to 10’’ and ‘‘I guess I keep going through, working the numbers’’. The visual-imagination group’s verbal protocols indicate vague, poorly defined thought processes.

These thought processes can be compared to statements from the audio/visual-imagination group which demonstrated a deliberate, focused pattern of thinking; ‘‘I see the table, have a picture of it in my head, so I can remember it.... multiply the score by the frequency in my head, which I can do ... then I get the answer’’ and ‘‘what I am doing is multiplying the score by the f value to get each fx value’’.

Also evident through the analysis of verbal protocols is the fractured and on occasions incorrect nature of the visual-imagination students’ understanding of concepts; ‘‘ first the mode is the most popular number ... the mode on the table is five, the most popular number; I would then subtract it from the smallest number to get 5’’. This provides an example of a student confusing the mode and range.

Although this student could provide the correct meaning for the mode, she failed to understand the process of finding the mode from the frequency distribution table. She had formed an incorrect procedure for calculating the mode and on all test questions requiring the calculation of the mode performed a subtraction. A comparison may be made with the audio/visual-imagination group where all four students made reference to looking at the frequency column to find the largest number and then observing what number from the score column corresponded with the frequency value; ‘‘The mode, the number that occurred the most. I look down the frequency column, find the largest frequency, I am scanning in my head that column then I look at the score number next to the largest frequency number’’.

In summary, the verbal protocols indicate some of the cognitive processes that can be used to explain the quantitative results of Experiment 1. Imagining material results in processing that is focused on the entities and relations that need to be learned while studying the material can result in the use of a somewhat vague search for rele-

361 vant patterns. Furthermore, that process of imagining is assisted by learning using audio/visual rather than visual only materials.

General discussion The cognitive processes engaged in by students imagining audio/visual materials are complex. Nevertheless, information is becoming available concerning many of the relevant variables and relations and that information is beginning to have instructional implications. Cognitive load theory (Sweller, 2005) with its emphasis on the instructional implications of human cognitive architecture can frequently be used as a guide. The theory has previously generated the instructional modality (Tindall-Ford et al., 1997) and imagination (Leahy & Sweller, 2005) effects. Particular interaction patterns between these two effects are suggested by the theory. Those interactions were investigated in the current work.

Some instructional materials necessarily include disparate sources of information that must be mentally integrated before they become intelligible because each source of information is unintelligible in isolation (Sweller et al., 1990). Integrating disparate sources of information imposes a heavy working memory load. Presenting some of that information in spoken or auditory mode rather than presenting all of it in written or visual mode can reduce the load on visual working memory. In other words, transferring some of the information to the auditory channel should reduce the load imposed on the visual channel. Experiment 1 provided evidence for this modality effect with audio/visual instructions proving superior to visual only instructions under imagination (although not study) conditions.

Instructional materials, which are imagined, can sometimes be learned better than materials that are merely studied. Whether imagining or studying is better depends, at least in part, on the levels of expertise of the learners. In order to imagine a procedure, the schemas associated with it must be sufficiently established to enable the learner to process the procedure in working memory. Without that possibility, imagination instructions may be difficult or impossible to follow, resulting in improved performance by learners studying rather than imagining the material. Thus, an imagination effect may only be obtained using material for which learning has already commenced.

In the initial phases of learning, the effect is reversed with study instructions proving superior. While not tested in the present

362 experiments, that effect has been consistently obtained (Cooper et al., 2001; Ginns et al., 2003; Leahy & Sweller, 2005).

If the imagination effect requires that learning be advanced to some extent and if the modality effect indicates that dual mode presentations advance learning more than single mode presentations, it follows that the imagination effect is more likely to be obtained following dual mode instructions rather than single mode instructions.

Experiment 1 demonstrated the imagination effect under audio/visual conditions but not under visual only conditions. In turn, the verbal protocols of Experiment 2 provided evidence for some of the proposed theoretical constructs. They suggested that learners processed the material differently when asked to study or imagine it and that the imagination process was assisted by the use of audio/visual instruction.

Instructional implications flow from the results of these experiments. First, obtaining the modality effect under imagination conditions and obtaining the imagination effect under audio/visual conditions confirm the importance of these instructional design effects. When presented with multiple sources of information that cannot be understood in isolation, using dual mode instructions can be beneficial. (It must be emphasized, that previous work has indicated that these may be the only conditions under which dual mode conditions are beneficial, e.g., Kalyuga et al., 2000.) In addition, asking learners to imagine rather than study information to be learned also may be beneficial providing they have progressed sufficiently to enable them to imagine the information. The interaction between the modality and imagination effects provides the major new finding of this study. If dual modality presentation can enhance learning (as occurred under imagination conditions) and if some degree of learning is essential before the material can be imagined, then dual mode presentation followed by imagination instructions should be advantageous. That result was obtained.

The current work demonstrated the advantages of using dual modality instruction when asking learners to imagine rather than simply study examples. Given the failure to find a modality effect under study conditions, further work is required to delineate the precise conditions under which this interaction may or may not occur. For example, Tindall-Ford et al. (1997) found that the modality effect was only obtainable under conditions of high rather than low informational complexity. It is reasonable to suppose that similar results may be obtainable for the interaction between the modality and imagina-

363 tion effects. Lastly, while the current results were obtained under realistic conditions, the study (particularly Experiment 1) was conducted as a controlled experiment. Long-term educational studies testing the suggestion that learning will be facilitated if learners are presented dual-modality instructions and asked to imagine rather than study instructional material are required.

Acknowledgements The work reported in this paper was supported by a grant from the Australian Research Council to the second author. The authors wish to thank Brother Patrick Howlett, the Principal of Marcellin College and the staff and students of Marcellin College for their participation in Experiment 1. We also acknowledge the support and assistance of the Principal, Miss Rosalyn Bird and the staff and students of Danebank, Anglican School for Girls Hurstville who participated in Experiment 2.

References Baddeley, A. (1992). Working memory. Science 255: 556–559.

Bru¨nken, R., Plass, J. & Leutner, D. (2004). Assessment of cognitive load in multimedia learning with dual-task methodology: Auditory load and modality effects.

Instructional Science 32: 115–132.

Bru¨nken, R., Steinbacher, S., Plass, J.L. & Leutner, D. (2002). Assessment of cognitive load in multimedia learning using dual-task methodology. Experimental Psychology 49: 1–12.

Clark, R., Nguyen, F. & Sweller, J. (2006). Efficiency in learning: evidence-based guidelines to manage cognitive load. San Fransisco: Pfeiffer.

Cooper, G. & Sweller, J. (1987). Effects of schema acquisition and rule automation on mathematical problem solving transfer. Journal of Educational Psychology 79: 347–362.

Cooper, G., Tindall-Ford, S., Chandler, P. & Sweller, J. (2001). Learning by imagining.

Journal of Experimental Psychology: Applied 7: 68–82.

Driskell, J.E., Copper, C. & Moran, A. (1994). Does mental practice enhance performance? Journal of Applied Psychology 7: 481–492.

Ericsson, K.A. & Kintsch, W. (1995). Long-term working memory. Psychological Review 10: 211–245.

Ericsson, K.A. & Simon, H.A. (1993). Protocol analysis: Verbal reports as data.

Cambridge, Mass: MIT Press, Revised edn.

Etnier, J.L. & Landers, D.M. (1996). The influence of procedural variables on the efficacy of mental practice. The Sport Psychologist 10: 48–57.

Ginns, P., Chandler, P. & Sweller, J. (2003). When imagining information is effective.

Contemporary Educational Psychology 28: 229–251.

364 Kalyuga, S., Ayres, P., Chandler, P. & Sweller, J. (2003). The expertise reversal effect.

Educational Psychologist 38: 23–31.

Kalyuga, S., Chandler, P. & Sweller, J. (1998). Levels of expertise and instructional design. Human Factors 40: 1–17.

Kalyuga, S., Chandler, P. & Sweller, J. (2000). Incorporating learner experience into the design of multimedia instruction. Journal of Educational Psychology 92: 126–136.

Leahy, W. & Sweller, J. (2005). Interactions among the imagination, expertise reversal and element interactivity effects. Journal of Experimental Psychology: Applied 11: 266–276.

Low, R. & Sweller, J. (2005). The modality principle. In R.E. Mayer, ed., Cambridge handbook of multimedia learning, pp. 147–158. Cambridge University Press: New York.

Mayer, R. (2005). Cognitive theory of multimedia learning. In R.E. Mayer, ed., Cambridge handbook of multimedia learning, pp. 31–48. Cambridge University Press: New York.

Mayer, R. & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist 38: 43–52.

Miller, G.A. (1956). The magical number of seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review 63: 81–97.

Moreno, R. & Mayer, R.E. (1998). A split-attention effect in multimedia learning: Evidence for dual-processing systems in working memory. Journal of Educational Psychology 90: 312–320.

Mousavi, S.Y., Low, R. & Sweller, J. (1995). Reducing cognitive load by mixing auditory and visual presentation modes. Journal of Educational Psychology 87: 319–334.

Paivio, A. (1986). Mental representations: a dual coding approach. Oxford, England: Oxford University Press.

Penney, C.G. (1989). Modality effects and the structure of short term verbal memory.

Memory and Cognition 17: 398–422.

Peterson, L.R. & Peterson, M. (1959). Short term retention of individual items. Journal of Experimental Psychology 58: 193–198.

Rawlings, E.I. & Rawlings, I.L. (1974). Rotary pursuit tracking following mental rehearsal as a function of voluntary control of visual imagery. Perceptual and MotorSkills 38: 302–309.

Romero, K. & Silvestri, L. (1990). The role of mental practice in the acquisition and performance of motor skills. Journal of Instructional Psychology 17: 218–221.

Shick, M. (1970). Effects of mental practice on selected volleyball skills for college women. Research Quarterly 41: 88–94.

Surburg, P.R. (1968). Audio, visual and audio-visual instruction with mental practice in developing the forehand tennis drive. Research Quarterly 39: 728–734.

Sweller, J. (1999). Instructional design in technical areas. Victoria, Australia: ACER Press.

Sweller, J. (2003). Evolution of human cognitive architecture. In B. Ross, ed., The psychology of learning and motivation, pp. 215–266. Academic Press: San Diego.

Sweller, J. (2004). Instructional design consequences of an analogy between evolution by natural selection and human cognitive architecture. Instructional Science 32: 9–31.

Sweller, J. (2005). Implications of cognitive load theory for multimedia learning. In R.E. Mayer, ed., Cambridge handbook of multimedia learning, pp. 19–30. Cambridge University Press: New York.

365 Sweller, J., Chandler, P., Tierney, P. & Cooper, M. (1990). Cognitive load as a factor in the structuring of technical material. Journal of Experimental Psychology: General 119: 176–192.

Sweller, J. & Cooper, G. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and Instruction 2: 58–89.

Sweller, J., van Merrienboer, J. & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review 10: 251–296.

Tabbers, H., Martens, R. & Van Merrie¨nboer, J. (2004). Multimedia instructions and cognitive load theory: Effects of modality and cueing. British Journal of Educational Psychology 74: 71–81.

Tindall-Ford, S., Chandler, P. & Sweller, J. (1997). When two sensory modes are better than one. Journal of Experimental Psychology: Applied 3: 257–287.