Assessing Teacher's Epistemological and Ontological World Views

Chapter 2

Assessing Teachers’ Epistemological and Ontological Worldviews Gregory J. Schraw and Lori J. Olafson

Abstract This chapter focuses on issues surrounding the assessment of teachers’ epistemological and ontological beliefs. Epistemology is the study of beliefs about the origin and acquisition of knowledge. Ontology is the study of beliefs about the nature of reality. Previous research has focused primarily on epistemological beliefs using self-report Likert scales. We discuss several limitations of this approach, including lack of agreement about the dimensionality of epistemological beliefs.

Few studies have examined teachers’ ontological beliefs, nor have any studies investigated the joint contribution of epistemological and ontological beliefs. The present chapter proposes an integrated, holistic system in which teachers rate themselves using the two-dimensional scale shown in Fig. 2.1. Teachers are asked to situate their epistemological and ontological beliefs on scales ranging from realist to relativist perspectives (see Instructions). We report on the feasibility of using the two-dimensional system, including potential strengths and weaknesses of the system, and directions for future research.

2.1

Introduction

This chapter focuses on issues surrounding the assessment of teachers’ epistemological and ontological beliefs. Epistemology is the study of beliefs about the origin and acquisition of knowledge (Hofer, 2004). Ontology is the study of beliefs about the nature of reality (Lincoln & Guba, 2000; Shadish, Cook, & Campbell, 2002).

Research over the last three decades has focused primarily on the structure and development of college students’ epistemological beliefs (Baxter-Magolda, 1999, 2002; King & Kitchener, 1994; Kuhn, Cheney, & Weinstock, 2000; Perry, 1970; Schommer, 1990; Hofer & Pintrich, 1997). Most studies have measured epistemological beliefs using self-report scales (Hofer, 2000; Schommer, 1990; Schraw et al., 2002), although some have used interview techniques to determine a holistic epistemological stance (Kitchener & King, 1981; Kuhn, 1991; Perry, 1970), open-ended questionnaires

University of Nevada, Las Vegas, USA

M. S. Khine (ed.), Knowing, Knowledge and Beliefs: Epistemological Studies across Diverse Cultures. © Springer 2008

25

26

G. J. Schraw and L. J. Olafson

(Yang, 2005), and content analysis of verbal explanations (Slotta & Chi, 2006) to measure epistemological and ontological assumptions. These studies suggest important differences among students that are related to learning outcomes such as achievement, critical thinking, metacognition, and strategy use (Baxter-Magolda, 2002; Hofer, 2001; Schommer, 1994; Schraw & Sinatra, 2004).

Researchers have only begun a serious investigation of teachers’ epistemological beliefs in the last decade. Much of this research has been conducted independently from the research on students’ epistemological beliefs. Several recent reviews have tried to connect these bodies of literature (Buehl & Alexander, 2001; Schraw & Olafson, 2002; Woolfolk-Hoy et al., 2006). One problem has been how to conceptualize teachers’ epistemological beliefs separate from the previous research on college students’ beliefs. Generally speaking, researchers view teachers’ epistemological beliefs as being more holistic in nature as compared to research investigating college students’ epistemological beliefs, which tended to parcel beliefs into four or five separate dimensions (Hofer, 2000; Schommer-Aikins, 2002). Thus, most studies classify teachers as endorsing a particular epistemological worldview based on interviews (Levitt; 2001; White, 2000; Wilcox-Herzog, 2002) or by match to prototypical worldviews presented in short vignettes (Schraw & Olafson, 2002).

This chapter describes a strategy for simultaneously assessing teachers’ epistemological and ontological beliefs. This issue is important for several related reasons. One is that previous research with both students and teachers has confounded different types of beliefs within the general category of epistemological beliefs (Hofer & Pintrich, 1997; Schraw & Olafson, 2002). One goal of the present research is to articulate the difference between epistemological and ontological beliefs for the purpose of assessing different stances on each dimension (FarnhamDiggory, 1994). A second reason is to explore measurement options other than factor analytic rating scales for teachers and students that have been used to self-report personal beliefs on a number of different dimensions. Thus, a second goal of this chapter is to describe an assessment system that can be used with fundamentally different types of beliefs such as epistemology and ontology, yet measure these beliefs using a comparable scale. Using a common metric scale is important in order to determine whether epistemological and ontological beliefs are related to each other, as well as to other variables such as teachers’ motivational beliefs (e.g., self-efficacy), curricular choices, and pedagogical strategies used in the classroom.

We emphasize at the outset that our purpose is to propose and describe a new assessment technique that requires additional testing. Nevertheless, we believe there is an acute need to develop a separate conceptual definition of epistemological and ontological beliefs, as well as to develop a methodological strategy for assessing these beliefs in a manner that enables researchers to determine how they are related to many other teacher and classroom variables. In this light, we stress the exploratory, developmental nature of our ongoing research.

This chapter is divided into seven main sections. Section 2.1 reviews and critiques measurement strategies used to assess students’ epistemological beliefs in previous research. Section 2.2 reviews and critiques measurement strategies used to assess

2 Assessing Teachers’ Epistemological and Ontological Worldviews

27

teachers’ epistemological beliefs, noting that previous research has treated students and teachers differently in terms of the conceptual structure of beliefs, as well as strategies for measuring beliefs. Section 2.3 makes an argument for distinguishing between teachers’ epistemological and ontological beliefs, and for the importance of a strategy to measure each on the same scale. Section 2.4 describes a technique we refer to as the four-quadrant scale, which we describe in detail and present preliminary results based on a sample of 24 practicing teachers. Section 2.5 discusses several potential advantages of the four-quadrant scale. Section 2.6 considers reliability and validity issues, while section 2.7 discusses six questions for future research.

2.2

Previous Research

A number of researchers have attempted to measure epistemological beliefs and world-views using self-report scales or holistic descriptions (Hofer, 2001; Kuhn, 1991; Schommer, 1990; Schraw et al., 2002). We use the term epistemology in its broadest sense to refer to a theory of knowledge (Lehrer, 1990; Pollock, 1986).

Hofer (2002, p. 4) defines epistemology as being “concerned with the origin, nature, limits, methods, and justification of human knowledge.” We use the term epistemological beliefs in this article to refer to “beliefs about knowledge and knowledge acquisition.” In contrast, we use the term epistemological worldview to refer to the collective set of epistemological beliefs that comprise a holistic belief system. We make this distinction because some researchers have attempted to measure epistemological beliefs, while others have attempted to measure epistemological worldviews.

Much of the research over the last two decades has focused on epistemological beliefs, which have been defined by different researchers as beliefs about a specific facet of knowledge such as certainty, complexity, or the source of knowledge and knowing. Schommer (1990) proposed five independent beliefs based on the work of Perry (1970) pertaining to certain knowledge (i.e., absolute knowledge exists and will eventually be known), simple knowledge (i.e., knowledge consists of discrete facts), omniscient authority (i.e., authorities have access to otherwise inaccessible knowledge), quick learning (i.e., learning occurs in a quick or not-at-all fashion), and innate ability (i.e., the ability to acquire knowledge is endowed at birth). Currently, there is debate as to whether Schommer’s five beliefs constitute genuine epistemological dimensions (Hofer & Pintrich, 1997). Most researchers agree that beliefs about the certainty and simplicity of knowledge constitute genuine epistemological beliefs. In contrast, many researchers argue that beliefs about innate ability reflect some other, non-epistemological dimension (Hofer & Pintrich, 1997).

Schommer (1990) developed the Epistemological Questionnaire (EQ) to assess the five dimensions described above. The EQ consisted of 62 simple statements that individuals responded to using a five-point Likert scale, indicating the extent to which they agreed or disagreed with the statement. The EQ has been used frequently over the last 15 years by Schommer and other researchers (Chan & Elliott, 2004;

28

G. J. Schraw and L. J. Olafson

Kardash & Scholes, 1996; Schommer-Aikins, 2002). Results have been somewhat mixed in a variety of ways. First, factor analyses typically reported less than five interpretable factors (Hofer, 2001; Schraw et al., 2002). Second, some analyses have yielded factors with an interpretation that was not predicted by Schommer (1990). For example, a factor might pertain to the source of knowledge or have item loadings that are difficult to interpret in a manner consistent with Schommer’s proposed five-factor structure. Third, the factors reported in the analyses often had a small number of items with acceptable factor loadings, and therefore had low or unacceptable reliability coefficients. Fourth, solutions usually explained a relatively small proportion of sample variance (e.g., 20–35%), which raised concerns about the construct validity of the questionnaire.

Schraw et al. (2002) developed a modified version of Schommer’s EQ called the Epistemic Beliefs Inventory (EBI) to address these methodological problems.

The EBI contained 32 items based on the five factors proposed by Schommer (1990). The EBI typically yielded the five proposed factors; however, reliabilities tended to be low (e.g., .50–.65) and results varied depending on the age and gender of the sample. In addition, like the EQ, the EBI explained a relatively small proportion of sample variation (e.g., usually less than 40%).

Hofer (2001) proposed an alternative four-factor framework and developed an instrument called the Epistemological Beliefs Questionnaire (EBQ) to assess these factors. She proposed four factors, which were subsumed under two general dimensions referred to as the nature of knowing and the process of knowing. The former refers to what knowledge is presumed to be, while the latter refers to how one comes to know and understand knowledge. The “nature” dimension included two factors called certainty of knowledge (i.e., the degree to which one sees knowledge as fixed versus fluid and changeable) and simplicity of knowledge (i.e., the degree to which knowledge is viewed as individual facts versus complex, interrelated concepts). The “process” dimension included two factors called source of knowledge (i.e., the extent to which credible knowledge is self- or other generated) and justification of knowing (i.e., the rules and criteria that individuals use to evaluate knowledge claims).

Hofer (2001) reported four empirically derived factors that differed somewhat from the four proposed factors described above. Her four observed factors included certainty of knowledge, source of knowledge, justification, and attainment of truth.

A “simplicity of knowledge” factor was not observed, whereas an “attainment of truth” factor was observed. The attainment of truth factor was interpreted as the extent to which experts can attain deep knowledge (i.e., “truth”) within their area of expertise. Like the EQ and EBI, the EBQ explained approximately 45% of sample variance, had several factors with few items that loaded satisfactorily, and had factors with low reliability coefficients.

Thus far, self-report instruments that have been developed to assess multiple epistemological beliefs have had mixed success. One important contribution of this research is that there has been a great deal of productive discussion regarding the set of constructs that comprise the domain of epistemological beliefs (Buehl & Alexander, 2001; Hofer, 2001, 2004; Hofer & Pintrich, 1997; Olafson & Schraw,

2 Assessing Teachers’ Epistemological and Ontological Worldviews

29

2002; Schraw & Sinatra, 2004). These discussions are crucial to mapping the possible domain of epistemological beliefs both conceptually and methodologically.

A second contribution has been the preliminary findings concerning the relationships among epistemological beliefs and a variety of outcomes variables such as age, education level, gender, moral reasoning skills, and academic achievement.

2.3

Four Measurement Concerns

Notwithstanding the contributions of current instruments, there are a number of ongoing concerns that encouraged us to explore other assessment strategies. One concern is that existing self-report instruments have not agreed on what should and can be measured by these instruments. Specifically, there is disagreement about how many separate epistemological beliefs should be included in the general domain of epistemological beliefs. For example, while most researchers agree on beliefs related to the complexity and certainty of knowledge, there is less agreement regarding the source and origin of knowledge, and still less agreement regarding issues related to the attainment of truth. Neither is it well understood how these separate beliefs are interrelated. Some researchers have argued that epistemological beliefs are unrelated, while others have argued that separate beliefs (SchommerAikins, 2002; Schommer, 1990) are related to a broader epistemological meta-construct (Schraw & Olafson, 2002). A closely related issue is whether epistemological beliefs are unique to each domain (Hofer, 2000, 2001) or common across domains (Olafson & Schraw, 2006).

A second concern is the low predictive validity between epistemological factors used in ongoing research and various outcome variables such as academic achievement. For example, correlations between epistemological beliefs and academic performance typically account for 3–8% of sample variance in the outcome measure (Hofer, 2001). Similar findings have been reported for reading (Schommer, 1994) and problem solving (Schommer et al., 1992). The fact that beliefs are not correlated highly with academic outcomes may be due to low reliability and restriction of range in the epistemological measurements, or to the possibility that sophisticated personal beliefs have little effect on day-to-day academic outcomes. In any case, findings would be more useful and generalizable if self-reported epistemological factors explained larger proportions of variation in salient outcomes such as academic achievement.

A third concern is that self-report instruments attempt to measure narrowly defined epistemological beliefs rather than holistic epistemological worldviews that are assumed to represent an integrated set of beliefs about knowledge. A number of studies have assessed holistic epistemological worldviews based on self-report measures (White, 2000), interviews in which individuals reason about complex problems (King & Kitchener, 1994; Kuhn, 1991), or rate the degree to which they endorse different epistemological worldviews based on written vignettes (Schraw & Olafson, 2002). These studies have linked holistic epistemological worldviews to

30

G. J. Schraw and L. J. Olafson

complex behaviors such as argumentative reasoning (Kuhn, 1991) and teachers’ curricular and instructional choices (White, 2000). One potential advantage of this approach is that researchers or participants can identify a multifaceted set of beliefs that describes the epistemological milieu that guides the individual’s thought and professional choices. Using a broad rather than narrow aperture to examine epistemological beliefs may provide a richer description of a person’s epistemological worldview. In addition, it is important to note that measuring epistemological beliefs via separate self-reported beliefs versus a holistic stance are not mutually exclusive measurement strategies. Schraw and Olafson (2002) found significant relationships between three different holistic epistemological stances and the five epistemological beliefs as measured by the Epistemological Belief Inventory (EBI). Using the two strategies simultaneously to cross-validate one another may enhance future research endeavors.

A fourth concern, and one of special importance to this chapter, is that previous research has focused exclusively on epistemological beliefs without also considering what we refer to henceforth as ontological beliefs. Epistemological beliefs have been defined in the literature as beliefs about the origin and nature of knowledge.

For present purposes, we define ontological beliefs as beliefs about the nature of reality (Lincoln & Guba, 2000; Mertens, 2005). The distinction between epistemological and ontological beliefs and/or worldviews is an extremely important one for both conceptual and methodological reasons. From a conceptual standpoint,philosophers of science have traditionally distinguished between the two and argued that both contribute to the way in which social scientists view and conduct research and construe theories of metascience (Kuhn, 1962; Lakatos, 1978; Popper, 1959; Shadish et al., 2002). From a methodological perspective, it is unclear how epistemological and ontological beliefs are related to one another, to student achievement, or to teachers’ instructional practices.

We believe it is crucial to measure both epistemological and ontological beliefs and to do so in a manner in which each type of belief is assessed on the same measurement scale. The focus of this chapter is on describing such an approach and summarizing the advantages of using this strategy. As noted earlier, using the strategy described later does not preclude other measurement strategies; thus, we believe that it is best to use multiple measurement strategies when assessing personal beliefs.

2.4

The Four-quadrant Scale

We describe a new strategy for assessing epistemological and ontological beliefs using a common measurement scale. We refer to this as the four-quadrant scale because there are four distinct quadrants into which a person can be classified based on self-report of external judgment by a researcher. This approach is an application of the issues discussed by Shadish et al. (2002), regarding differences in individual beliefs about the theory and conduct of social science research. We begin with

2 Assessing Teachers’ Epistemological and Ontological Worldviews

31

definitions of epistemological and ontological beliefs based on Shadish et al.

(2002), then describe the structure of the four-quadrant scale, and summarize some pilot findings using the scale.

2.4.1

Definitions

We define epistemological beliefs as our collective beliefs about the origin and acquisition of knowledge. We assume that different individuals hold different epistemological beliefs, both in terms of the content of their beliefs, as well as the relative sophistication of beliefs. We do not assume that these beliefs are necessarily explicit and subject to reflection, although they may be, and hopefully become more explicit and structured as individuals develop expertise within a domain and become more sophisticated thinkers (Cunningham & Fitzgerald, 1996; Hofer & Pintrich, 1997; Kuhn, 1999; Prawat, 1992). We define ontological beliefs as beliefs about the nature of reality. We assume that different individuals hold different beliefs and that these beliefs differ with respect to their explicitness and sophistication.

We make no assumption currently about whether these beliefs are related or how they develop.

As noted above, previous research has attempted to identify and measure separate epistemological beliefs (Hofer, 2000; Schommer, 1990; Schraw et al., 2002). For present purposes, we focus on holistic epistemological worldviews rather than individual epistemological beliefs. Schraw and Olafson (2002) used the term epistemological worldview to refer to collective beliefs about the nature and origin of knowledge. Researchers have attempted to measure epistemological worldviews using verbal explanations (Kuhn, 1991), written vignettes (Schraw & Olafson, 2002), and open-ended responses (Yang, 2005). Similarly, we focus on ontological world-views rather than ontological beliefs. Several studies have measured ontological beliefs using think-alouds, though none have examined teachers’ ontological beliefs, or the relationship between epistemological and ontological beliefs (Slotta & Chi, 2006).

2.4.2

Creating the Scale

Shadish et al. (2002) provided a detailed discussion of the role of epistemological and ontological beliefs with respect to the process of scientific inquiry. They suggested that each type of belief exists on a continuum that ranges from realist to relativist endpoints. A realist believes that entities or phenomena (e.g., knowledge or physical matter) exist and can be understood and explained to some degree, even if experts do not currently understand the phenomenon that is being considered. For example, a physicist may believe that “dark matter” exists in open space even though it is currently undetectable. The basis for their belief may be theoretical

32

G. J. Schraw and L. J. Olafson

(e.g., mathematical models), partial empirical evidence, or faith. In contrast, a relativist believes that entities may exist in an ever-changing manner (e.g., the changing nature of human rights), or that we can never know with certainly whether something exists (e.g., that God exists). From an educational perspective, a teacher with a realist world-view would be more likely to endorse a belief in a universal curriculum that is transmitted to students via a knowledgeable teacher; whereas a relativist would be more likely to endorse a constructivist view that each student constructs knowledge that is relevant to him or her, given help from the teacher (Brownlee & Berthelsen, 2006; Chan & Elliott, 2004).

We wish to elaborate on two assumptions made by Shadish et al. (2002) that we concur with. One is that beliefs occur on a continuum and may change over time from realist to relativist or the reverse. A second is that the epistemological beliefs and worldviews held by an individual may be at one point on the continuum even though the same individual’s ontological beliefs and worldview may be at a different point on the continuum. Thus, a person’s commitment to a realist or relativist point of view may change over time and differ across epistemological and ontological dimensions. We assume that beliefs are changeable due to a variety of factors, but especially education, explicit inquiry, collaborative discussion, and developing critical reasoning skills (Kuhn, 1999; White & Frederiksen, 2005). We also assume that realist world-views are related to traditional teaching practices, while relativist worldviews are related to constructivist practices (Brownlee & Berthelsen, 2006; Levitt, 2001; Olafson & Schraw, 2002; White, 2000).

Our goal was to develop an assessment tool that allowed individuals to situate their epistemological and ontological beliefs in an easy to understand manner. Two steps were necessary to do so. First, we needed to create a set of instructions for how to identify one’s epistemological and ontological worldviews. This seemed especially important for individuals unfamiliar with these terms or who had not considered their worldviews explicitly. Second, we needed to create an easy to use scale on which individuals could rate their worldviews. We created the instructions shown in Table 2.1 that were used in a pilot study of the four-quadrant scale with practicing teachers. The purpose of this study was to ask teachers to situate their epistemological and ontological worldviews using the four-quadrant scale.

The four-quadrant scale partitions epistemological and ontological worldviews into two axes at right angles to each other that range from realist to relativist on each axis. This yields four quadrants in which a person can rate oneself as realistrealist, realist-relativist, relativist-realist, or relativist-relativist. Individuals are able to select a point in any area of the four-quadrant array that best corresponds to their personal epistemological and ontological worldviews about teaching.

Participants read general instructions and summaries of epistemological and ontological realist and relativist positions shown in Table 2.1. They next considered their own worldviews and then rated themselves on the four-quadrant scale shown in Fig. 2.1 by placing an X in one of the four quadrants that best corresponded to their personal epistemological and ontological worldviews.

Several comments are warranted regarding the rating sheet in Fig. 2.1. First, the four quadrants are shown in a box-shaped figure that is subdivided into four

2 Assessing Teachers’ Epistemological and Ontological Worldviews

33

Table 2.1 Instructions to participants Instructions We want you to rate and explain your epistemological and ontological worldviews. Please read the following description of terms used in this study. Then indicate with an “X” where you would place yourself in the four quadrants shown on the Rating Sheet. To make your X, find the point where your ratings intersect on the epistemology dimension and the ontology dimensions.

Please note that the descriptions provided below represent endpoints on each of the scales. Your own beliefs may lie anywhere between these two endpoints. You may use any part of the four quadrant area.

After you make your rating, please describe in as much detail as possible on the Explanation Sheet your reasoning for your self-rating.

Epistemology Epistemology is the study of what can be counted as knowledge, where knowledge is located, and how knowledge increases. The personal epistemology of teachers is characterized by a set of beliefs about learning and the acquisition of knowledge that drives classroom instruction.

Epistemological Realist An epistemological realist would believe that there is an objective body of knowledge that must be acquired. From a teacher’s perspective, this position would hold that curriculum is fixed and permanent and focuses on fact-based subject matter. An epistemological realist might believe the following: ● ● ●

There are certain things that students simply need to know.

I am teaching information that requires memorization and mastery.

There are specific basic skills that need to be mastered.

Epistemological Relativist An epistemological relativist would describe curriculum as changing and student-centered.

Problem-based or inquiry curricula are examples at the other end of the continuum from a perspective of a one-size-fits-all curriculum. One of the central features of curriculum from this position is the notion that curriculum is not fixed and permanent. An epistemological relativist might agree with the following statements: ● ● ● ●

The things we teach need to change along with the world.

The content of the curriculum should be responsive to the needs of the community.

It is useful for students to engage in tasks in which there is no indisputably correct answer.

Students design their own problems to solve.

Ontology Ontology is the study of beliefs about the nature of reality. The personal ontology of teachers is characterized by a set of beliefs regarding whether students share a common reality and what a classroom reality should look like.

Ontological Realist A teacher who is an ontological realist assumes one underlying reality that is the same for everyone. Instructionally, this means that all children should receive the same type of instruction at the same time regardless of their individual circumstances and context. An ontological realist would agree with the following: ● ● ●

Student assignments should always be done individually.

It is more practical to give the whole class the same assignment.

The teacher must decide on what activities are to be done. (continued)

34

G. J. Schraw and L. J. Olafson

Table 2.1 (continued) Ontological Relativist An ontological relativist assumes that different people have different realities. From an instructional perspective, teachers are seen as collaborators, co-participants, and facilitators of learning who work to meet the individual needs of students. Instructional practices are less teacher-directed, such as: ●

●

Students need to be involved in actively learning through discussions, projects, and presentations.

Students work together in small groups to complete an assignment as a team.

Quadrant 4

Ontological Relativist

Epistemological Realist

Quadrant 3

Quadrant 1

Epistemological Relativist

Ontological Realist

Quadrant 2

Fig. 2.1 The four-quadrant scale

subsections. It is unclear presently whether the outside perimeter to the box should or should not be present. We note that in the pilot study, where the perimeter was not included, participants sometimes placed their X outside the boundary of the perimeter. Second, Fig. 2.1 shows the upper right-hand quadrant as quadrant 1.

The lower right-hand corner was designated as quadrant 2, with quadrants 3 and 4 occurring in a counterclockwise manner. The labels quadrant 1–4 did not appear on the rating sheet used in the pilot study, but was used as an interpretative convenience. In contrast, the four labels indicating epistemological and ontological realism/relativism were included as anchor points in the pilot study. Third, the figure used in the pilot study used axes of equal length. We used a scale with two right-angle axes of 150 mm length (i.e., approximately 6 in. in length). A fourquadrant scale of these dimensions fits onto a page well and seems large enough to allow raters a wide selection of possible choices. The location of each participant’s

2 Assessing Teachers’ Epistemological and Ontological Worldviews

35

X can be scored on a scale of 1–150 using a ruler scaled in millimeters. For example, an X in the extreme upper right corner would be scored as a 150 on the epistemological axis and a 150 on the ontological axis. In contrast, an X at the intersection point of the two axes would be scored as a 75 on each dimension. This enables researchers to assess both types of worldviews on the same scale.

2.5

Pilot Study

Twenty-four practicing teachers participated in the study. All participants were enrolled in a graduate level course in curriculum and instruction at a large western university and working toward a graduate degree in education. Half the participants were enrolled in a M.Ed. program, while half were enrolled in a Ph.D. program.

The average age was 38.2 years. Teachers had between two and 23 years teaching experience.

Participants took part in the pilot study as volunteers during their regularly scheduled class. The four-quadrant scale was administered by their classroom instructor. Participants were not allowed to discuss their worldviews with others prior to making their ratings; however, the remainder of their class was devoted to discussing and comparing worldviews after completing their rating and justification.

Individuals read instructions and completed the rating sheet. This required approximately 5 min. Individuals next received 10 min to provide a written explanation of their epistemological and ontological worldviews.

2.6

Preliminary Findings and Comments

Visual results are shown in Fig. 2.2. Several preliminary findings were germane to the development and piloting of the four-quadrant scale. One finding was that participants were able to complete the four-quadrant scale quickly and efficiently.

Individuals reported that the instructions were easy to understand and that they had a good idea of what they were expected to do. The instructions shown in Table 2.1 are appropriate for teachers’ worldviews, but would need to be modified to use the four-quadrant scale with other populations such as educational researchers, historians, or philosophers.

A second finding was that most respondents were in quadrants 1 and 4, with only two respondents in quadrant 3, and none in quadrant 2. Approximately 22 of the 24 participants rated themselves as ontological relativists of some degree.

In contrast, approximately 45% of respondents rated themselves as epistemological realists of some degree. A content analysis of written explanations suggested that individuals in quadrant 3 were more likely to endorse traditionalist views, which included support for a universal curriculum based on core knowledge and skills, whereas individuals in quadrant 1 supported constructivist views, which emphasize

36

G. J. Schraw and L. J. Olafson

Ontological Relativist

Epistemological Relativist Epistemological Realist

Ontological Realist Fig. 2.2 Visual results from pilot study

the role of student inquiry, collaboration, and self-learning. These results are consistent with several recent studies that have compared differences between prototypical realist and relativist worldviews (Brownlee & Berthelsen, 2006; Chan & Elliott, 2004; Schraw & Olafson, 2002).

A third finding was that we observed a statistically significant positive relationship between the epistemological and ontological dimension, r = .45, p < .05, using a two-tailed test. We computed this correlation by assigning two separate scores to each teacher’s rating. One score was scaled from 1–150 on the epistemological dimension, while the second score was scaled from 1–150 on the ontological dimension. The finding of a significant positive correlation between the two suggests that realist beliefs on one dimension are associated with realist beliefs on the second dimension. However, we emphasize that the present sample was highly selective in that it included experienced teachers enrolled in advanced graduate courses. This outcome may be unrepresentative of a larger, more diverse sample.

Overall, the pilot study indicated that the four-quadrant scale can be used with teachers in a quick and straightforward manner to assess their epistemological and ontological worldviews. Preliminary findings suggested that different worldviews may correspond to differences in the beliefs that teachers hold about curriculum, and pedagogy. Although not part of the pilot study, teachers’ worldviews may be related to other classroom factors such as assessment and discipline practices (Brownlee & Berthelsen, 2006; Olafson & Schraw, 2006, 2002; White, 2000; Wilcox-Herzog, 2002).

2 Assessing Teachers’ Epistemological and Ontological Worldviews

2.6.1

37

Potential Strengths of the Four-quadrant Scale

The pilot study described earlier suggests that the four-quadrant scale provides a viable measure of teachers’ worldviews. There are several potential advantages of the four-quadrant scale compared to previous assessment strategies. One of the system’s most important advantages is that it enables researches to distinguish clearly between epistemological and ontological worldviews. This distinction is extremely important because the two belief systems represent different assumptions on the part of teachers that may affect classroom practices in different ways. The pilot study yielded a correlation between the two dimensions of r = .45, suggesting that the two world-views may be correlated moderately under certain circumstances. Future studies should be conducted to examine in more detail when the two dimensions are and are not interrelated.

A second advantage is that the four-quadrant scale is quick and easy to use. Data can be collected in 10–20 min, including written justifications from teachers (or other research participants) regarding that self-ratings. Data can be scored and interpreted quickly and without in-depth technical expertise. In contrast, survey results typically need to be scored, tabulated, and evaluated using complex statistical procedures such as factor analysis. Similarly, verbal reports (Slotta & Chi, 2006), open ended questionnaires (Yang, 2005), or holistic ratings based on written essays (Kuhn, 1991) are time consuming and subject to differences in judges who evaluate the responses. Given additional validation research on the four-quadrant scale, we believe that it offers researchers a quick and efficient way to collect important information about teachers’ worldviews that does not preclude other data collection strategies such as surveys and interviews.

A third advantage of the four-quadrant scale is that it provides measures of epistemological and ontological worldviews on the same metric scale. We used a 150 mm scale in the pilot study for each of the two dimensions because it seemed visually optimal. Future studies may choose to compare different scaling systems to determine whether different scales increase or decrease the sensitivity of the method. Nevertheless, the four-quadrant scale as described earlier enables researchers to make meaningful comparisons given that the epistemological and ontological dimensions share the same scale. This should facilitate both quantitative (e.g., correlation and regression analyses) and qualitative comparisons (e.g., differences across dimensions regarding a commitment to relativism).

A fourth advantage is that the system enables researchers to compare teachers in each of the four quadrants to those in other quadrants on critical variables such as gender, years of experience, curricular and instructional practices, self-efficacy beliefs, as well as a variety of other variables. Presently it is unclear whether differences within a quadrant are as important as differences between quadrants.

Future research should investigate how seemingly major differences such as those between extreme points in quadrants 1 and 3 are related to teaching practice and student achievement.

38

2.6.2

G. J. Schraw and L. J. Olafson

Reliability and Validity

Currently there is no data to address the reliability or validity of the four-quadrant scale, although such data can be collected in a straightforward manner. It is not possible to compute the internal consistency of the four-quadrant self-rating given that it is a single, onetime event. However, it is possible to compute test-retest reliability on a sample. We would expect this index to be quite high over a short period of time such as 2â&#x20AC;&#x201C;4 weeks.

Both construct and criterion-related validity are important aspects of any assessment instrument. Construct validity refers to the degree to which an inference is warranted about the phenomenon of interest within a particular sampling context.

Construct validity typically has been determined using correlational techniques in which the construct of interest (e.g., epistemological worldviews) are related to similar and dissimilar constructs. Positive correlations with similar constructs (e.g., ontological worldviews, different measures of epistemological worldviews, or measures of critical thinking) constitute convergent validity evidence; whereas negative correlations with dissimilar constructs (e.g., support for book censorship) constitute discriminant validity (Mertens, 2005). A strong case for construct validity is made when a researcher selects several similar and dissimilar constructs, predicts which will be positively and negatively correlated with the construct of interest, and those predictions are supported empirically. Strong support for the four-quadrant scale would be obtained if an individual completed the four-quadrant approach, as well as other measures of epistemological beliefs such as the Epistemological Questionnaire (Schommer, 1994) and the self-report vignettes used by Schraw and Olafson (2002), and the individual scored similarly on all three measures. Currently, we are engaged in several research studies intended to provide evidence in support of the four-quadrant scaleâ&#x20AC;&#x2122;s construct validity.

Criterion-related validity refers to the degree to which the construct of interest is related to other constructs such as gender, education, academic achievement, etc.

Two types of criterion-related validity are useful to researchers, including concurrent and predictive validity. Concurrent validity occurs when the construct of interest is related to a different construct such as academic achievement when measured at the same point in time. Predictive validity occurs when the construct of interest is related to a different construct of interest school at some point in the future. We anticipate that teachers who score higher on the epistemological and ontological dimensions (i.e., endorse a strong relativist view) would tend to have more education, support constructivist pedagogy, and employ a more diverse curriculum than teachers who scores low on the two dimensions.

We believe that the four-quadrant scale should be validated using other available measure of epistemological beliefs (Hofer, 2001; Schommer, 1994; Schraw et al., 2002) and worldviews (Schraw & Olafson, 2002), measures of teachersâ&#x20AC;&#x2122; curricular and instructional choices and practices, measures teacher beliefs such as self-efficacy and goal orientations, and measures of student outcomes such as interest, motivation, and achievement.

2 Assessing Teachers’ Epistemological and Ontological Worldviews

2.7

39

Questions for Future Research

A number of questions for future research are related to the development of the four-quadrant scale. We focus on six of these questions. One is a more detailed description and analysis of each of the four quadrants. We assume that each quadrant represents a fundamentally different perspective on knowing and reality.

Some of these positions, such as individuals in quadrants 1 and 3, seem to be highly consistent in that individuals endorse either universally realist of relativist worldviews. In contrast, individuals in quadrant 3 seem to hold perhaps inconsistent beliefs in that they endorse a relativist ontology, coupled with a realist epistemology.

Additional qualitative analyses of the four quadrants are needed to understand the thinking that supports each of these worldviews, as well as the degree to which individuals reconcile their beliefs on the epistemological and ontological dimensions.

A second is the extent to which the type and amount of preservice teacher education training affects epistemological and ontological worldviews (Britzman, 2000; Brownlee et al., 2001). Few studies have examined preservice teachers’ epistemological worldviews in detail, and none that we know of have examined ontological beliefs. It is possible that preservice training has little effect on preservice teachers’ worldviews. However, it is likely that the type of pre-service training a student receives affect his or her beliefs differently (Joram & Gabriele, 1998; Laplante, 1997). One possibility is that traditional classroom-based preservice training has a smaller effect than field-based apprenticeships (Olafson & Schraw, 2002; White, 2000).

A third question concerns teacher development over time (Garet et al., 2001; Kuhn et al., 2000). Previous research suggests that teachers develop over time regarding teaching practices and that their beliefs change as part of their development (Brownlee et al., 2001; Gill et al., 2004; Woolfolk-Hoy et al., 2006).

Very little is known currently regarding the development of teachers’ epistemological and ontological beliefs (Bendixen, 2002). One issue is whether the two beliefs develop in tandem or follow separate trajectories. A second issue is the general trend in development. We assume that beliefs become more relativist over time, but may become stable at some point, or perhaps move from a relativist to realist worldviews past a certain number of years of experience (Lieberman, 1995; Reybold, 2001). Cross sectional studies would help address these questions, although longitudinal studies would be especially useful for understanding the development of beliefs.

A fourth question relates to understanding the differences within and among the four quadrants shown in Fig. 2.1. Teachers within a quadrant may differ from other teachers within the same quadrant. Presently, it is unclear whether these differences matter in terms of type or degree. Teachers also differ from teachers in other quadrants, and these differences are assumed to be much more substantial in nature. For example, teachers in quadrant 1 tend to endorse relativist views on both dimensions, whereas teachers in quadrant 3 tend to endorse realist views on

40

G. J. Schraw and L. J. Olafson

both dimensions. Future research should help to develop a prototype teacher for each of the four quadrants and to compare systematically the differences among the four quadrants. We assume that teachers in the four quadrants differ in many regards, including beliefs and practices about curriculum, pedagogy, assessment strategies, and classroom management (Borko & Putnam, 1996; Johnston et al., 2001; Schraw & Olafson, 2002).

A fifth question concerns the relationship between teachers’ beliefs and practices (Wilcox-Herzog, 2002). At least two critical issues are in need of research.

One is the separate and interactive rolls of epistemological and ontological beliefs on teachers’ curricular and instructional choices. Previous research suggests that teachers with high versus low self-efficacy adopt different instructional and classroom management strategies (Calderhead, 1996; Goddard et al., 2000; Pajares, 1996; Woolfolk-Hoy & Burke-Spero, 2005). Several recent studies also suggest that different epistemological beliefs lead to different teaching practices (Brownlee & Berthelsen, 2006; Chan & Elliott, 2004; Yang, 2005). We anticipate that teachers with strong relativist worldviews on either of the epistemological or ontological dimensions are more likely to conduct a student-centered classroom (Brownlee & Berthelsen, 2006; Hashweh, 1996). A second issue is the extent to which teachers’ epistemological and ontological beliefs are related to student engagement and achievement. We assume that relativist teachers conduct a more constructivist oriented classroom that is more likely to engage students and promote deeper learning (Holt-Reynolds, 2000).

A sixth question concerns ethnic and cultural differences among teachers’ epistemological and ontological beliefs. Previous research suggests that beliefs differ as a function of academic discipline (Buehl & Alexander, 2001; Hofer, 2001; Jehng et al., 1993) and years of teaching experience (Woolfolk-Hoy et al., 2006).

Very little is known currently regarding cross-cultural differences in teachers’ beliefs, but especially epistemological and ontological beliefs, which have received less attention from researchers, although existing studies report important differences (Ceci & Roazzi, 1994). We conjecture that cultural differences occur, but that highly effective teachers tend to adopt relativist views of learning regardless of cultural differences (Brownlee & Berthelsen, 2006; Chan & Elliott, 2004). This hypothesis remains to be tested.

2.8

Summary

The main goal of this chapter was to describe a new measurement strategy that enables researchers to examine the separate and combined effects of teachers’ epistemological and ontological beliefs. We summarized previous research, discussed the pros and cons of existing assessment strategies, and made a case for some of the advantages of the four-quadrant scale. The two main advantages are that the four-quadrant scale incorporates ontological as well as epistemological worldviews, and also provides a format for assessing epistemological and

2 Assessing Teachers’ Epistemological and Ontological Worldviews

41

ontological worldviews using the same scale metric. For this reason, both teachers’ and students’ epistemological and ontological worldviews can be compared in a more parsimonious manner.

Our proposed system requires a great deal of validation research. We outlined several of the critical issues involved in establishing the construct and criterionrelated validity of an interpretation based on the four-quadrant scale. Validation studies should occur early in the overall research program to assure that the fourquadrant scale is trustworthy. Given support for the four-quadrant scale, a variety of important research questions can be addressed, including the development of beliefs, and the relationship among beliefs, worldviews, and teaching practices.

Consistent with the main theme of the present volume, we encourage researchers to develop profiles of prototypical teachers in each of the four quadrants and to compare these profiles across cultures and different school environments. It is unclear presently what cultural and environmental factors shape teachers’ beliefs, and how beliefs relate to teaching practices in different cultures. One possibility is that two teachers with highly similar worldviews adopt different teaching practices, in part due to cultural mandates and expectations. Of special interest is whether epistemological and ontological beliefs affect teaching practices to the same degree across cultures, or are equally permeable to change through education and experience.

References Baxter-Magolda, M. B. (1999). The evolution of epistemology: Refining contextual knowing at twenty something. Journal of College Student Development, 36, 205–216.

Baxter-Magolda, M. B. (2002). Epistemological reflection: The evolution of epistemological assumptions from age 18 to 30. In B. Hofer & P. R. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing (pp. 89–102). Mahwah, NJ: Erlbaum.

Bendixen, L. D. (2002). A process model of epistemic belief change. In B. K. Hofer & P. R. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing (pp. 191–208). Mahwah, NJ: Erlbaum.

Borko, H., & Putnam, R. T. (1996). Learning to teach. In D. C. Berliner & R. C. Calfee (Eds.), Handbook of educational psychology (pp. 673–708). New York: MacMillan.

Britzman, D. (2000). Teacher education in the confusion of our times. Journal of Teacher Education, 51(3), 200–205.

Brownlee, J., Purdie, N., & Boulton-Lewis, G. (2001). Changing epistemological beliefs in preservice teaching education students. Teaching in Higher Education, 6, 247–268.

Brownlee, J., & Berthelsen, D. (2006). Personal epistemology and relational pedagogy in early childhood teacher education programs. Early Years, 26, 17–29.

Buehl, M. M. & Alexander, P. A. (2001). Beliefs about academic knowledge. Educational Psychology Review, 13, 385–418.

Calderhead, J. (1996). Teachers: Beliefs and knowledge. In D. C. Berliner & R. C. Calfee (Eds.), Handbook of educational psychology (pp. 709–725). New York: Macmillan.

Ceci, S. J., & Roazzi, A. (1994). The effects of context on cognition: Postcards from Brazil. In R. J. Sternberg, & R. K. Wagner (Eds.), Mind in context: Interactionist perspectives on human intelligence (pp. 74–104). Cambridge, England: Cambridge University Press.

Chan, K., & Elliott, R. G. (2004). Relational analysis of personal epistemology and conceptions about teaching and learning. Teaching and Teacher Education, 20, 817–831.

42

G. J. Schraw and L. J. Olafson

Cunningham, J. W., & Fitzgerald, J. (1996). Epistemology and reading. Reading Research Quarterly, 31, 36–60.

Farnham-Diggory, S. (1994). Paradigms of knowledge and instruction. Review of Educational Research, 64, 463–477.

Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38, 915–945.

Gill, M. G., Ashton, P. T., & Algina, J. (2004). Changing preservice teachers’ epistemological beliefs about teaching and learning I in mathematics: An intervention study. Contemporary Educational Psychology, 29, 164–185.

Goddard, R. D., Hoy, W. K., & Hoy, A. W. (2000). Collective teacher efficacy: Its meaning, measure and impact on student achievement. American Educational Research Journal, 37, 479–507.

Hashweh, M. Z. (1996). Effects of science teachers’ epistemological beliefs in teaching. Journal of Research in Science Teaching, 33, 47–63.

Hofer, B. K. (2000). Dimensionality and disciplinary differences in personal epistemology.

Contemporary Educational Psychology, 25, 378–405.

Hofer, B. (2001). Personal epistemology research: Implications for learning and teaching.

Educational Psychology Review, 13, 353–384.

Hofer, B. (2002). Epistemological world views of teachers: From beliefs to practices. Issues in Education: Contributions from Educational Psychology, 8(2), 167–173.

Hofer, B. (2004). Exploring the dimensions of personal epistemology in differing classroom contexts: Student interpretations during the first year of college Contemporary Educational Psychology, 29, 129–163.

Hofer, B. K., & Pintrich, P. R. (1997). The development of epistemological theories: Beliefs about knowledge and knowing and their relation to learning. Review of Educational Research, 67, 88–140.

Holt-Reynolds, D. (2000). What does the teacher do? Constructivist pedagogies and prospective teachers’ beliefs about the role of the teacher. Teaching and Teacher Education, 16, 21–32.

Jehng, J. J., Johnson, S. D., & Anderson, R. C. (1993). Schooling and student’s epistemological beliefs about learning. Contemporary Educational Psychology, 18, 23–35.

Johnston, P., Woodside-Jiron, H., & Day, J. (2001). Teaching and learning literate epistemologies.

Journal of Educational Psychology, 93(1), 223–233.

Joram, E., & Gabriele, A. J. (1998). Preservice teachers’ prior beliefs: Transforming obstacles into opportunities. Teaching and Teacher Education, 14, 175–191.

Kardash, C. M., & Scholes, R. J. (1996). Effects of preexisting beliefs, epistemological beliefs, and need for cognition on interpretation of controversial issues. Journal of Educational Psychology, 88, 260–271.

King, P. M., & Kitchener, K. S. (1994). Developing reflective judgment. San Francisco, CA: Jossey-Bass.

Kitchener, K. S., & King, P. A. (1981). Reflective judgment: Concepts of justification and their relationship to age and education. Journal of Applied Developmental Psychology, 2, 89–116.

Kuhn, T. S. (1962). The structure of scientific revolutions. Chicago, IL: University of Chicago Press.

Kuhn, D. (1991). The skills of argument. New York: Cambridge University Press.

Kuhn, D. (1999). A developmental model of critical thinking. Educational Researcher, 28, 16–26.

Kuhn, D., Cheney, R., & Weinstock, M. (2000). The developmental of epistemological understanding. Cognitive Development, 15, 309–328.

Lakatos, I. (1978). The methodology of scientific research programmes. Cambridge, England: Cambridge University Press.

Laplante, B. (1997). Teachers’ beliefs and instructional strategies in science: Pushing analysis further. Science Instruction, 81, 277–294.

Lehrer, K. (1990). Theory of knowledge. San Francisco, CA: Westview Books.

2 Assessing Teachers’ Epistemological and Ontological Worldviews

43

Levitt, K. E. (2001). An analysis of elementary teachers’ beliefs regarding the teaching and learning of science. Science Education, 86, 1–22.

Lieberman, A. (1995). Practices that support teacher development: Transforming conceptions of professional development. Phi Delta Kappan, April, 591–596.

Lincoln, Y. S., & Guba, E. G. (2000). Paradigmatic controversies, contradictions, and emerging confluences. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (2nd edition, pp. 163–188). Thousand Oaks, CA: Sage Publications.

Mertens, D. M. (2005). Research and evaluation in education and psychology: Integrating diversity with quantitative, qualitative, and mixed methods (2nd edition). Thousand Oaks, CA: Sage Publications.

Olafson, L. J., & Schraw, G. (2002). Some final thoughts on the epistemological melting pot.

Issues in Education, 8, 233–246.

Olafson, L. J., & Schraw, G. (2006). Teachers’ beliefs and practices within and across domains.

International Journal of Educational Research, 45, 71–84.

Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of Educational Research, 66, 543–578.

Perry, W. G., Jr. (1970). Forms of intellectual and ethical development in the college years. New York: Academic Press.

Pollock, J. L. (1986). Contemporary theories of knowledge. Savage, MD: Rowman & Littlefield.

Popper, K. R. (1959). The logic of scientific discovery. London: Hutchinson.

Prawat, R. S. (1992). Teachers’ beliefs about teaching and learning: A constructivist perspective.

American Journal of Education, 35, 354–395.

Reybold, L. E. (2001). Encouraging the transformation of personal epistemology. Qualitative Studies in Education, 14, 413–428.

Schommer, M. (1990). Effects of beliefs about the nature of knowledge on comprehension.

Journal of Educational Psychology, 82, 498–504.

Schommer, M. (1994). Synthesizing epistemological belief research: Tentative understandings and provocative confusions. Educational Psychology Review, 6, 293–320.

Schommer-Aikins, M. (2002). An evolving theoretical framework for an epistemological belief system. In B. K. Hofer & P. R. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing (pp. 103–118). Mahwah, NJ: Erlbaum.

Schommer, M., Crouse, A., & Rhodes, N. (1992). Epistemological beliefs and mathematical text comprehension: Believing it is simple does not make it so. Journal of Educational Psychology, 84, 435–443.

Schraw, G., & Olafson, L. J. (2002). Teachers’ epistemological worldviews and educational practices. Issues in Education, 8, 99–148.

Schraw, G., & Sinatra, G. M. (2004). Epistemological development and its impact on cognition and academic domains. Contemporary Educational Psychology, 29, 95–102.

Schraw, G., Bendixen, L. D., & Dunkle, M. E. (2002). Development and validation of the Epistemic Belief Inventory (EBI). In B. K. Hofer & P. R. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing (pp. 261–275). Mahwah, NJ: Erlbaum.

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.

Slotta, J. D., & Chi, M. (2006). Helping children understand challenging topics in science through ontology training. Cognition and Instruction, 24, 261–289.

White, B. C. (2000). Pre-service teachers’ epistemology viewed through perspectives on problematic classroom situations. Journal of Education for Teaching, 26, 279–305.

White, B., & Frederiksen, (2005). A theoretical framework and approach for fostering metacognitive development. Educational Psychologist, 40, 211–223.

Wilcox-Herzog, A. (2002). Is there a link between teachers’ beliefs and behaviors? Early Education and Development, 13, 79–106.

Woolfolk-Hoy, A., & Burke-Spero, R. (2005). Changes in teacher efficacy during the early years of teaching: A comparison of four measures. Teaching and Teacher Education, 21, 457–471.

44

G. J. Schraw and L. J. Olafson

Woolfolk-Hoy, A., Davis, H., & Pape, S. J. (2006). Teacher knowledge and beliefs. In P. A. Alexander & P. H. Winne (Eds.), Handbook of educational psychology (2nd edition, pp.

715â&#x20AC;&#x201C;737). Mahwah, NJ: Erlbaum.

Yang, F. (2005). Student views concerning evidence and the expert in reasoning a socio-scientific issue and personal epistemology. Educational Studies, 31, 65â&#x20AC;&#x201C;84.