There’s No Place Like Home Re-discovering ourselves by making mindful connections to our community with natural mathematical questions
Project Loci Amanda Milewski Carolyn Hetrick Justin Dimmel Chandler Brown University of Michigan School of Education January 6, 2021
I regard it as the foremost task of education ensure the survival of these qualities: an enterprising curiosity, an indefatigable spirit, tenacity in pursuit, readiness for sensible self denial, and, above all, compassion. - Kurt Hahn
II
Table of Contents
Contents 03 05 09 13 21 39 67 68
Executive Summary Introduction Guiding Questions Understanding Student Experience Examining the Potential of Place-Based Learning Preliminary Impact of the Intervention Summary References & Appendicies
1
2
Executive Summary
Executive Summary For some time now, there has been a growing national concern about the number of high school and college students suffering from mental health issues, such as depression, anxiety, and eating disorders. These issues also often present with severe symptoms that interact with social and academic functioning. In summer 2017, the University of Michigan School of Education (U-M SOE) and Byram Hills High School (BHHS) faculty established a partnership to modify some aspects of instruction in selected academic subject classes so as to (1) maintain academic excellence and college entrance competitiveness while also (2) increasing student autonomy and self-motivation, with the aim of reducing the occurrence of excess stress and related disabling conditions. The mathematical core of the partnership is grounded in place-based learning, which is an instructional approach that connects learning and teaching to the local heritages, cultures, landscapes, opportunities, and experiences in students’ own communities. Place-based learning is a particularly good match for this project because there is evidence that students’ experiential engagement with real-life problems through service learning opportunities in their local communities can have real benefits not only for students academically, but also for their socialemotional learning. This report describes our work with mathematics teachers at BHHS to experience place-based learning—by participating in a week-long immersive professional development experience at a semester program—and then design place-based learning experiences that could be completed by students at Byram. We describe below the design, development, and measurement processes we engaged in to implement place-based learning at BHHS and to understand the effects of place-based learning on student experiences in school.
Place-based learning is a particularly good match for this project because there is evidence that students’ experiential engagement with real-life problems through service learning opportunities in their local communities can have real benefits not only for students academically, but also for their socialemotional learning.
3
Best, Brightest — and Saddest? New York Times // April 11, 2015
Today’s Exhausted Superkids New York Times // July 29, 2015
Why are more American teenagers than ever suffering f rom severe anxiety? New York Times // October 11, 2017
Pressure over college admissions ‘out of control’ Palo Alto News // March 14, 2019
Scandal lays bare stress of elite college admissions process Portland Press Herald // March 16, 2019 4
Introduction
Introduction For some time now, there has been a growing national concern about the number of high school and college students suffering from mental health issues, such as depression, anxiety, and eating disorders. These issues also often present with severe symptoms that interact with social and academic functioning. As part of an effort to both understand and proactively address these concerns, Byram Hills High School (BHHS) and the U-M Depression Center established a three-year collaboration in which U-M Depression Center provided BHHS professionals with training and resources to support their use of cognitive behavioral therapy and mindfulness. This initiative has been assessed as useful by BHHS staff for reducing stigma and giving students, teachers, and administrators a shared language as well as strategies for addressing mental health and wellness in general. Through this initial work, a consensus emerged that more targeted efforts were needed to modify classroom instruction in ways that improve student learning experiences in their core classes while also supporting student attainment on more standard measures.
An approach to instruction that adopts complex real-world problems and students’ natural questions as vehicles to drive student learning—an objective that already had a foothold among teachers at BHHS
5
Executive Summary
Setting the Course In summer 2017, the U-M SOE and BHHS faculty established a partnership to modify some aspects of instruction in selected academic subject classes so as to • maintain academic excellence and college entrance competitiveness while also • increasing student autonomy and self-motivation, with the aim of reducing the occurrence of excess stress and related disabling conditions. To inform the ways this work would unfold, the faculty at the U-M began this partnership by working closely with the mathematics department to explore possible solutions by • conducting a needs assessment—identifying the kinds of solutions the members of the mathematics department were initially most interested and willing to explore and • surveying the literature—identifying what is known about various ways to increase students’ academic autonomy and motivation within mathematics coursework.
What did we learn from the needs assessment? In the early discussions of the BHHS-U-M research practice partnership team, we considered two possible approaches known to be connected to supporting the growth of students’ autonomy and motivation while maintaining competitiveness on metrics such as college entrance exams: • standards-based grading and • problems-based learning. Standards-based grading breaks down subject matter into small “learning targets,” tracks students’ progress towards those targets, and provides students with regular feedback regarding their progress. Problem-based learning is an approach to instruction that adopts complex real-world problems and students’ natural questions as vehicles to drive student learning—an objective that already had a foothold among teachers at BHHS
Complex tasks that were utilized by math teachers at BHHS prior to the establishment of the U-M partnership Examples on pg. 7
Thus, the problems-based principles of discovery and exploration as well as the integration of knowledge across disciplines was already familiar for BHHS teachers. We decided that the project’s design and implementation of problem-based instruction could build on this momentum, and for the first year of the project, this was the focus of our collaboration.
6
Introduction Examples provided by teachers to represent tasks they were already using prior to the intervention
Discovering a formula (task originally from Phillips Exeter) We see a focus on students discovering trigonometric identities that are more typically told to students.
Collecting data about the temperature of coffee to discover Newton’s Law of Cooling Provides students the opportunity to explore exponential decay by observing a physical law
Examining the quadratic nature of the path of projectile objects Engages students in an exploration of quadratic functions by embedding that exploration in an engineering task.
In these tasks, we see a variety of ways that teachers were taking up problems-based learning with their students.
7
Introduction
What does the literature say?
Giving students the opportunity to
The literature on problems-based learning and its connections to students’ autonomy and motivation is vast (some of which is outlined below). To begin, there are several different kinds of problems-based learning (PBL), each with their own unique focus on where to gather inspiration for tasks. Some of the more long-standing PBL approaches take their influence from mathematics as it is done in the discipline while others take influence from mathematics as it is used by citizens in the real world. We see both of these influences in the tasks described above.
express their creativity and autonomy
A third approach to PBL is called place-based learning. This approach takes its influence from questions that students themselves would have as they immerse themselves in the local heritages, cultures, landscapes, opportunities, and experiences in their very own communities. Place-based learning is a particularly good match for this project because of its potential for integrating a PBL approach with the project’s overall focus on supporting the development of students’ well-being and emotional competencies such as self-awareness, social awareness, and responsible decision making. In particular, there is evidence that students’ experiential engagement with real-life problems through service learning opportunities in their local communities can have real benefits for students’ academic and social-emotional learning.
8
on rich mathematical tasks can foster originality, a collaborative spirit and can lead to a greater feeling of satisfaction during learning.
Furthermore, there is evidence that PBL with rich mathematical tasks can help foster and maintain students’ interest in and attitudes towards mathematics, sense of competence and autonomy, creativity and originality, and satisfaction during learning. On the other hand, a mathematics learning experience that privileges procedural learning at the expense of opportunities to explore and engage with mathematical concepts through rich tasks could lead to lesser engagement with mathematics during learning and even to attitudes of disillusionment with mathematics years after high school.
Guiding Questions
Guiding Questions The following three areas, and the accompanying questions, have guided the partnership work.
AREA 1 Understanding the BHHS Student Experience To begin, we were interested in understanding (without intervention): “What do the mathematical experiences of students at BHHS look and feel like?” In particular, we wanted to understand the following:
9
Guiding Questions
AREA 2 Assessing the feasibility of the intervention Next, we wanted to understand “To what extent are teachers able to take up place-based instructional practices in their work?” In particular, we wanted to understand, from the teachers’ perspectives: What barriers exist for taking up this kind of work? What supports seem to make a difference? In order to answer these questions, we enacted and studied the following:
10
Guiding Questions
AREA 3 Preliminary impact of the intervention on BHHS Student Experiences Finally, we wanted to know (given the intervention): “What do the mathematical experiences of students engaged in place-based work at BHHS look and feel like?” In order to answer these questions, we enacted and studied the following:
11
12
Understanding Student Experience
AREA 1: Understanding the BHHS Student Experience In order to address the first set of research questions, we engaged in two kinds of data collection: student surveys & student interviews. Here we briefly describe our methodological design and choices, describe the samples, and report on our findings.
Student Surveys As part of our plan to understand the variety of BHHS students’ experiences, it was important to select previously-validated instruments that measured the constructs of interest with relatively few questions (so as to not take too much instructional time for administration) and that would allow us to compare the experiences of those students that completed place-based units to those that did not. The TRIPOD and Attitudes toward Mathematics Instrument (ATMI) satisfied these conditions. The TRIPOD was designed by educational economists at Harvard University for offering streamlined, actionable feedback for teachers and school leaders. The TRIPOD was used as one of the nationally-administered instruments in the Measures of Effective Teaching Projects (MET and MET II) funded by the Bill and Melinda Gates Foundation. Those projects were focused on determining how to best identify and promote excellent teaching. The research from that project and others has demonstrated that better results on the constructs underlying the TRIPOD predict higher student achievement, engagement and motivation, as well as success skills and mindsets (e.g. Ferguson & Danielson, 2014; Ferguson et al., 2015; Kane & Cantrell, 2010; Kane et al., 2013; Stuit, Ferguson, & Phillips, 2013). From the TRIPOD instrument, we hoped to gauge students’: • Perception of mathematics instruction, including: perception of personal support: the extent to which students are comfortable in the classroom and perceive their ideas are valued and welcomed (TRIPOD constructs care and confer) perception of academic support: the extent to which students perceive the curriculum as engaging, accessible, and coherent (TRIPOD constructs captivate, clarify, and consolidate)
it was important to select previously-validated instruments that measured the constructs of interest
perception of academic press: the extent to which students perceive the environment as one that holds high expectations for them to achieve their highest potential (TRIPOD constructs challenge and classroom management) perception of peer support overall engagement & motivation overall success skill and mindsets The ATMI was designed by mathematics education researchers (Tapia & Marsh, 2000, 2002) and has been used internationally—with a wide variety of students (including students identified as “gifted and talented”). The ATMI has been used primarily by researchers to understand a variety of research topics, such as (1) the relationships between students’ mathematical attainment and their attitudes toward mathematics, (2) the differences between groups of students (e.g. male and female) in terms of their attitudes toward mathematics, and (3) changes in students’ attitudes toward mathematics over time (e.g. Hargreaves, Homer, Swinnerton, 2008; Karjanto, 2017). More generally, the positive scores on the constructs underlying the ATMI have been demonstrated to be related with students’ mathematical engagement and attainment (e.g. Hargreaves et al., 2008; Sundre et al., 2012); while negative scores have been shown to be related with students’ mathematical anxiety (e.g. He, 2007). With the ATMI instrument, we planned to gauge students’: • confidence in doing mathematics • enjoyment of mathematics • value for mathematics • motivation towards mathematics
13
Understanding Student Experience
Figure 4. Illustrates each of the above constructs from both the TRIPOD and ATMI with sample items.
Instrument Construct TRIPOD
Sample Item
Personal Support Care
My teacher makes me feel that he/she really cares about me.
Confer
My teacher gives us time to explain our ideas.
Academic Support Captivate
My teacher makes lessons interesting.
Clarify
My teacher knows when the class understands and when we do not.
Consolidate
My teacher takes time to help us remember what we learn.
Academic Press
ATMI
14
Challenge
My teacher makes sure that I try to do my best.
Classroom Management
Our class stays busy and does not waste time.
Engagement & Motivation
I have pushed myself hard to completely understand my lessons in this class.
Success Skills & Mindsets
In this class, students learn to be more organized.
Mathematical Confidence
I am able to solve mathematics problems without too much difficulty.
Enjoyment
I really like mathematics.
Value
Mathematics is a very worthwhile and necessary subject.
Motivation
The challenge of math appeals to me.
Understanding Student Experience
Survey Sample The TRIPOD and ATMI surveys were administered to students before and after participation in the place-based learning experience (Fall 2018 and Spring 2019). Multiple responses over time makes it possible to observe changes in perceptions during the learning experience. These pre- and post-survey instruments were administered to 317 and 265 BHHS students respectively. This was done in a way to capture students across a variety of grades, courses, and tracks (see Appendix A for description of the total sample). The largest difference in pre/post survey administration shows up across the courses with students in the 12th grade and we suspect has to do with running out of time due to the compressed schedule in those courses from seniors’ early release and additional end-of-year testing.
Multiple responses over time makes it possible to observe changes in perceptions during the learning experience.
For the purposes of assessing the first set of research questions, we have restricted our analysis to only those responses we gathered from the pre-survey (n=317)— leaving the analysis of the pre-post survey for the third section of the report. In our analysis of this pre-survey data, we observed some differences in students’ experiences according to track (Advanced, Accelerated, On-Track). More specifically, we observed significant differences across most of the 7Cs outcomes and all the ATMI for students in the Advanced and Accelerated tracks when compared with the on-track students (Model 1 in Table 1). However, when we accounted for differences among students’ engagement & motivation as well as success skills & mindsets (Model 2 in Table 1), we found that these positive results either disappeared or became negative for the 7Cs outcomes. For ATMI outcomes, these results changed only for the Advanced track. This means that, after accounting for differences in students’ engagement & motivation and success skills & mindset, the students in the advanced track report lower than expected levels of care (b=-0.346, s.e.=0.106) and consolidate (b=-0.215, s.e.=0.103) than their on-track peers (see Table 1). Students in the Accelerated track reported higher levels of enjoyment (b=0.413, s.e.=0.101), motivation (b=0.585, s.e.=0.131), and value (b=0.316, s.e.=0.145) than their on-track counterparts, and they also reported a lower average level of self-confidence (b=-0.313, s.e.=0.117).
15
Understanding Student Experience
Table 1. Student Pre-Survey Responses on 7Cs Outcomes ADVANCED
ACCELERATED
TRIPOD
Model 1
Model 2
Model 1
Model 2
Care
0.095 (0.125)
-0.346*** 0.106)
0.060 (0.115)
-0.173+ (0.094)
Confer
0.247+ (0.147)
-0.094 (0.143)
0.119 0.136)
-0.066 (0.126)
Captivate
0.516*** (0.112)
-0.025 (0.096)
0.148 (0.112)
-0.110 (0.089)
Clarify
0.438*** (0.133)
-0.038 (0.079)
0.244* (0.122)
0.017 0.070)
Consolidate
0.234+ (0.131)
-0.215* (0.103)
0.211+ (0.123)
-0.027 (0.091)
Challenge
0.310** (0.102)
-0.044 (0.081)
0.183+ (0.094)
-0.004 (0.072)
Classroom Management
0.428* (0.169)
0.087 (0.164)
0.073 (0.161)
-0.077 (0.158)
ADVANCED
16
ACCELERATED
ATMI
Model 1
Model 2
Model 1
Model 2
Enjoyment
0.228* (0.100)
0.039 (0.105)
.822 *** (0.062)
0.413*** (0.101)
Motivation
0.348* (0.146)
0.175 (0.145)
0.964*** (0.114)
0.589*** (0.131)
Self-confidence
-0.266** (0.101)
-0.125 (0.115)
-0.730*** (0.081)
-0.313** (0.117)
Value
0.293* (0.133)
0.089 (0.131)
0.669*** (0.105)
0.316** (0.145)
Understanding Student Experience
The care construct sits within the personal support panel and is defined by TRIPOD as follows: Teachers who care show concern for students’ emotional and academic well-being. They develop supportive, personalized relationships with students, cultivate an emotionally safe environment, and respond consistently to students’ social, emotional, and academic learning needs. The consolidate construct sits within the academic/curricular support panel and is defined by TRIPOD as follows: Teachers who consolidate help students integrate and synthesize key ideas. They summarize and make connections in ways that help students see relationships within and across lessons, remember ideas, and build understanding over time. Both constructs are posited to be an important part of essential elements of effective instructional practice. We find elements of these results compelling, given the original problem as framed by the school, with some amount of worry about the students enrolled in these top courses experiencing inordinate amounts of stress as part of their experiences there. That said, because of the non-random design of the data collection, we cannot warrant causal claims about this relationship. Beyond the two confounds that we checked, there could be other confounding variables in the data, such as students’ overall attainment. Our hope is that these gleanings would spur some conversation among the BHHS faculty about some of the underlying causes of these relationships that we observed.
Student Interviews In order to more fully understand the BHHS student experiences as well as help us interpret the results from these pre-surveys, we interviewed 2-3 students from each of the classes we surveyed. Since we wanted to understand the variety of student experiences at BHHS, we asked teachers to identify 1-2 students from each course they perceived as thriving as well as 1-2 that were not. We followed up with each of these students and their parents to gain permission and conduct the interviews. During the interview, we asked students to engage in three different kinds of questions. The first kind of question aimed at having the students describe their more typical and less typical activities, activity structures, and homework in their current mathematical class with questions like “Can you describe what a {regular/ atypical} day in your mathematics class looks like?”
The second kind of question asked students to describe their personal experience with mathematics problems with questions like “What about these kinds of {more typical/less typical} homework problems (if anything) do you enjoy?” “What about these {more typical/less typical} homework problems (if anything) frustrates you?” The third kind of question asked students to reason through some story problems not unlike the ones you would expect to see in a middle school mathematics textbook. We selected middle school content, rather than high school, because we could be reasonably assured that the mathematical content embedded in the story problem would be familiar for all of the students we interviewed. Furthermore, we are interested in understanding how BHHS students reason about mathematics as embedded in realistic contexts (as they would need to do in the case of place-based learning), rather than trying to understand how students handle mathematics concepts that may be unfamiliar to them.
Interview Sample In total, we have interviewed 38 of the 317 BHHS students we surveyed across a variety of courses (see Appendix A for more detailed description of sample). For the purposes of this section of the report where we aim to understand the BHHS student experience, without intervention, we restrict our analysis of the 38 total interviews strictly to those students who participated in the non-intervention courses. While we report some numbers of students or groups of students to give a sense of scale for our claim-making, we did not do a quantitative analysis of the interview data. Instead, we qualitatively analyzed interviews to get a thicker, more nuanced picture of students’ thematic experiences. Of the 38 students we interviewed, 22 students were from the non-intervention courses. Some further disaggregate information about these 22 students is listed in Figure 5. Note that, because the number of tenth-grade interviewees in non-intervention classes is so small (n=2), we do not include tenth-grade as a disaggregate group in our analysis. However, we do include these students in our sample for analysis when looking at the accelerated track disaggregate, to which they belong.
17
Understanding Student Experience
Figure 5 Grade level and course disaggregate for students in the non-intervention group (n=22)
18
On-Track
Accelerated
Advanced
Total
Ninth Grade
0
5
1
6
Tenth Grade
0
2
0
2
Eleventh Grade
2
1
3
6
Twelfth Grade
2
3
3
8
Total
4
11
7
22
Understanding Student Experience
Describing a Typical Day Most non-intervention students we interviewed described typical class days following an arc of (a) having an opening task or set of problems, (b) learning the new content through teacher exposition and class dialogue, and (c) practicing concepts in groups and/or independently. When asked “What does an atypical day look like in your math class?,” students in the non-intervention classes included the following categories of activities in their descriptions of out-of-theordinary days: tests, group work, free work days, review, presentations, and special labs or projects. However, students in this group most often identified days with group work or free work as being out of the ordinary. For example, one student shared that “There [are] some days where we get a group assignment and then we split up into separate groups, like usually partnered groups and we should try to fill out a packet. It could be graded, it could just be for practice.” Much like other respondents, this student clarified that an “out of the ordinary day” was rare, estimating that something like the group work activity described above would occur about twice during a semester.
When asked about the balance of teacher-talk and student-talk in their math classes, about 40-percent of respondents (n=9 of 22) in non-intervention classes said that the balance of talking was about even between teachers and students on any given day. However, respondents in the accelerated track more consistently identified that students did most of the talking in their classes, while respondents in the advanced track more commonly identified that the teacher did most of the talking in their classes. There was no clear variation in patterns of teacher- and student-talk across grade level disaggregate responses. To elicit student descriptions of typical math class experiences, we also asked interviewees about how much time it typically took them to do their math homework. Non-intervention students we interviewed reported an overall average range of 28-47 minutes. Figure 6 details variations in average ranges and minimum and maximum reported times disaggregated by course tracks. Students’ responses disaggregated by their grade level did not reveal a meaningful difference in their answers.
Figure 6 Minimum, maximum, and average ranges of student time spent on homework, by course
19
Understanding Student Experience
Impressions and Experiences of Math Classes Students we interviewed enrolled in non-intervention classes reported enjoying several and varied elements of their mathematics learning, including feelings of “getting it,” being challenged, having an easy time understanding concepts, and the satisfaction of problem-solving. There was some variation between what students in the separate track disaggregates most often identified as enjoyable. Overall, students in the non-intervention advanced and accelerated tracks more often reported enjoying affective-cognitive aspects of mathematical learning (e.g. challenge, ease, complexity). On the other hand, students in the non-intervention on-track classes more often reported enjoying processes of mathematical learning (e.g. collaborative work, class activities). When looking at non-intervention students’ responses by grade level, we found that ninth- and tenth-grade students were more likely to share that they enjoyed feeling a sense of ease or understanding in math; this theme did not emerge nearly as much in the eleventh- and twelfth-grade students’ responses. When we asked students, “What, if anything, frustrates you about [typical problems in your mathematics class]?” there were some interesting variations in responses across grade-level and track aggregates. Specifically, advanced track students were more typically frustrated by what they perceived as unclear wording or instructions, on-track students were more typically frustrated by not understanding the mathematics of a problem, and students in the accelerated track were comparatively more likely to be frustrated by the multistep nature of some problems. Ninth-grade students in non-intervention classes also commonly reported being frustrated by the multistep nature of some problems, whereas eleventh- and twelfth-grade students were more likely to be frustrated by a specific topic.
20
A supermajority of all students interviewed across both track and grade-level disaggregates shared that they considered mathematics homework to be an important part of their learning. Student responses also suggested that most students make some sort of distinction between the value or purpose of mathematics homework from those of homework in other subject areas. In particular, students named mathematics homework as an important part of practicing and testing their understanding of concepts. When we asked students if their mathematics learning in school was related to the real world, an interesting theme emerged in their responses. Of the non-intervention participants, over 75% (n=17) of them affirmed a connection between mathematics and the real world. However, only about 18% (n=4) of them volunteered that word problems were an example of how mathematics learning in schools is related to the real world. Other responses included references to particular subject-specific topics (e.g. calculating rates) or careers (e.g. engineering). This suggests a possible disjuncture between the common intention for word problems to serve as grounded applications for mathematical concepts and students’ perceptions of them.
Examining the Potential
Area 2: Examining the Potential of Place-Based Learning
Details about the interventions & assessing the feasibility of the intervention
In order to support teachers in making shifts in their instruction, the UM faculty provided BHHS with one day of onsite training, three days of immersive training, seven sessions of support for lesson planning across the 2018 calendar year, provided onsite support for teachers’ implementations of those lessons during the 2018-2020 school years, documented those implementations, and constructed reflection activities for those teachers that helped design and/or implement the lessons to support the refinement of the final versions of the lessons. A fourth site visit was planned for Spring of 2020 but ultimately canceled due to the national shuttering of classroom instruction during the COVID-19 pandemic. In total, the BHHS-UM RPP team worked on 4 placebased lessons—developing and implementing 3 of them into the pre-calculus course. The refinement of the lessons has also been documented into a library of place-based lessons for secondary mathematics teachers. Rather than using the typical written form which provides readers with an outline or “bird’s eye” view of the lesson, we have elected to represent the lessons using a storyboard medium which has the affordance of documenting the lesson at the momentto-moment “ground level.” The artifacts of these storyboarded lessons can provide continued support to BHHS teachers, and also to teachers beyond the BHHS district interested in learning more about what a placebased lesson could look like in a secondary mathematics classroom. Below we provide more detail about the work outlined above and provide some details about the ways we assessed the feasibility of the intervention work with teachers.
Teacher Intervention: PD at a Place-based Semester Program In order to support the BHHS teachers to gain more facility with place-based instruction, it was crucial to begin with a more immersive experience for teachers. We had hoped that such an experience would allow teachers to both: 1) experience placebased learning for themselves—getting a chance to pose natural questions while immersed in a particular place and do some authentic mathematics in order to explore those questions (see Figure 7) ; and 2) observe teachers and students engaged in authentic placebased learning (see Figure 8). Thus, we identified an environment, a place-based semester program, that allowed BHHS teachers to engage as mathematics learners and connect with and observe semester program faculty who have built a program around engaging students in a place-based curriculum. The semester program is a leadership and outdoor education academy whose organizing principle is place-based learning. We had anticipated that such experiences would support teachers to make shifts in their knowledge, beliefs, and dispositions towards place-based learning as well as provide them with ideas for how they themselves might design place-based learning opportunities for their students.
21
Examining the Potential
Figure 7
One of the BHHS mathematics teachers using a sextant to take a sighting of the sun. The use of the instrument was part of an immersive professional development experience in which teachers were tasked with figuring out “Where in the world are we?” in terms of longitude and latitude using concepts from spherical trigonometry.
How amazing [was] it for me to—for the first time—to really think about math and learning math in a place and asking questions in a place ... now we can explore that idea in our own locale.
22
Examining the Potential
Figure 8
BHHS Teachers observe semester program students in their mathematics class.
I think that some of the things that stood out for me is just the type of creativity that can be sparked from being in a new place. I think [it] makes your brain think in so many different ways than a problem that might be more straightforward—like if you were to just solve it in a math textbook—[it] automatically became something that was significantly more complex.
23
Examining the Potential
From these quotes, we gather the sense that the opportunity to do mathematics in this way was both novel for teachers as well as surprisingly challenging. This aligns with our own experiences working with teachers, who generally have a lot of experience and expertise solving problems as they are typically posed in mathematics textbooks, but have less experience doing mathematics in a way that takes as a starting point questions that emerge from one’s own experience of being in the world. If teachers are to engage students in place-based work, it is important that they themselves have opportunities to do this kind of work (Silver, 1994; Gonzales, 1996). Finally, it was in the context of this lesson that the teachers began imagining one of the four place-based lessons they could implement with students in BHHS: the longitude problem. Both the teachers’ accounting of their own experiences as well as the ways those experiences influenced their planning of place-based lessons suggest the importance of teachers’ engagement in immersive experience for supporting teachers to implement new kinds of lessons with students.
We see in these two quotes the teachers’ realization that students’ confidence and dispositions toward learning (engagement, willingness to try/explore, motivation toward learning) are intrinsically linked with the kinds of experiences students have the opportunity to engage in. And while these ideas are well documented in the literature about teaching and learning, we argue that it is impactful for teachers to see such links in action. Beyond seeing such connection, it was in this observation that two of the BHHS teachers began dreaming up ways to revise a previously developed problems-based lesson (called the Bottle Lab) to connect it with issues of flooding at BHHS. The ways that teachers described their experiences as well as the ways that these experiences gave birth to revisions of current lessons suggests that distinct from engaging in place-based learning themselves, there are important supports for teacher learning embedded in opportunities for teachers to observe teaching and learning contexts in which place-based tasks are being implemented.
We were interested in understanding what the BHHS teachers took away from their observations of classrooms at the semester program. From the teachers reports, it seems that these observational experiences supported them in ways unique from their own immersive experiences. When asked what stood out from the experience of observing the semester programs’ students engaged in their mathematics class and research class, one teacher remarked: So one of the main things that stood out to me about the [semester program] experience was initially how much confidence the students had, uh trying new things … observing their classes, it just seemed to permeate a willingness to explore and try things. And so many times the engagement problems that we may have in a classroom didn’t seem to be a problem there.
24
Examining the Potential
Lastly, we had hoped to understand the ways that the whole professional experience at the Semester Program impacted teachers as well as the kinds of things they hoped to take back with them as they reengaged back at Byram Hills. At the conclusion of the professional development about the impact of their experience at the Semester Program, the teachers were quite expressive about the meaningfulness of the experience for their own professional growth, saying things like: • A once-in-a-lifetime opportunity for me • Truly the most meaningful professional development experience I’ve had in my teaching career over 20 years • I’ve had many opportunities for professional development over my life but nothing is going to stick with me like this experience
But more than that, the teachers also identified ways that this experience would transform their instructional practices back in Armonk saying things like: • Even now just being back for a few days I’m a completely different teacher with a different outlook on my teaching • Just tying learning to experience I think strengthens [learning} ...for me as a teacher ... the same would be true for kids. So getting outside and relating mathematics and seeing mathematics in a real setting where [students are] outside of the classroom is going to reinforce what they learn • [This experience] just has me really inspired to make some big changes in my teaching and I think these are attainable changes—things that I could actually do • We are ready to be bold ... these are our ideas … let’s bring [them] back and get all of Byram involved
• It was just an awesome experience for [us] and I think [we] are clear on the vision
Finally, teachers were making connections between the kinds of experiences they were imagining for the BHHS students and how place-based experiences might help students take more autonomy in their learning: The things that really stuck out to me at the [semester program] that experience that I really, really want take back with me here to Byram Hills is just the beginning to place-based learning. Being—and just being somewhere or your senses of sight, smell, touch anything just inspire you to really be creative and and really ask questions really ponder. Cuz I feel like a lot of our students here now they feel confined to the four walls of the classroom, and they’re confined to curriculums, and they are confined to textbooks. Their intellectual curiosity isn’t where it could be if we do happen to just put them into a place like we were on day one where we went out to the sandbar—like just the questions that were going through my mind is almost overwhelming and then we got kind of narrowed it down to “where exactly where are we?” And then we took it and ran it from there. It was like really cool. And I really want to have my students kind of take ownership to their learning more.
When asked about what lessons might stick with them from that experience of observing semester program students, the same teacher said: There were so many other extra lessons the students were learning in addition to [mathematical] content which would really make them— sounds cliche —but lifelong learners, kids that have that thrive for learning. Having [students] realize that the learning doesn’t have to happen simply in a classroom— that it carries with you when it goes with you everywhere you are. So that natural experience of “Hey let me look at this” isn’t limited to just four walls or a specific classroom..
25
Examining the Potential
26
Examining the Potential
In the weeks following the trip, we began getting reports from the teachers that they were putting into action some of the ideas they had begun developing while at the semester program in their courses (see Figure 9a and 9b).
Figure 9a.
Algebra 2 students drawing graphs on the pavement in chalk and used their own bodies to create the third dimension in one teacher-developed place-based math lesson (Spring 2018).
Figure 9b.
Pre-calculus students on the football field using string and their bodies to represent vectors in another teacher-developed place-based math lesson (Spring 2018).
27
Examining the Potential
Teacher Intervention: Collaboratively designing place-based lessons In order to support the BHHS teachers as they started to design place-based lessons for implementation starting fall 2018, the University of Michigan team used a process called StoryCircles (Herbst & Milewski, 2018; Milewski, Herbst, & Stevens, 2020; Milewski, Stevens, Herbst, & Huhn, 2021). StoryCircles is an interactive form of professional education developed at the University of Michigan that engages practicing teachers in using a storyboard medium to collaboratively develop lessons using iterative phases of scripting, visualizing, and arguing about how a lesson could unfold (see Figure 10a).
28
Like the immersive PD, we expected this process to provide teachers with opportunities to grow professionally (Milewski, Herbst, Bardelli, Hetrick, 2018; Ko, Herbst, Milewski; 2020). Specifically, we anticipated that StoryCircles would provide teachers the chance to (1) engage in virtual professional experimentation— trying out and revising new instructional practices first with colleagues before actually implementing them with students and (2) in so doing have opportunities to gain new knowledge, beliefs, and dispositions about teaching. Across four summer sessions, the teachers generated four place-based lessons focused on local, national, and global issues (see Figure 11).
Figure 10a.
BHHS teachers meet with the University of Michigan team in July 2018 to begin developing the place-based version of the Bottle Lab which focused on flooding in Armonk. .
Figure 10b.
BHHS teachers work with the University of Michigan team in October 2018 to anticipate various ways students could redistrict a hypothetical district in more and less fair ways for the Gerrymandering Lab
Examining the Potential Figure 11. Description of the four teacher-generated place-based lessons Problem Context: Lesson Title:
Local - Flooding in Armonk and at BHHS. Solving a local engineering problem. Bottle Lab: Soil Composition
Lesson Students will head out to the field to inspect the flooding problem and identify possible Abstract variables that might affect drainage: Soil permeability They will test their hypotheses in a lab setting (including setting up intervention/nonintervention conditions for different materials), programming arduino sensors, gathering and representing data to draw conclusions from. Underlying Data Collection, Modeling, Fitting Functions to Real-Life Data Mathematics
Problem Context: Lesson Title: Lesson Abstract
Local – Flooding in Armonk and at BHHS Topography Lab Students will head out to the field to inspect the flooding problem and identify possible variables that might affect drainage: Topography. Students will learn to take measurements using trigonometry, read topographical maps, and model local topography in 2- and 3-dimensional topographical models using 3D printers.
Underlying Measurement, Trigonometry, Modeling, Gradient Function Mathematics
Problem Context: Lesson Title:
National – Gerrymandering. Using math to think about an enduring political issue Gerrymandering Lab
Lesson Students will be engaging with the recent national debates about defining the Abstract practice gerrymandering. Students will inspect how district lines are divided in Armonk and New York to gauge the “fairness” of recent election results. Underlying Sampling, Geometry of segmenting a plane Mathematics
Problem Context: Lesson Title:
Global – Locating oneself in the world. Understanding the role of mathematics in resolving a longstanding problem of circumventing the globe. Longitude Lab
Lesson Students will be studying the historical role of spherical trigonometry in resolving one of the Abstract most challenging, longstanding problems of modern history: longitude Along with reading excerpts from Endurance and Longitude, students will have a chance to learn how to locate themselves in the world using common mariner’s techniques (noonday sighting) and tools (nautical almanac, clock, and sextant). While this current issue is easily resolved using the Global Positioning System underlying smartphones, these skills are still required of all military personnel in the case of global emergencies where such systems might be compromised. Using these measurements, students will learn how seafarers from the past and today are able to say (without an iphone) with precision where in the world they are at any given moment. Underlying Spherical Trigonometry, Modeling Mathematics
29
Examining the Potential
After each of the summer planning sessions, the UM team represented the lesson as described by the teacher using storyboard representations See Figure 12—inviting teachers to specify more details about the lesson and make revisions to the lesson the teachers had laid out in those sessions. Finally, in the weeks and days leading up to implementations of the lessons, the UM team provided last minute consulting with the team (conducted both virtually and in person) to focus on final details and providing the team with feedback.
Figure 12. Selected frames from the storyboarded representations of lesson plans the teachers had shared in 2018 planning meetings.
Bottle Lab
30
Topography Lab
Examining the Potential
Instructional Intervention: Planning and implementation of place-based lessons at BHHS Two of the four lessons (Bottle lab and Gerrymandering lab) were implemented in the precalculus courses in early fall 2018. These particular implementations were not documented by the UM team.
water. To do this, students were taken outside prior to the lab to discuss various reasons why flooding may occur at Byram Hills High School. During that outdoor trip, students discussed many reasons why flooding occurred, including factors related to both weather, topography, and soil composition.
The Bottle lab was implemented again by 3 of the 4 teachers in December 2018 with 5 sections of the precalculus courses. In that lesson, students were asked to explore how soil composition impacted the flow of
Gerrymandering Lab
Longitude Lab
31
Examining the Potential
In the following week (on a particularly rainy day, when there happened to be flooding in the Byram Hills hallways), the students engaged in the lab in which they examined how soil composition might impact flooding. To do this they explored how different sized rocks, placed at the bottom of bottles, impacted the flow of water—filling soda bottles with water and then using arduino sensors connected to laptops to gauge changes to the height of the water in the bottle. During the implementation (see Figure 13), the project team noted ways that the teachers made incremental improvements to the lesson from the first to the last hour. Some of these included tweaking: (1) the ways they managed students as they entered the class (directing them where they should sit given the organization of the lab equipment), (2) the ways the directions for the lab were given to students (providing clearer
instructions about the overall structure of the day), (3) the ways in which the technology and materials were implemented and described to students (with a greater focus on the ways the materials connected to mathematical concepts), and (4) the ways they structured students’ discussions (taking time to have students document their ideas about the lab’s topic prior to jumping in with more technical instructions). Later that Spring (April 2019) the Gerrymandering lesson was implemented in the AP statistics class. In that lesson, students were asked to interact with a set of game boards used to represent a hypothetical district and three distinct sets of rules outlining different ways to parse the district up (see Figures 14a and 14b). The objective of the game was to provide the students some concrete context from which to think about what could be meant by “fair” in the context of redistricting a region.
Figure 13 December 2018 implementation of the Bottle lab lesson with one of the BHHS teachers helping students set up their arduino sensors
Figure 14 Image of gameboard used by teachers for Gerrymandering Lab. 32
Examining the Potential
During the implementation of the Gerrymandering Lab (see Figures 15a and 15b), students took turns going to the board to represent various ways that one might parse up a hypothetical district (represented in Figure 15a with a student demonstrating one of the ways to break the district up on the Smart Board). After completing the game boards, students were asked to argue about which ones of these sets of rules created fairer conditions for breaking up the district (represented in Figure 15b, with the students sharing their reasoning with the class). Unfortunately, there was only one section of AP statistics and so there was not an opportunity to implement the lesson on more than one occasion across the day. Shortly after the implementation of the Gerrymandering lab, the Longitude Lab was implemented in the Advanced Pre-Calculus course (May 2019, see Figure 16). Prior to the actual lab day,
the students were taken outside to discuss some of the background concepts—including reading and roleplaying scenes from Alfred Lansing’s Endurance: Shackleton’s Incredible Voyage to the Antarctic. When outside, students were introduced to some elements of nautical navigation through the retelling of Shackleton & Worsley’s monumental journey from Elephant Island to South Georgia to save a crew of men. This included learning about some of the basic tools (e.g., Nautical Almanac, sextant, chronometer) and techniques (e.g., taking sightings of celestial bodies, using local apparent noon and declination to calculate one’s latitude) used by the crew to locate their position on the Earth. On the following day, back in class, students explored models of equatorial and celestial coordinate systems used in nautical navigation—deriving some of the spherical trigonometry formulas used in such work.
Figure 15b BHHS teacher reacts to one of the arguments presented by a group of students about which of the districting methods was more “fair”—with students considering different ways to operationalize “fair” using concepts they had learned in statistics
Figure 16 May 2019 implementation of the Longitude lab lesson with a BHHS teacher helping students build a physical model of a global positioning system.
33
Examining the Potential
Like the Gerrymandering Lab lesson, there was only one section in which this lesson could be taught and so learning from the implementation was somewhat constrained. That said, even prior to implementation, the advanced pre-calculus teacher was already considering ways she might modify the lesson for future implementations to allow for a more immersive approach—including planning a field trip to the ocean front in Spring of 2020 that was unfortunately cancelled due to COVID-19. Following each of these implementations, the UM team organized segments of video into an annotation tool and shared them with teachers to reflect on the various aspects of the implementations. Using this tool, teachers identified ways the various implementations could have been improved. Using teachers’ initial ideas, as expressed in annotations on the lesson videos, we constructed new versions of the lesson storyboards. These storyboards were again provided to the teachers—inviting teachers to specify more details about the revisions they envisioned for the lessons. We also, with teacher permission, shared the storyboards with education faculty members at the University of Michigan and beyond.
34
Documenting lessons as a way to help expand the knowledge base for using a place-based approach in secondary mathematics As part of our goals to support place-based efforts in secondary mathematics classrooms both within Byram and beyond, we used the comments on these storyboards as the catalyst for constructing a library of storyboarded lessons that illustrate various ways secondary mathematics teachers can implement some of the place-based lessons developed in the context of the project. For each lesson, we have used the ideas raised by both the project teachers and university faculty to represent two distinct ways that the lesson could unfold. In each case, we elected to represent one storyline to stay quite close to either what happened in the lesson implementation or what the project teachers indicated would be feasible to do in future implementations. The other storyline takes up ideas suggested as laudable by either project teachers or university faculty, but we suspect might also be reasonably dismissed by teachers, more generally, as “far flung” in that they represent a version of the lesson that could be quite challenging to take up. To inform our notions of what might be challenging to take up, we turned to our field notes from across the project and the things we learned from the project teachers. We identify 3 areas of challenge for taking up place-
Examining the Potential
based lessons in secondary mathematics classrooms. Below we outline these challenges and illustrate them by considering how they could arise within the three lessons teachers developed. We also share a select set of frames from the two storyboarded versions of the three lessons to help make these challenges more tangible (see Figure 17 as well as Figures B1 and B2 in Appendix B). On the left-hand side of these figures, we represent how the lesson under discussion might play out if the challenge were not so central to the work of teaching mathematics in school— representing a lesson somehow unshackled from the consideration named. On the right-hand side of these figures, we represent how the lesson under discussion might play out when a teacher needs to attend to or accommodate a particular obligation/challenge. Institutional Challenges: As part of their roles, teachers have an obligation to attend to the norms and structures outlined by the institutions that make room for their instruction. This obligation, called the institutional obligation, goes with teachers even as they may seek opportunities to carry their instruction beyond the walls of their classrooms. Common institutional constraints for teachers include department resources, school calendars and time schedules, district assessments, and employment contract rules. The implementation of place-based lessons can create unique sets of
institutional challenges for teachers. For example, there are significant warrants for the implementation of an immersive version of the Horizon Problem as certain elements of the problem posing and problem solving that are challenged without visual access to the horizon. For example, the chance to wonder how far away the horizon is emerges quite naturally from those times when one is immersed within one of those rare locations where one has an unobstructed view of the horizon (such as standing on the shore of a very large body of water, standing in a very flat location—such as the salt plains, or elevated on some location where natural or manmade structure). Furthermore, the chance to consider how altitude might make a difference in how far one can see can be experienced in certain locations by simply stooping and noticing objects on the horizon disappearing. However, in a traditional school setting where field trips introduce logistical, safety, and time factors, this kind of place-based activity is less feasible. With consideration of these kinds of institutional challenges, teachers may elect to rework the Horizon Problem to engage similar naturalistic forms of inquiry without leaving the classroom. However, this kind of reworking can attenuate the immersive experience of place-based learning, showing how institutional constraints are factors in implementing this kind of lesson.
35
Examining the Potential
Figure 17. Frames from two versions of the Horizon Lesson that demonstrate possibilities for more institutional challenges (left) and fewer institutional challenges (right).
Possibilities for more institutional challenges
36
Examining the Potential
Possibilities for less institutional challenges
37
Examining the Potential
Disciplinary Challenges: Particularly in secondary levels of schooling, teachers have the obligation to be a representative of the knowledge and practices valued in their content field. Teachers have disciplinary obligations to the conventions and knowledge within their subject area. For example, a mathematics teacher confronts a disciplinary challenge when choosing which math problem to provide as an example in class. While many solutions may meet the superficial criteria of the content, the mathematics teacher invokes their disciplinary knowledge to know how and why some problems may be more favorable to others in illustrating a particular mathematical concept. Place-based lessons, often by nature, take up problems that span many disciplines. For example, in the context of the Longitude Lesson, there is an opportunity to engage in some substantial mathematics related to spherical trigonometry. As such, this lesson provides opportunities for students to engage mathematics useful for modeling and computing with precision one’s location on earth. However, to get to the statement of the problem as well as the solution in a compelling and coherent way, one might also need to touch on topics traditionally covered in other disciplines such as: (a) the history of maritime navigation (usually covered in social studies or literature) that lead to (b) the establishment of the global coordinate system of longitude and latitude (also covered in social studies) that is based on (c) a larger celestial coordinate system based on the predictable locations of celestial bodies such as the sun and moon, but also stars (usually covered in astronomy). While these topics make the problem and its solution more understandable for the students, they challenge the teachers’ obligation to do justice to their own discipline of mathematics as they take time away from topics that are more easily recognizable as mathematics. An illustration of the ways that disciplinary challenges might hypothetically play out in the context of the Longitude Lesson, is provided in Appendix B (Figure B1).
38
Interpersonal Challenges: Even in classrooms with low student-to-teacher ratio, part of the work of teaching is balancing each person’s needs in ways that are constructive to the classroom community. Teachers have a professional obligation to ensure that all members of the classroom are sharing resources (including both material and immaterial) appropriately. The teacher’s interpersonal obligation involves managing social and cultural considerations, and the use of place-based curricula can create challenges related to some of these considerations. For example, while the Gerrymandering Lesson can create opportunities for the teacher to talk about how math relates to larger societal issues related to justice and equity, it is also a context that may present the teacher with real interpersonal challenges to deal with in the classroom. Specifically, it is reasonable for teachers to have concerns that they may have trouble facilitating a safe dialogue around very real, heated political issues for which there is no widespread societal narrative or consensus. Furthermore, mathematics teachers are not typically trained in the facilitation of such discussions. In short, there could be a very understandable source of concern for a teacher using such a lesson in that the engagement in mathematics with an explicit link to politics could, if not facilitated skillfully, potentially harm classroom relationships, alienate some students from engaging in the mathematical content, or introduce other interpersonal frictions that might be difficult for the teacher to resolve. An illustration of the ways that interpersonal challenges might hypothetically play out in the context of the Gerrymandering Lesson is provided in Appendix B (Figure B2).
Preliminary Impact
Area 3: Assessing the preliminary impact of the intervention on teachers and students In order to address the last set of research questions, we relied on both student survey and interview data. Here, we describe our methodological design and choices, including our samples for analysis, and share our findings. We conclude this section by sharing findings from the mathematics portion of student interviews and connecting those findings to suggestions for future work and implementation around place-based mathematics.
Research Questions related to Survey Data: 1. What changes in students’ experiences in mathematics class and attitudes toward mathematics occurred during participation in a place-based learning experience? 2. What is the relationship between participation in a place-based learning experience and changes in students’ experiences in mathematics class and attitudes toward mathematics?
Results from the Pre-Post Survey Analysis: Sample For the purposes of this analysis, we constrained our sample of students that completed both pre- and postsurvey instruments. We defined “completed” as those students who submitted answers to the final survey question for both pre- and post- surveys. Taking this as our definition for “completed” means that we included students in the sample that may have elected to skip particular survey questions. To handle such omissions, we filled those responses with a “0” and adjusted for the presence of missing data in the analysis (see Appendix C for discussion of missing responses and Figure C1 for description of missingness for ). Omitting students who did not respond to the last survey question on either the pre- or the post-survey left 236 Byram students in the sample, 87 of which participated in the place-based learning intervention and 149 who participated in other courses at Byram (see Figure C2 in Appendix C for effective sample).
Analysis Methods related to Survey Data For evaluation question one, we used t-tests to determine if observed changes in experiences in mathematics class and attitudes toward mathematics were statistically different from zero. For this first research question, we limited analysis to those who participated in the place-based learning experience in order to directly respond to the first evaluation question. Because scores on the Tripod and ATMI scales generally had a strong relationship with each other, we conducted t-tests simultaneously rather than sequentially, which limits the risk of incorrectly finding changes in students’ attitudes toward mathematics during participation in the learning experience. For evaluation question two, we used regression analysis to estimate the relationship between participation in the place-based learning experience and changes in students’ experiences and attitudes. As for the t-tests, we conducted all regressions simultaneously rather than sequentially to limit the risk of incorrectly finding a relationship between the learning experience and students’ reported attitudes. Further, we adjusted estimates of the relationship between changes in student attitudes and participation in the BH intervention courses for other relevant factors such as prior mathematics achievement and academic track. To determine which variables were likely to affect our estimate of the relationship between program participation and changes in student experiences and attitudes, we used MANOVA. Using this analysis, we identified variables that were predictive of scores on the scales of student attitudes toward mathematics. Variables that had a relationship with the outcome were grade, gender, and academic track. In order to obtain an accurate estimate of the relationship between program participation and changes in experiences and attitudes, we included these variables in the regression analysis.
Results from the Survey Changes During Participation T-tests showed some changes in students’ experiences in mathematics class and attitudes toward mathematics. Significant changes in scores were present for four of seven Tripod scales and for zero of four ATMI scales. As shown in Figure 18a, students reported an increase in confer (0.241 points, p = .005), captivate (0.486 points, p < .001), clarify (0.193 points, p = .006), and consolidate (0.241 points, p = .006).
39
Preliminary Impact
Relationship between participation and changes Regression analysis provided statistical evidence of a relationship between participation in a placebased learning experience and changes in students’ experiences in mathematics class and attitudes toward mathematics. Specifically, participation in the learning experience was associated with less change on the Tripod care scale and greater change on the Tripod captivate scale (see Figure 18a). In greater detail, students who participated in the intervention classes reported an average change in captivate score 0.33 points greater than students who did not participate in those classes. Further, students who participated in the intervention classes reported an average change in the care score 0.5 points less than those who did not. While participation in the learning experience was not associated with change on the ATMI sub-scales, we do note that for those who participated in the intervention there was a smaller decrease in mathematical motivation and a larger increase in mathematical value (Figure 18b).
What’s happening in math class? Research Purposes related to part one and two of the interview data For our following analysis of the data collected within the first two sections of the interview (all of the interview with the exception of the three math problems), we analyzed interview responses from all 38 participants (n = 38). While we examined disaggregate groups in our analysis, we primarily focused on patterns that emerged in comparing responses from students in the intervention classes to those from students in the non-intervention classes. Overall, our aim was to describe and explain:
40
• the ways that Byram students perceived their mathematics classrooms • any differences we observed in students’ perceptions when we disaggregate the data according to students’ participation in the intervention classes • the ways that Byram students reacted to various story problems that provided them with opportunities to make sense of the realistic context in which the mathematics was embedded • any differences we observed in students’ tendencies to suspend or engage in sense making when we disaggregated the data according to students’ participation in the intervention classes We organize some of the things we noticed from our analysis of data gathered in our interviews with students according to the following categories: • types of interactions described by students, • equity of voice, • comparisons between typical and atypical days, • connectedness of work to real world applications, • appraisal of homework practices • overall attitudes about mathematics.
Types of interactions described by students Across the first half of the interview, we asked students a series of questions about how things went in their mathematics class, on both typical and atypical days. Here we focus on the data gathered from students’ descriptions of a typical day (with the atypical described below). We coded that data using a framework that we adapted from Lemke (1990) which aimed to characterize the various structures that can commonly be observed across classroom interactions (see Figure 19).
Preliminary Impact
Figure 18a.
TRIPOD 7-C scales demonstrating ways that students who participated in the intervention classes scored differently than those who did not
Figure 18b.
ATMI 4 sub-scales demonstrating ways that students who participated in the intervention classes scored differently than those who did not
41
Preliminary Impact
Figure 19. Types of interaction segments coded in interview data
Situation Type (Interaction Segment)
Definition
Do Now activity
Pre-lesson activity designed to engage students in an introductory warm-up to the day’s central content
Going over homework
Activity in which student-teacher discourse (in any configuration) is focused on homework
Review
Activity focused on reviewing past instructional material or content
Teacher exposition and/or teacher demonstration
Activities in which the teacher is leading, discursively, to explain or show content/materials
Boardwork
Activities in which a combination of the teacher and students perform mathematics on the board and that boardwork constitutes a central focus of the class discourse
Class dialogue
Class dialogue includes triadic dialogue (in which the teacher facilitates, mediates, and responds to students’ dialogue) and other forms of typical classroom dialogue (e.g. teacher posing questions to class, students engaging in whole-class discussion)
Student group work
Students work in groups with other students (of varying sizes)
Student seat work
Students work in their seats, largely independently
While such categories might initially strike the reader as quite banal, we join others in suggesting that such structures have the potential to send subtle messages to students about what it might mean to “do mathematics.” The interaction types act as a kind of container for students’ engagement with the content, one another, and their teacher. Like containers, these structures both afford and constrain certain kinds of participation. According to Lemke (1990), these interaction types “are not just form, they are part of the content, part of the message” (p. 19) teachers send to students about what it means to engage with mathematics. 42
Preliminary Impact
Figure 20. Two illustrative quotes from Byram Hills students describing a typical day in the mathematics classroom
Excerpt from an interview with student assigned to an intervention classroom
Excerpt from an interview with student assigned to a non-intervention classroom
Usually, we’ll come in and then, if we were
If we have any homework, we’ll hand it in, or
like assigned homework, we’ll go over that
maybe we’ll go over it. Then [the teacher will] give
[the teacher] kind of like teaches through the
us a packet to do with our table. Usually there
homework… And then sometimes [the teacher]
[are] three or four, five people at a table and we’ll
would go on to create another problem, but that
do the packet. If we have any questions, then we’ll
specifies the target that [the teacher is] trying to
go over the packet or [the teacher] can answer
explain and then [the teacher] will write on the...
them...Sometimes we’ll go over the packet as
board…. And what I really like in [this] class, which
a class and [the teacher will] stand in front of
I’ve never had before, but I like it a lot is: we sit at
the class and tell us the correct answers. If we
these tables of four. So there’s a group and in the
have any questions, [they’ll] answer it in front of
middle of the table there’s a whiteboard. I like it
the class—but sometimes if it’s a longer packet,
because we’re able to write out our ideas before
then we won’t finish it and we might have to
copying them down.
finish it for homework.
Our analysis revealed some patterns that we describe here. Before sharing these trends, we provide two illustrative quotes above (see Figure 20) from students in the intervention and nonintervention classrooms.
43
Preliminary Impact
Students across the sample reported teacher exposition and demonstration as the most common interaction segment, followed by class dialogue and group work, and review and independent seatwork as least common. When we disaggregated the data according to those assigned to intervention classrooms versus those who were not, we note that those patterns held—but students in the intervention classrooms reported higher frequencies of time spent in the following situation types: • Going over homework There was a notable difference between the intervention group and the non-intervention group in that the “going over homework” segment was reported as a routine part of how the beginning of class was organized, while students in the non-intervention group reported this segment as happening occasionally, but not part of a regular routine. • Boardwork Not only was this segment more frequently named by intervention students, but when it was named, it was more typically the case that students were frequently named as the ones presenting at the board. • Student group work While the amount of time students reported spending in group work was perceived as ample across both groups, it was noticeably more in the intervention group. The reports provided by intervention students were more likely to note a critical role played by the teacher during this segment—with intervention students describing consistent teacher facilitation and interactive dialogue as an important part of their experience during group work.
Equity of Voice When asked about who typically does most of the talking in their classrooms, the plurality of students across the sample reported that the balance of talking in their classrooms was fairly evenly split between teachers and students. Furthermore, there was no apparent variation between how different disaggregates of students reported the balance of classroom talk.
Typical & Atypical Days In addition to asking students to describe a typical day in their math class, we also asked questions about atypical days, such as “What does an atypical day in your math class look like?” Because students’ responses to these interview questions featured activity structures that were mostly outside those things that Lemke (1990) aimed to describe, namely activities that are common or routine structures used by teachers to organize classroom exchanges, we needed a different coding scheme. For this, we used a combination of deductive coding (looking for common classroom practices we were familiar with from our work with BHHS) and inductive coding (generating descriptive and categorical codes from student responses) to develop codes for capturing themes that emerged from the students’ descriptions of their mathematics class on atypical days. This process resulted in the creation of five categories for capturing the types of activity students recounted engaging in on an atypical day in mathematics class: Tests, group work, free work days, review, presentations, and special labs/projects. Unsurprisingly, the disaggregation of this coding according to students’ assignments into intervention and non-intervention courses reveal stark differences. To begin, the students assigned to intervention classrooms were more likely to identify having days they considered out of the ordinary. Further, there were differences in the types of activities that students reported about when describing an atypical day. Students assigned to nonintervention classrooms were more likely to describe an atypical day as those in which they were engaged in group work or free work. Students assigned to the intervention classrooms were more likely to describe atypical days as containing special labs or projects. Below we share two quotes (Figure 21) to illustrate the ways that the students’ description of atypical days differed across these two groups
44
Preliminary Impact
Connectedness to Real World In the context of speaking about what mathematics class looks like, many of the students interviewed identified places in which class activities connect to the real world. That said, students in intervention courses were much more likely to bring up these connections. All of the students interviewed from the intervention classrooms positively identified a real world connection in their math class, compared to about three-quarters of the students interviewed from the non-intervention classrooms. From the descriptions of those reallife applications, those students in the intervention
classrooms tended to recall specific concepts from their special lab/project hands-on applications, such as working with flashlights to find parabola cones. In contrast, those students from the non-intervention classrooms were slightly more likely than those in the intervention classrooms to name a specific mathematical topic or concept, such as connecting derivatives to speed, velocity, and acceleration. Below we include two illustrative quotes (Figure 22) from interviews with students across the intervention and non-intervention courses.
Figure 21. Two illustrative quotes from Byram Hills students describing an atypical day in the mathematics classroom Excerpt from an interview with student assigned to an intervention classroom
Excerpt from an interview with student assigned to a non-intervention classroom
We did a lab sort of thing. We’d have a bottle and
At the beginning of the unit, [the teacher] gives us
there was a hole through the bottom and we
a really broad question that we’re supposed to try
would fill it up with water and certain materials
to answer—to understand by the end of the unit.
like big rocks, large rocks, marbles and stuff like
And when we answer that question, we give the
that and see the flow rate… We just turned it
class presentations with our group. And so class
in, but we’re not really sure what we’re doing
presentation days are kind of out of the ordinary
with it yet. But I mean obviously we’re going to
because everyone’s kind of nervous.
be doing something with it. The lab part was pretty much just a test of something that we’re going to be learning.
Figure 22. Two illustrative quotes from Byram Hills students describing ways their mathematics work connects to real-world applications Excerpt from an interview with student assigned to an intervention classroom
Excerpt from an interview with student assigned to a non-intervention classroom
We’re learning about how in a flashlight, there’s
I mean I think the teacher certainly tries [their]
a parabola cone in the inside, so when the light
best to make sure that we’re learning that this
bounces off that, it goes straight out. We’re kind of
stuff is actually useful and has uses instead of
building something like that on Monday. But we’re
just learning it for fun. [They] try to segway in
going to cook. It’s going to be like tinfoil in the
applications and a few of the problems. We
shape of a parabola and then a metal wire kind
haven’t done that in a while, [but an example
of in the middle. And we’re going to cook hot dogs
is] connecting derivatives to speed, velocity, and
and marshmallows.
acceleration. And [the teacher] actually talked a little about fuel efficiency and how you can get that by minimizing your acceleration
45
Preliminary Impact
Homework We asked students: (1) how much time their mathematics homework usually took them, (2) how they felt about their homework, generally, and, (3) more specifically, if they thought they learned from their homework. The large majority of students reported a range for homework with the least reported time being 5 minutes and the largest reported time being 90 minutes. More typically, the students reported taking anywhere from 25 to 35 minutes of time on homework each night. Students’ appraisals of homework (positive, negative, or ambivalent) were consistent across students in the intervention and non-intervention courses. In general, students across the sample offered positive appraisals of their homework in mathematics class. Interestingly, nearly all of the students interviewed distinguished math homework from homework in other subjects in a way that cast math homework as uniquely helpful. For example, one student captured this general sentiment when they responded to our interviewer’s question of “Overall, how would you say you feel about homework?” by saying:
While this student’s appreciation of their mathematics homework certainly does not communicate an unbridled enthusiasm, it does capture the overarching value that students expressed for the importance of math homework in providing independent practice and evaluation. The most oft-cited reason for the overall positive appraisal of homework was that it was good practice and helped them feel prepared for class.
It actually doesn’t really annoy me right now because I understand what’s happening in class a decent amount. I think math is one of the subjects that it definitely helps to have homework practice to see if you can do what you’re learning on your own, as opposed to other classes. I feel like in high school we don’t really get worksheets and stuff for homework anymore, [but this is] one class where it’s honestly not that bad.
46
Preliminary Impact
Attitudes Towards Mathematics We asked students to describe what they enjoyed and what frustrated them (if anything) about the work in their math classes. Students reported frustration and enjoyment with their mathematics classes at a relatively consistent rate across intervention and non-intervention courses, with enjoyment of mathematics being the more plentiful of the two. In Figures 23 and 24, we list the kinds of things students described and illustrative quotes.
Figure 23. Summary of the kinds of things Byram Students reported as enjoyable or frustrating elements of their mathematical experiences Things students enjoyed
Things students found frustrating
Achieving a sense of understanding
Potential for cascading errors in multi- step problems
Perceived ease or challenge with mathematical tasks
Problems that required a lot of information management
Perceived quality of teachers’ exposition of material
Insufficient teacher explanation
Specific mathematical topics
Classroom dynamics
47
Preliminary Impact
Figure 24. Illustrative quotes from Byram Students about elements of their math classes they find enjoyable or frustrating
THINGS STUDENTS FOUND ENJOYABLE I really enjoy thought-provoking problems...where you can play around “ with it and you can’t get the answer at first. You need to think of things you need to know about the problem...in order to be able to solve it. So challenging problems are fun to me
“
It’s a lot of letting us explore on our own, which is, I mean it’s, I find it really, really good. Understanding the concepts—like when it comes easy “ to me, it’s enjoyable to just keep getting it right and [to] be able to do them
THINGS STUDENTS FOUND FRUSTRATING Sometimes I make really stupid mistakes because on occasion [the “problems] are not really worded a way that makes it completely clear, which really does annoy me. Like, I had like a test and I’m like, what is it asking? I don’t really understand. So that’s one [frustration] because there are just so many parts, too. That is a little overwhelming Whenever we break off and do work, it’s the same five people “that go to help the rest of us who are still struggling, who don’t have our voices heard as much by [the teacher] because [they are] preoccupied with certain kids and like [they] kind of go to wherever the voices are the loudest.
“
Problems that kind of are multistep—like they’ll ask you to find one thing and then off that find soething else and then keep going for four or five times.... really frustrates me ‘cause if I get one thing wrong then it could mess up the whole entire problem and then I’ll have to go all the way from the beginning or something like that
48
Preliminary Impact
49
Preliminary Impact
In addition to these more straightforward kinds of appraisals of mathematical activity that students enjoyed or found frustrating, there were other instances that were more mixed. These cases are important to consider because they provide some texture for the ways that various students (with different mathematical backgrounds and perspectives) may relate to the abovenamed experiences in slightly more nuanced ways. One concept from educational research that surfaced in our analysis of students’ appraisals is self-efficacy. Self-efficacy, as defined by Zimmerman (2000), refers to four dimensions of students’ self-assessment of their capabilities to meet particular academic standards: (1) a grounded self-assessment (i.e., not an assessment of one’s general psychological traits, but a sense of one’s context-specific competencies), (2) a domain-
specific self-assessment (i.e., self-efficacy can differ across subjects and types of tasks), (3) a contextual self-assessment (i.e., consideration of external factors that could influence someone’s enactment of their capabilities), and (4) a non-comparative assessment (i.e., students’ assessment about their capabilities vis-a-vis their own measures of success rather than a normative measure of success). We examine various dimensions of self-efficacy along with other elements of appraisal. For our first example, we will take a look at an excerpt of a response from a student who had participated in different math tracks during high school. They spoke to the confluence of classroom processes, prior challenges with more interactive mathematical experiences, and self-efficacy. The student said:
I like the collaboration because...in my other math classes, they were kind of different…. So this year [in a differentlytracked class] there’s more collaboration and the concepts are kind of explained more, instead of [in the other classes we would] discover them for ourselves and kind of go over them.
This quote captures the nuance involved with considering classroom processes and structures and student enjoyment, motivation, and self-efficacy. For this student, more collaboration and more teacher explanation were associated with greater self-efficacy and familiarity with tasks. This does not mean that student-led exploration is contrary to building student self-efficacy. Rather, it means that studentled exploration requires the appropriate scaffolding and support for students who do not have a reliably developed sense of self-efficacy in mathematics or for whom 50
self-efficacy is highly contextual.
Preliminary Impact
In our second example, which augments this first one, a student’s response demonstrates ways that students’ self-efficacy can be closely associated with their enjoyment and motivation towards mathematics (Hackett & Betz, 1989; Zimmerman, 2000). The student said:
Well, I don’t enjoy math per se, but I definitely get the most gratitude [sic] out of doing problems with variables because… they’re very complicated, but I do understand it and by working through the complications...it’s so much more understandable to me—but it’s still very difficult to do.
This student’s reflection demonstrates how self-efficacy can be tied to positive motivations that are related to, but not determined by, more typical means students might use to gauge their own achievement, such as the perception of problems being easy to complete. The positive associations between selfefficacy and enjoyment and motivation were significant across aggregates of students, although some finer differences in motivations and attitudes towards mathematics may exist across different kinds of groups.
An important take-away here is that students generally feel motivated—and experience enjoyment—when they positively evaluate their own capability to perform mathematical tasks. Whether or not they identify these tasks as comparatively easy or difficult and whether or not they hold these tasks in high esteem are somewhat distinct aspects of students’ perceptions and motivations in mathematics. This has important implications for all teachers at Byram, but particularly for those teachers electing to use a problems-based approach. While we advocate strongly for the use of innovative forms of instruction, such as those used in the intervention courses, we add a dose of caution to such implementations. Namely, teachers’ instructional practices—specifically the extent to which they can help students feel supported in less structured tasks— are a significant factor in the success of these forms of innovative mathematics instruction.
51
Preliminary Impact
52
Preliminary Impact
An important take-away here is that students generally feel motivated—and experience enjoyment—when they positively evaluate their own capability to perform mathematical tasks. Whether or not they identify these tasks as comparatively easy or difficult and whether or not they hold these tasks in high esteem are somewhat distinct aspects of students’ perceptions and motivations in mathematics. This has important implications for all teachers at Byram, but particularly for those teachers electing to use a problems-based approach. While we advocate strongly for the use of innovative forms of instruction, such as those used in the intervention courses, we add a dose of caution to such implementations. Namely, teachers’ instructional practices—specifically the extent to which they can help students feel supported in less structured tasks— are a significant factor in the success of these forms of innovative mathematics instruction.
Discussion of Mathematical Sensemaking and Intervention Activities In the second half of the student interviews, we posed three mathematics story problems to the interviewees. Of the total 38 Byram students we interviewed, 22 were enrolled in non-intervention classes and 16 were enrolled in intervention classes. Our study focused on developing problem-based learning as a strategy to engage students in rigorous, complex, and real world thinking while maintaining competitive achievement standards. To help investigate the potential impacts our intervention might have, we asked students to solve mathematics story problems that researchers have previously used to study the phenomenon of students’ realistic mathematical sensemaking (sometimes just called sensemaking) (Reusser & Stebler, 1997). While a general aim of mathematics story problems (sometimes called word problems) is to engage students in real-
world, conceptual applications of mathematics, extant research has found that this is not reliably true in practice (Palm, 2008). Particularly, researchers have identified that students often suspend their realistic sensemaking, such that they attempt to solve story problems with complex conceptual dimensions by relying on more procedural mathematics. Researchers call this phenomenon of students suspending sensemaking in order to adhere to conventions of procedural school mathematics playing school. In theory, we hypothesized that the implementation of a problem- and place-based learning intervention could have a positive relationship with students’ engagement with realistic sensemaking. Below, we share and interpret a summary of the results of Byram students’ engagement with the three mathematics problems. In order to add another analytical dimension, we also share comparative results of students’ responses who were enrolled at the place-based semester program during the time of our interviews. This additional information allows us to pose generative questions and suggestions.
Methods of Study Design, Data Collection, & Analysis Question Selection The three mathematics story problems we asked students to solve have been used by other mathematics education researchers across numerous studies of students’ variable engagement with realistic sensemaking. Figure 25 details each story problem’s text and anticipated possible answers (based on previous study results, the “expected answer” is the answer students would find if they played school and the “realistic answer” is the answer students would find if they engaged sensemaking).
Figure 25. Story problems used in the interview and descriptions of expected versus realistic answers for each Problem
Problem Text
Expected Answer
Realistic Answer
Rope Problem
A man wants to have a rope long enough to stretch between two poles 12 meters apart, but only has pieces of rope 1.5 meters long. How many of these would he need to tie together to stretch between the poles?
8 pieces
More than 8 pieces
Runner Problem
John’s best time to run 100 meters is 17 seconds. How long will it take him to run 1 kilometer?
170 seconds
More than 170 seconds
Bruce & Alice Problem
Bruce and Alice go to the same school. Bruce lives at a distance of 17 km from the school and Alice lives at 8 km. How far do Bruce and Alice live from each other?
9 km or 25 km
Between 9 & 25 km
53
Preliminary Impact
Table 2. Percentage of students demonstrating realistic reactions to the given problems across 4 previously-conducted studies.1 Rope Problem
Runner Problem
Bruce & Alice Problem
Study 1 (n=100)
12%
6%
N/A
Study 2 (n=67)
6%
3%
N/A
Study 3 (n=75)
0%
3%
3%
Study 4 (n=45)
2%
7%
2%
Table 2 details the findings from some of the previously conducted studies that examined students’ reactions to these problems without intervention. As documented in the table, a very low percentage of students demonstrated engagement in realistic sensemaking in their responses to these three problems. These findings demonstrate the persistence of playing school among students encountering story problems.
Analysis Methods
Interview Methods
After coding students’ final solutions into one of the above general categories, we then coded their total responses for any indications of realistic sensemaking. These indications were variable, but for example could include a verbal recognition of the complexity of the conceptual work in the problem, a caveat that their answer was only true in a non-realistic world, or a drawing showing that they were accounting for a realistic implication of the problem. Since the first set of codes assesses the students’ final answer, this second set of codes illustrates whether or not a student demonstrated any signs of realistic sensemaking during their solution process—something that is possible regardless of the final answer. This second set of coding used (+) and (-) signs to indicate the presence or absence of realistic sensemaking, respectively. Thus, with the exception of Realistic answer (RA) (a category that already indicates realistic sensemaking), every category of final answer was also qualified with a (+) or (-), as detailed below, in Figure 27.
In the interviews, students were instructed to show1 their work on a piece of paper while completing the problem and were encouraged to think aloud in order to share their thought process. When they completed the problem, the interviewer asked for the student to hold up their work to the camera and describe again what they did. Those interviews were video recorded, and the video records were transcribed using natural language processing (NLP) software. Once transcribed, the video records were used by a research assistant to improve the NLP transcript and segment the transcript.2 In order to gauge students’ sensemaking across the three word problems, we analyzed three records: (1) The answer as verbally provided by the student (2) the answer as written on paper and shown to the interviewer, and (3) the students’ description of their thought process accounted for in the recorded interview and transcript.
Because these problems were canonical in other mathematics education research, we replicated the coding scheme used by Verschaffel et al (1994).3 We first coded students’ final answers into one of four general categories that capture the essence of their solution response (detailed in Figure 26)
1 Study 1: Greer (1993)—reported on 100 students between the ages of 13 and 14 years old from Northern Ireland; Study 2: Reusser & Stebler (1995)—reported on 67 students between the ages of 10 and 12 from Switzerland; Study 3: Verschaffel et al (1994)—reported on 75 students between the ages of 10 and 11 years old from Belgium; Study 4: Yoshida et al (1997)—reported on 45 students in 5th grade from Japan 2 The project used Temi, a Natural Language Processing software with a self-reported 90-95% accuracy rate for good-quality audio files (www.temi.com). Once audio files were uploaded to and processed by Temi, a project research assistant reconciled the transcript to the interview audio by conducting two rounds of listening and resolving errors in the NLP transcription. For the first listening session, they played the interview audio files at regular speed; for the second, they played the interview audio files at double speed. Still, knowing that the transcriptions were not 100% accurate, project researchers would go back and review and improve transcription as needed for analysis and writing.
54
3 We ultimately applied four of the coding categories that Verschaffel et al. outlined; in our coding, we did not have any basis to apply their code for “other answer” (OA)
Preliminary Impact
Finally, in order to be able to compare student responses along the broader distinction of the presence or absence of realistic sensemaking,4 we analyzed responses in two collapsed categories: responses with any indication of realistic sensemaking (regardless of final answer) and responses without any indication of realistic sensemaking (i.e., responses that demonstrated playing school). The collapsed coding categories are detailed in Figure 28. 4 Our collapsed category of “sensemaking” aligns with those responses coded as Realistic Reactions in the studies reported in Table 1
Figure 26. Four broad codes for categorizing students’ responses.
Figure 27. Sub-codes for categorizing realistic sensemaking and suspension of realistic sensemaking.
Figure 28. Collapsed Sensemaking/Playing School Coding
Playing School Codes
Realistic Sensemaking Codes
NA-
NA+
EA-
EA+
TE-
TE+ RA
55
Preliminary Impact
Next, we share results from our analysis of students’ realistic sensemaking and playing school in the mathematics portion of the interviews.
A man wants to have a rope long enough to stretch between two poles 12 meters apart, but only has pieces of rope one and a half meters long. How many of these would he need to tie together to
Results for the Rope Problem
stretch between the poles?
As noted in Table 1, the text of the rope problem presented to interviewed students (both orally and in writing) was:
Examples of Playing School Responses for the Rope Problem Figure 29a.
56
Explanation
Example Response
NA No answer; suspension of sensemaking
Response declines to provide an answer to the problem and does not indicate any awareness of implications of the act of tying the ropes together.
All interviewed students provided an answer; this code was not applicable for this problem.
EA Expected answer, suspension of sensemaking
Responses that did not take into account the reality that the act of tying the ropes reduces the overall length and therefore one needs to compensate for that loss in length with some number of additional ropes.
[Verbal] I would do 12 divided by 1.5...Okay. I would say eight meters
TE Technical error, suspension of sensemaking
A response that both did not account for additional length needed to tie knots and included a mathematical error of some kind in the student’s intended solution attempt.
[Written] 1.5 + 1.5 + 1.5 + 1.5 + 1.5 + 1.5 = 12. He will need 6 pieces of rope
Preliminary Impact
Before sharing how each of the groups performed on this item, we share some examples from the data to illustrate the various categories of playing school (Figure 29a) and realistic sensemaking (Figure 29b) within the context of the rope problem.
Examples of Making Sense on the Rope Problem: Figure 29b. Explanation
Example Response
NA+ No answer with realistic sensemaking
For an NA+ response, the student’s final response is to decline to provide an answer to the problem. However, in their reasoning process they indicate awareness that tying the ropes together and to the poles would introduce relevant information to solving the problem.
All interviewed students provided an answer; this code was not applicable for this problem.
EA+ Expected answer with realistic sensemaking
An EA+ response means that the student’s final answer is the expected (straightforward) answer of eight (8) ropes. However, the student’s solution process includes some accounting for the reality that the act of tying the ropes reduces the overall length and therefore one needs to compensate for that loss in length
[Verbal] It’s 8 if this was a math problem, but it wouldn’t be 8 if this was like a real life scenario.
TE+ Technical error with realistic sensemaking
A TE+ response does account for additional length needed to tie knots, but also includes a mathematical error of some kind in the student’s intended solution attempt.
No interviewed students demonstrated realistic sensemaking and also made a technical error; this code was not applicable for this problem.
RA Realistic answer; uses realistic sensemaking
An RA response means the student accounts for additional length needed to tie knots and this accounting is reflected in the student’s final answer.
[Verbal] First I would see how many times 1.5 can go into 12. So that is 8. And then when he’s tying them, he’s obviously losing length. So I would say the answer is nine.
57
Preliminary Impact
As Table 3 shows, students from both the intervention and non-intervention group from Byram Hills and students at the place-based semester program were far more likely to give the expected answer of 8 without any evidence of realistic sensemaking by not taking into account losing rope each time one of the segments are tied together (rates of expected answers were 56%, 82%, 76% respectively). Some students from both schools mentioned that tying knots might lead to needing more rope, however they did not account for the precautionary rope in their mathematical solving, rather only in their interview as they announced their
thought processes. The majority of students at each school, regardless of intervention grouping, “played school” in their answers. Respectively, 87% of Byram non-intervention students, 82% of semester program students, and 56% of Byram intervention students played school in their responses. However, the Byram Hills students in the intervention classes demonstrated the highest rate of sensemaking of any group, with 44% of their responses being coded as either EA+ or RA (compared to 13% of Byram Hills non-intervention students and 18% of semester program students).
Examples of Playing School Responses for the Runner Problem Figure 30a. Explanation
Example
NANo answer, suspension of sensemaking
Response declines to provide an answer to the problem.
All interviewed students provided an answer; this code was not applicable for this problem.
EAExpected answer, suspension of sensemaking
A student whose answer was coded EA- would have given the straightforward answer of 170 seconds, not accounting for fatigue. These students performed conversions and calculations correctly, but did not indicate any awareness of the need to account for fatigue in their solution process.
[Written]
An answer coded TE- both did not include any accounting for fatigue in solving the problem and included a technical error in either conversion or another calculation.
[Written]
TETechnical error in answer, suspension of sensemaking
100m in 17s x10 x10 1000 m 170s
100/17= 0.1/x 100/17= 0.1/x 100x/100= 0.17/100 100x = .17 x = 1700 0.0017 seconds
58
Preliminary Impact
Results for the Runner Problem As noted in Table 3, the text of the rope problem presented to interviewed students (both orally and in writing) was: John’s best time to run 100 m is 17 sec. How long will it take him to run 1 km?
Again, before sharing how the various groups performed on this item, we share some examples of students’ responses to this prompt from various coding categories (Figure 30a and 30b).
Examples of Making Sense on the Runner Problem: Figure 30b. Explanation
Example
NA+ No answer with realistic sensemaking
Student’s final response is to decline to provide an answer to the problem, but in their reasoning process they indicate awareness that the runner would not realistically run at a constantly scaleable rate.
All interviewed students provided an answer; this code was not applicable for this problem.
EA+ Expected answer with realistic sensemaking
Student’s final response is the expected answer, but the student’s solution process includes some accounting for the reality that a person could neither run at a perfectly constant rate nor that that rate would be a linear proportion
[Verbal] I wouldn’t assume that his pace for these 17 seconds would be the same for the kilometers. But there’s no real way to look at that right now without more information. So I’ll go ahead and estimate that it would be...170 seconds.
TE+ Technical error with realistic sensemaking
A response that does account for the reality that a person could neither run at a perfectly constant rate nor that the rate would be a linear proportion. However, the final response includes a technical mathematical error.
No interviewed students demonstrated realistic sensemaking and also made a technical error; this code was not applicable for this problem.
RA Realistic answer; uses realistic sensemaking
A student’s final response accounted for the reality that a person could neither run at a perfectly constant rate nor that the rate would be a linear proportion.
[Verbal] ...Just take into account the fatigue, which is not really calculate-able. So I’d say the best time would be 17 times 10, but I’m taking into account fatigue. I mean, I guess my best guess would be maybe 200 seconds, but I’ll go with something over 170.
59
Preliminary Impact
Like the rope problem, Table 4 shows that students across all groupings were far more likely to give an expected “playing school” answer to the runner problem. The runner problem requires a unit conversion in the solution process; this was an aspect of the problem the place-based semester program students had more difficulty with than the Byram non-intervention and intervention students. However, it is not possible to know what role, if any, the conversion element may play in students’ engagement or suspension of sensemaking. A realistic answer was coded when students mentioned
the need to add time after the mathematical solving was complete due to fatigue, while an EA+ was coded, like the first math problem, when a student commented on the nuance in the final solution, but did not mathematically add any time to John’s kilometer run. As with the rope problem, the Byram intervention students were most likely to make sense in their solution processes (at a rate of 31%, compared to the semester program and Byram non-intervention students, with rates of sensemaking of 12% and 9% respectively).
Examples of Playing School in the Bruce and Alice Problem Figure 31a.
60
Explanation
Example
NANo answer; suspension of sensemaking
Response declines to provide an answer to the problem and the student does indicate sensemaking about the range of possible spatial relationships in the problem.
[Verbal] So I’m going to use the Pythagorean Theorem to find the altitude...I don’t know. I don’t even know.
EAExpected answer; suspension of sensemaking
A student whose answer was coded EAwould have given any single numerical answer within the range of 9 to 25 km (and so did not account for the range of possible answers).
[Written] 17 + 8 = 25
TETechnical error in answer; suspension of sensemaking
A response coded TE- meant the student did not demonstrate engagement of realistic sensemaking in answering the question and that the student conducted a technical error in their solution (e.g., a calculation error).
[Verbal] Okay...I’ll draw a little house for Bruce and then Alice lives at [sic] 8 kilometers from the school. Yeah....then I would do 17 kilometers minus eight kilometers, which is 11.”
Preliminary Impact
Results from the Bruce and Alice Problem As noted in Figure 35, the text of the Bruce and Alice problem presented to interviewed students (both orally and in writing) was: Bruce and Alice go to the same school. Bruce lives at a distance of 17 km from the school and Alice at 8 km. How far do Bruce and Alice live from each other?
Again, we begin by sharing some examples of playing school (Figure 31a) and making sense (Figure 31b) in the context of this problem.
Examples of Making Sense on the Bruce and Alice problem: Figure 31b. Explanation
Example
NA+ No answer with realistic sensemaking
Student’s final response was “no answer,” but the student identified that there was a lack of information needed to determine a single solution.
[Verbal] I mean you can’t really say based on this because you don’t know at what, like where they are in regards to each other.
EA+ Expected answer with realistic sensemaking
Students indicated an understanding that there was not a determinable single numerical solution to the problem, and so provided a single numerical answer with the caveat that other answers were possible (within a range).
[Verbal] If Alice lived on the east side of the school eight miles away and Bruce lived on the west side 17 miles, then they could live much farther than if they both lived on the east side. So, if you’re assuming that they live on the same side of the school, then I would say that they live nine kilometers away from each other.
TE+ Technical error with realistic sensemaking
A response coded as TE+ reflected a student applying an assumption of a linear relationship between the three points in the problem, naming it as an assumption for the purposes of solving the problem (and thereby signalling sensemaking), but making a calculation error that led to their final answer being incorrect per the operation they wrote and used. These students either attempted to use a pythagorean triplet incorrectly or made a simple miscalculation
[Verbal + Written] 8^(2 )+ 15^2= 17^2 64 + 225 = 289 289 = 289 Pretty much what I did is automatically saw 17, 8, 15. I mean that’s kind of Pythagorean triplets. And then just to make sure, I was like, would that actually work as a right triangle? And it does if Bruce and Alice live directly like that from each other [references picture and calculations].
RA Realistic answer; uses realistic sensemaking
Student’s response used the information given to find a finite range of possible solutions, such as 9 ≤ x ≤ 25, which would account for the houses being any direction away from the school.
[Verbal] ...it could be a triangle where if he’s going off in one direction, she’s going in the opposite direction. It can be at most 25 kilometers, but then it could also be 9 if it’s the same line. So, x has to be less than or equal to 25 and greater than or equal to 9 with x being the difference between Bruce and Alice.
Preliminary Impact
Table 3. Rope Problem Results N
EA-
TE-
EA+
RA
BH N-Intervention
22
82%
5%
13%
0%
Semester Program
17
76%
6%
0%
18%
BH Intervention
16
56%
0%
6%
38%
N
Playing School
Making Sense
BH N-Intervention
22
87%
13%
Semester Program
17
82%
18%
BH Intervention
16
56%
44%
Table 4. Runner Problem Results
62
N=
NA-
EA-
TE-
EA+
RA+
BH N-intervention
22
0%
82%
9%
5%
5%
Semester Program
17
6%
76%
6%
6%
6%
BH Intervention
16
0%
56%
13%
19%
13%
N
Playing School
Making Sense
BH N-intervention
22
91%
9%
Semester Program
17
88%
12%
BH Intervention
16
69%
31%
Preliminary Impact
Table 5. Bruce and Alice Results N
NA-
EA-
TE-
NA+
EA+
TE+
RA
BH N-Intervention
22
5%
36%
5%
18%
32%
0%
5%
Semester Program
17
0%
53%
0%
0%
41%
0%
6%
BH Intervention
16
0%
31%
0%
13%
31%
6%
13%
N
Playing School
Making Sense
BH N-Intervention
22
46%
54%
Semester Program
17
53%
47%
BH Intervention
16
37%
63%
As with the two previous math problems, the Byram intervention students demonstrated the highest rate of realistic sensemaking (63%, compared to 47% of the place-based semester program students and 54% of Byram non-interventions students). Overall, all three groups of students demonstrated their highest rates of sensemaking with this problem. Many students were aware that there seemed to be some information missing in the problem. We cannot say conclusively why this difference in sensemaking was evident in students’ responses to this problem; this remains a question for further study and analysis.
Conclusions In closing, when we look at students’ tendencies to make sense across the context of these three story problems we noticed that students from both schools are generally more likely to play school than attend to realistic considerations in their mathematical solving (see Table 6). The most important finding from this section of our student interviews was that, across all three problems, the Byram intervention students consistently demonstrated the greatest rate of sensemaking. This result opens up fascinating questions about both the potential impact of and distinctions between place-based learning strategies (like those implemented at Byram) based on immersive placebased learning schools (like the place-based semester program). Positively, these results suggest that there is some relationship between the intervention and students’ use of realistic sensemaking in story problems.
We do not claim that this relationship is causal, but its existence is encouraging for considering the utility of place-based learning interventions in highly-competitive mathematics programs. The fact that, overall, the nonintervention Byram students engaged sensemaking at rates similar to or greater than the place-based semester program students is also an important result that warrants further investigation. When placed side-byside with the results from prior studies with similarlyaged students for which there was no intervention, we see some interesting trends. To begin, the performance of the BH-non-intervention group performed slightly better than students in three of the four prior studies and quite similar to those students in Greer’s (1993) study. One factor to consider here might be age, as Greer’s study reported on students aged 13 - 14 years old, while the other three studies were slightly younger students (between ages 10 - 12 years old). Next, turning our attention to the Semester Program, students in that group outperformed students in all of the prior studies, however, given the immersive nature of the intervention, the difference is somewhat underwhelming. Finally, the BH Intervention group handedly outperforms students in all four prior studies. We are still grappling with these results as they only align with part of what we thought we would find. We are somewhat surprised that students in the BH intervention group so clearly outperformed students at the place-based semester program as the nature of the semester program intervention is quite a bit more immersive.
63
Preliminary Impact
We have three initial hypotheses that might account for these results. First, we cannot rule out the possibility that the tasks themselves are problematic in terms of their
is, while students’ mathematical experiences outside school in the kind of rich, immersive environment offered by the semester program may teach them
ability to gauge students’ sense making. This is certainly not a new argument and has been raised before as a possibility by prior scholars (Gerofsky, 1996). Second, it is also possible that the length of the intervention at Byram Hills could have made a difference—the Byram Hills Intervention was longer (spanning an entire school year) while the semester program intervention was contained within 90 days (or a single semester). Finally, the hypothesis we find most compelling, is the possibility that the very nature of the intervention at the two sites could account for the difference. In particular, the supplementary nature of the semester program intervention may create some important dissonances in the students that are overcome by Byram Hills students whose instruction had been intervened on in the context of school. The work of numerous scholars on story problems has suggested that school has indoctrinated students into a certain way of engaging in that work, and that indoctrination includes the need to suspend sense-making (Palm, 2008).
something important about themselves and learning more generally, it may also fail to shape the way students see school and what’s possible there. If this were the case, when confronted with story problems that look like the problems from school mathematics, the semester program students may understand the task is to behave like they do when in school—adhering to the norms that school inscribes for how one is supposed to solve a word problem.
The fact that the semester program happens outside of school may leave intact students’ understandings about the norms of school mathematics, including how they think they are supposed to tackle story problems. That
64
The Byram Hills intervention, in contrast, was happening in school. Simply because it was happening in school, students may be less able to dismiss it as distinct from school. This new way of doing mathematics, supported and sanctioned by their school teacher, may create more opportunities for students to question previously established norms for doing mathematics in general, and engaging in story problems in particular. If this were the case, when confronted with the same three story problems Byram Hills students may have had some additional sense of freedom for applying what they had been practicing in math class—breaching the norms that school inscribes for how one is supposed to solve a word problem.
Preliminary Impact
Table 6.
Percentage of students demonstrating realistic reactions to the given problems across Byram Hills (both intervention and non-intervention courses) and the semester program
N
Rope Problem
Runner Problem
Bruce & Alice Problem
BH N-intervention
22
14%
9%
54%
Semester Program
17
18%
12%
47%
BH Intervention
16
44%
31%
63%
Greer (1993)
100
12%
6%
N/A
Reusser & Stebler (1995)
67
6%
3%
N/A
Verschaffel et al. (1994)
75
0%
3%
3%
Yoshida et al. (1997)
45
2%
7%
2%
65
66
Summary
Conclusions The University of Michigan - Byram Hills High School research practice partnership has thus far inspired a cohort of precalculus teachers to re-imagine how their project-based activities can be integrated with local, regional, and national contexts about which students are naturally curious. We have gathered preliminary survey and interview data that has provided baseline measures to describe the pre-intervention context of mathematics classrooms at BHHS. The immersive professional development experience that took place at the place-based semester program was, according to the teachers’ reflections, transformative. From the first half of our interviews with Byram Hills students, we suggest that students placed in the intervention classes did perceive meaningful differences in their experiences within mathematics classrooms—as noted in the differences in ways that intervention and nonintervention students described both their typical and atypical days. From the survey of students at both Byram Hills (both intervention and non-intervention) and the semester program, we have reason to believe that these kinds of experiences can play a role in shaping some aspects of their attitudes towards mathematics as well as their perception of classroom instruction. Finally, the data gathered in the second half of the interview with students across both Byram Hills
(intervention and non-intervention) and the semester program suggests that intervening on students’ mathematical experiences with place-based learning in school may have some important affordances over similar or even more intense experiences that can happen in supplementary programs, such as semester programs. This finding has some important implications for the field and leaves open many interesting questions for further exploration. We are grateful for the opportunity to engage in this critical work and look forward to sharing our work with those at BHHS and beyond.
67
References
References Boaler, J., & Selling, S. K. (2017). Psychological imprisonment or intellectual freedom? A longitudinal study of contrasting school mathematics approaches and their impact on adults’ lives. Journal for Research in Mathematics Education, 48(1), 78-105. Collaborative for Academic, Social, and Emotional Learning (CASEL). (2013). Implementing systemic district and school social and emotional learning. Chicago: CASEL. De Lange, J. (1996). Using and applying mathematics in education. In International handbook of mathematics education (pp. 49-97). Springer, Dordrecht. Hargreaves, M., Homer, M., & Swinnerton, B. (2008). A comparison of performance and attitudes in mathematics amongst the ‘gifted’. Are boys better at mathematics or do they just think they are?. Assessment in Education: Principles, Policy & Practice, 15(1), 19-38. He, H. (2007). Adolescents’ Perception of Parental and Peer Mathematics Anxiety and Attitude Toward Mathematics: A Comparative Study of European-American and Mainland-Chinese Students (Doctoral dissertation, Washington State University). Herbst, P., & Milewski, A. (2018). What StoryCircles can do for mathematics teaching and teacher education. In Scripting Approaches in Mathematics Education (pp. 321-364). Springer, Cham. Karjanto, N. (2017). Attitude toward mathematics among the students at Nazarbayev University Foundation Year Programme. International Journal of Mathematical Education in Science and Technology, 48(6), 849-863. Ko, I., Herbst, P., Milewski, A. (2020). A heuristic approach to assess change in mathematical knowledge for teaching geometry after a practice-based professional learning intervention. Research in Mathematics Education. 22(2), 188-208. Milewski, A., Herbst, P.G., Bardelli, E., & Hetrick, C. (2018). The role of simulations for supporting professional growth: Teachers’ engagement in virtual professional experimentation. Journal of Technology and Teacher Education, 26(1), 103-126. Milewski, A.M., Herbst, P.G., & Stevens, I. (2020). Managing to collaborate with secondary mathematics teachers at a distance: Using storyboards as a virtual place for practice and consideration of realistic classroom contingencies. In Ferdig, R.E., Baumgartner, E., Hartshorne, R., Kaplan-Rakowski, R. & Mouza, C. (Eds.) Teaching, technology, and teacher education during the COVID-19 pandemic: Stories from the field. (pp. 623 - 630). Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/p/216903/.
68
References
(References continued) Milewski, A., Stevens, I., Herbst, P., & Huhn, C. (under review). Confronting teachers with contingencies to support their learning about situation-specific pedagogical decisions in an online context. In Hollebrands, K. & Anderson, R. (Eds). Online Learning in Mathematics Education. Submitted to Springer in November 2020. Palm, T. (2008). Impact of authenticity on sense making in word problem solving. Educational Studies in mathematics, 67(1), 37-58. Payton, J. W., Wardlaw, D. M., Graczyk, P. A., Bloodworth, M. R., Tompsett, C. J. and Weissberg, R. P. 2000. Social and emotional learning: A framework for promoting mental health and reducing risk behavior in children and youth. Journal of School Health, 70(5): 5–179. Reusser, K., & Stebler, R. (1997). Every word problem has a solution—The social rationality of mathematical modeling in schools. Learning and Instruction, 7(4), 309-327. Schukajlow, S., & Krug, A. (2014). Do multiple solutions matter? Prompting multiple solutions, interest, competence, and autonomy. Journal for Research in Mathematics Education, 45(4), 497-533. Smith, G. A. (2002). Place-based education: Learning to be where we are. Phi delta kappan, 83(8), 584-594. Sundre, D., Barry, C., Gynnild, V., & Ostgard, E. T. (2012). Motivation for achievement and attitudes toward mathematics instruction in a required calculus course at the Norwegian University of Science and Technology. Numeracy, 5(1), 4. Tapia, M., & Marsh, G. E. (2000). Effect of gender, achievement in mathematics, and ethnicity toward mathematics. Presented at the annual meeting of the Mid-South Educational Research Association, Bowling Green, KY Tapia, M., & Marsh, G. E. (2002). Confirmatory factor analysis of the attitudes toward mathematics inventory. Paper presented at the Annual meeting of the Mid-South Educational Research Association. Chattanooga, TN. Thuneberg, H. M., Salmi, H. S., & Bogner, F. X. (2018). How creativity, autonomy and visual reasoning contribute to cognitive learning in a STEAM hands-on inquiry-based math module. Thinking Skills and Creativity, 29, 153-160. Van den Heuvel-Panhuizen, M., & Drijvers, P. (2014). Realistic mathematics education. Encyclopedia of mathematics education, 521-525. Zins, J. E., Bloodworth, M. R., Weissberg, R. P., & Walberg, H. J. (2007). The scientific base linking social and emotional learning to school success. Journal of educational and psychological consultation, 17(2-3), 191-210.
69
Appendicies
Appendix A: Total sample of students for Area 1 Figure A1. Total sample of students for the TRIPOD and ATMI pre-survey Course
70
Grade
Track
Pre-Survey
Post-Survey
Geometry A
9th
Advanced
15
14
Geometry (9th)
9th
Accelerated
44
43
Algebra 2/Trig A
10th
Advanced
15
14
Algebra 2/Trig (10th)
10th
Accelerated
13
13
Algebra 2/Trig (11th)
11th
On-Track
15
15
Adv. Pre-Calculus
11th
Advanced
16
16
Pre-Calculus (11th)
11th
Accelerated
64
66
Pre-Calculus (12th)
12th
On-Track
75
35
Calculus BC
12th
Advanced
22
18
AP Stats
12th
Advanced
26
21
Calculus
12th
Accelerated
12
10
Appendicies
Figure A2. Total sample of students for the interview study
Course
Grade
Track
Number of Students that have taken the Pre-Survey
Geometry A
9th
Advanced
1
Geometry (9th)
9th
Accelerated
5
Algebra 2/Trig A
10th
Advanced
3
Algebra 2/Trig (10th)
10th
Accelerated
2
Algebra 2/Trig (11th)
11th
On-Track
2
Adv. Pre-Calculus
11th
Advanced
4
Pre-Calculus (11th)
12th
Accelerated
8
Pre-Calculus (12th)
12th
On-Track
5
Calculus BC
12th
Advanced
1
AP Stats
12th
Advanced
2
Calculus
12th
Accelerated
1
71
Appendicies
Appendix B: Illustrations of Place-Based Lessons Figure B1. Frames from two versions of the Longitude Lesson that demonstrate possibilities for more subject-area disciplinary challenges (left) and fewer subject-area disciplinary challenges (right). MORE
Insert a few frames of a version of the Longitude Lesson that draws on interdisciplinary knowledge to build a model that we hypothesize to have more disciplinary challenges
FEWER
Insert a few frames of a version of the Longitude Lesson that draws almost exclusively on mathematical knowledge to build a model that we hypothesize to have less disciplinary challenges
Figure B2. Frames from two versions of the Gerrymandering Lesson that demonstrate possibilities for more interpersonal challenges (left) and fewer interpersonal challenges (right). MORE Few frames from a version of the Gerrymandering Lesson that makes use of current events that we hypothesize to have more interpersonal challenges
72
FEWER Few frames from a version of the Gerrymandering Lesson that mostly avoids use of current events that we hypothesize to have less interpersonal challenges
Appendicies
Appendix C: Effective Samples & Missing Response Description After adjusting the sample to only include “completers,” we were satisfied that the omissions were unlikely to impact the overall results we report here. To illustrate this for the reader, we report on items that contained missing responses in the pre-survey. A total of 77 of the 153 questions had at least one student skipping the questions. Of those 77 questions that contained some missing responses, • All 77 questions had response rates greater than 96% (skipped by 10 or fewer students) • 68 questions had response rates greater than 99% (skipped by 3 or fewer students) • 20 questions were unrelated to the constructs we report on here: 9 questions intended to gauge students’ background (to enable us to understand the way effects may vary for different groups of students), for example: race gender current GPA language spoken at home parents’ education 11 questions intended to gauge related constructs (to enable us to identify possible covariates to avoid making spurious conclusions), for example: students’ perception of peer culture students’ success skills and mindsets students’ motivation in school more generally only 5 questions related to constructs of interest were skipped by 5 or more students (see Figure C2)
73
Appendicies
Figure C1. The five construct-related items for which 5 or more of the total 236 participants left blank Construct
Item
# BLANK
Response Rate
7Cs Care
My teacher seems to know if something is bothering me.
10
96.78%
7Cs Challenge
My teacher asks students to explain more about answers they give.
6
98.07%
7Cs
Student behavior in this
5
98.39%
Classroom Management
class is under control.
7Cs Classroom Management
I hate the way that students behave in this class.
5
98.39%
7Cs Clarify
We get helpful comments to let us know what we did wrong on assignments.
5
98.39%
Figure C2. Effective sample of students for the TRIPOD and ATMI pre-post-survey Course
74
Grade
Track
Count
Geometry A
9th
Advanced
12
Geometry (9th)
9th
Accelerated
41
Algebra 2/Trig A
10th
Advanced
14
Algebra 2/Trig (10th)
10th
Accelerated
12
Algebra 2/Trig (11th)
11th
On-Track
9
Adv. Pre-Calculus
11th
Advanced
14
Pre-Calculus (11th)
11th
Accelerated
60
Pre-Calculus (12th)
12th
On-Track
27
Calculus BC
12th
Advanced
18
AP Statistics
12th
Advanced
19
Calculus
12th
Accelerated
10
Appendicies
Appendix D: Comparison of Byram Intervention to Comparison Semester Program Figure D1.
Significant changes demonstrated for students in the intervention classes on four of the seven TRIPOD scales
75
Appendicies
Figure D2. Significant changes demonstrated for students in the intervention classes on one of the four ATMI scales
76
77
© University of Michigan 2021
V.08.08.21
University of Michigan School of Education 610 East University Drive Ann Arbor, MIchigan 48109