Seeing the classroom through students' eyes

Page 1

Working Paper – Please do not cite without permission

Seeing The Classroom Through Students’ Eyes

Bryan Mascio Harvard Graduate School of Education 13 Appian Way Cambridge, MA 02138 bryan_mascio@mail.harvard.edu

A Teacher Professional Development Induction to Facilitate Social Perspective Taking Joseph McIntyre Harvard Graduate School of Education 13 Appian Way Cambridge, MA 02138

Abstract When teachers struggle to understand what makes a student tick, the teacher-student relationship typically suffers as a result. Social perspective taking – understanding an individual’s thoughts, feelings, and motivations – might serve as an oil to lubricate these social interactions and relationships. In this field experiment, teachers (N=106) in an urban charter network participated in a new social perspective taking induction aimed at understanding their most perplexing students. The induction caused treatment teachers to try harder to understand these students and improved teachers’ perspective of their relationships with students months later. In addition, these students obtained higher course competency scores (akin to grades) than their control counterparts. This study supplies empirical connective tissue for the theoretical links between social perspective taking and teacher-student relationships.

Hunter Gehlbach Gevirtz Graduate School of Education #3113 University of California, Santa Barbara Santa Barbara, CA 93106-9490 hgehlbach@education.ucsb.edu

+

“No significant

Introduction

learning can

Few educators doubt that strong teacher-student relationships (TSRs) can bolster learning alongside a broad array of other student

occur without a significant

outcomes. But what happens when teachers’ relationships with their students are downright challenging? Even the most skilled teachers struggle to connect with at least one or two students. Anecdotally, on

relationship.” – Dr. James Comer

teachers’ frustrations with the “10% of students who take up 90% of their time” deplete much of their psychological energy. In this article, we describe the results of an intervention designed to change teachers’

Seeing The Classroom Through Students’ Eyes

perceptions and understanding of these students, with the hope of improving these often challenging relationships and catalyzing downstream benefits for students. A TSR must be understood as exactly that - a relationship. Dyadic in nature, it requires each party to understand the other.

Although the need for understanding is mutual, these

relationships are asymmetric with respect to power and responsibilities (Gehlbach & Robinson, in press).

Because of their roles, teachers hold more power and thus, arguably, bear more

responsibility for fostering a positive relationship. Teachers must foster many more of these relationships than students do. In elementary school, students typically have one main teacher; in secondary school this number is often closer to seven. By contrast teachers frequently manage 25 to over 125 TSRs, respectively. Given their disproportionate power and number of relationships, facilitating teachers’ understanding of their more cryptic students’ thoughts, feelings, and motivations might pay substantial dividends. Despite the important role of TSRs for an array of student outcomes (Juvonen, 2007), it has been unclear what skills and/or dispositions might improve these relationships. This study investigates teachers’ social perspective taking (SPT) as a promising, malleable precursor to TSRs in schools. We begin by reviewing the empirical evidence that signals the importance of TSRs. Next, we locate our research within a broader framework that theoretically connects teachers’ SPT capacities with their TSRs. Ultimately, we introduce our intervention – a SPT induction for teachers – and the resulting evidence of its impact on TSRs and students’ academic outcomes.

Teacher-Student Relationships in Schools Perhaps because classrooms are fundamentally social contexts (Gehlbach, 2010), TSRs are associated with numerous valued student outcomes. The vast majority of studies that document these associations are correlational. However, the sheer number of associations in these studies combined with the diversity of positive student outcomes signal the promise of improving TSRs as a target for intervention research. As one example, TSRs have a well-documented association with students’ academic performance. Recent meta-analyses show that students at all levels who have better TSRs tend to perform better academically both in terms of grades and standardized tests (Cornelius-White, 2007; Roorda, Koomen, Split, & Oort, 2011). Interestingly, at least with respect to grades, recent experimental studies suggest that the improved grades may have more to do with the teachers’ perceptions of the relationship than with the students (Gehlbach et al., 2016). Mascio, McIntyre & Gehlbach

2

Seeing The Classroom Through Students’ Eyes TSRs also seem to matter for students’ affect towards school. Sakiz, Pape, and Hoy (2012) found that teachers’ supportiveness was positively associated with students’ sense of belonging and their academic enjoyment as well as negatively associated with middle school students’ academic hopelessness in math class. When students and teachers lack a bond or have a negative relationship, students are more likely to feel alienated (Murdock, 1999). Cornelius-White’s (2007) meta-analysis finds overall associations between the positivity of students’ TSRs and increased student participation and attendance, as well as a reduction in disruptive behavior and droppingout. Finally, TSRs may impact students’ motivation. For instance, Wentzel, Battle, Russell, and Looney (2010) found that students with more supportive, less critical teachers more frequently pursued social goals in the classroom. Adolescents’ perceptions of teacher support and caring predict student effort, whether student effort is reported by teachers (Goodenow, 1993; Murdock & Miller, 2003) or by students (Wentzel, 1997). Sakiz et al. (2012) also found that those students with more supportive teachers had significantly greater self-efficacy in their math classes. The meta-analysis of Roorda et al. (2011) showed consistent positive overall association between TSRs and students’ engagement at the secondary level. In sum, beyond being a crucial outcome in their own right, TSRs are also associated with a broad constellation of valued student outcomes. Our brief review of the associations between TSRs and academic achievement, affect, and motivation represents but a small cross section of a vast body of research. Our point is that TSRs appear to be a key barometer of students’ academic and personal well-being. Thus, developing effective TSR interventions may pay important dividends for students.

Conceptualizing Teacher-Student Relationships Our conceptualization of TSRs builds from, and extends beyond Pianta’s (1999) idea of TSRs as dyadic social interactions. In other words, we also see these relationships as streams of interactions between teachers and students within and outside of the classroom. Teachers and students repeat these interactions and adapt them based on feedback, with certain transaction patterns being reinforced over time. However, TSRs encompass more than just behaviors. They are also perceptions of behaviors, personality, motivations, and attitudes (Gable, Reis, & Downey, 2003). After any given interaction, each party might walk away with very different impressions. As Gable et al. (2003) Mascio, McIntyre & Gehlbach

3

Seeing The Classroom Through Students’ Eyes note, “Patterns of interaction depend on the actions and reactions of both partners, and their actions and reactions depend on each individual’s perceptions and interpretations of the other’s behavior” (p. 100). In sum, in our theoretical conception of TSRs, we incorporate Pianta’s (1999) traditional focus on dyadic interactions, but also emphasize teachers’ aggregated and ongoing perceptions of their students (as well as their interactions) over time. Because these perceptions are stored in memory, they serve to guide future interactions with students.

Social Perspective Taking Given this conceptualization of TSRs, levers to improve TSRs might focus on interactions and/or perceptions. One hypothesized precursor, SPT, has sparked particular interest among relationship researchers (e.g., Galinsky, Ku, & Wang, 2005). SPT entails discerning the thoughts, feelings, and motivations of others as well as their point of view and perceptions of the situation (Gehlbach, Brinkworth, & Wang, 2012). It is a complex aptitude (Corno et al., 2002), encompassing both motivation and ability. In other words, for social perspective taking to improve relationships, a “perceiver” not only has to have the ability to accurately read or infer the thoughts and feelings of a “target,” but s/he must also be motivated to enact that ability (Gehlbach, Brinkworth, & Wang, 2012). At a broad theoretical level, we posit two pathways through which SPT might influence TSRs – by changing perceptions or interactions. Perceptions. First, improved SPT (motivation or accuracy) is likely to de-bias teachers’ perceptions of their students. Motivating participants to engage in social perspective taking has reduced stereotyping of and prejudice against others in multiple experiments (Galinsky & Moskowitz, 2000; Ku, Wang, & Galinsky, 2010). In addition, participants who adopt certain perspective taking strategies have improved the accuracy of their social perceptions of others (Eyal & Epley, 2010; Liberman, Minson, Bryan, & Ross, 2011; Lord, Lepper, & Preston, 1984). Because most misperceptions between distinct groups such as teachers and students are likely to be negative (Hewstone, Rubin, & Willis, 2002), improving the accuracy of a teacher’s perceptions of the student should, on average, reduce negative perceptions thereby improving relationships. Thus, an important theoretical pathway through which SPT motivation and ability could enhance TSR is by helping teachers to give students the benefit of the doubt, i.e., reducing any negative bias through which they perceive the student. Amongst the many cognitive biases that can interfere with accurate SPT, fundamental Mascio, McIntyre & Gehlbach

4

Seeing The Classroom Through Students’ Eyes

attribution error, confirmation bias, and naïve realism may be particularly important for teachers to overcome (Brinkworth & Gehlbach, 2015). The fundamental attribution error describes people’s pervasive tendency to explain the social behavior of others by disproportionally focusing on the causal role of an individual’s personality traits and neglecting situational causes (Ross, 1977). For example, when a student puts his head down on the desk during a lesson, a teacher may perceive a lack of interest in learning as opposed to exhaustion from working late to help support his family. Our intervention specifically seeks to have teachers overcome the fundamental attribution error and incorporate situational factors into their thinking. Confirmation bias refers to people’s tendency to ignore or devalue contradictory information while seeking out and privileging information that corroborates their favored hypothesis (Wason, 1960). In our example, the teacher would particularly note the student’s other behaviors that confirm her “not caring” hypothesis – skipping assignments, missing classes, or socializing at in appropriate times. In contrast, the teacher overlooks or explains away the students’ completed assignments, attendance when feeling sick, and periods of focused attention. Thus, the intervention addresses confirmation bias and ways to mitigate it. Naïve realism refers to our closely-held belief that we see objective reality. Those who see the world in a different way must be (a) subject to different (presumably lesser) information, (b) incompetently processing the information, or (c) biased (Ross and Ward 1996). Here, once the teacher has determined that her student doesn’t care about learning, she may resist alternative perspectives. When a colleague speaks of the student as responsible and committed, the teacher may dismiss her colleague’s perceptions as unaware of the real story, too “soft” in her expectations, and so forth. Thus, presenting the teacher with contrasting views of the student is unlikely to alter her “true” understanding of the student.

Countering naïve realism is also central to our

intervention. Interactions. Second, improved SPT seems likely to enhance interactions. Prior research on SPT indicates that it could help heighten social awareness (Selman, 2003), improve people’s communication (Nickerson 1999), and contribute to improved conflict resolution (Corcoran and Mallinckrodt 2000). For instance, SPT motivation has improved interactions during negotiations. Those motivated to engage in perspective taking more regularly reach agreements and mutually beneficial outcomes more often than their control counterparts (Galinsky, Maddux, Gilin, &

Mascio, McIntyre & Gehlbach

5

Seeing The Classroom Through Students’ Eyes

White, 2008; Trötschel, Hüffmeier, Loschelder, Schwartz, & Gollwitzer, 2011). Longitudinal investigations suggest that SPT accuracy may also be important to interactions within relationships. Husbands and wives who more accurately read each other report lower rates of conflict in their marriages (Bissonnette, Rusbult, & Kilpatrick, 1997). Thus, teachers who more effortfully and/or more accurately read their students may manifest improved social coordination (Galinsky et al., 2005), thereby interacting with their student in more accommodating, appropriate ways. The perceptions of the teacher in the previous example who decided her student did not care about learning are bound to impact their interactions. Like anyone in a human relationship, teachers will be impacted by whether their relationship partner (in this case her student) invests in the relationship. The teacher may respond to misbehavior with more severe punishment than she would for a student who she believes is trying hard and is committed. The teacher may expend less of her already strained time and energy when she believes that it will be unappreciated or inconsequential to a student that doesn’t care. Additionally, if a teacher attributes her student’s difficulty to the student’s own lack of caring, this belief may undermine her attempts to help. Merged pathways.

At the theoretical level, these two pathways offer important

distinctions: cognitions versus behaviors; perceptions versus interactions; etc. However, in a naturalistic setting, it may be impossible to disentangle these two interrelated pathways. One party’s perceptions affect their behaviors, which in turn, affect the other party’s perceptions and then their corresponding behavior. While we assume that these two pathways are theoretically distinct, in the present field experiment, , we treat them as too inextricably linked to distinguish. The promise of social perspective taking. Given these pathways, SPT seems like a promising lever for influencing TSRs for three reasons: past empirical evidence that SPT matters for relationships, the frequency with which SPT occurs, and the potential of SPT to mitigate bias. First, there are strong empirical connections yoking SPT to multiple aspects of social relationships (Hall, Andrzejewski, & Yopchick, 2009). SPT may contribute to conflict resolution (Corcoran & Mallinckrodt, 2000), supportiveness (Verhofstadt, Buysse, Ickes, Davis, & Devoldre, 2008), and more positive social interactions (Gruhn, Rebucal, Diehl, Lumley, & Labouvie-Vief, 2008). Experimental manipulations of SPT have facilitated altruism (Batson, Early, & Salvarani, 1997), diminished aggression (Richardson, Green, & Lago, 1998), and enhanced perceptions of perceivertarget similarity (Davis, Conklin, Smith, & Luce, 1996). These experiments have also established

Mascio, McIntyre & Gehlbach

6

Seeing The Classroom Through Students’ Eyes

that SPT is malleable, in each case using simple manipulations to bolster SPT motivation or accuracy in a treatment group relative to a control group. Second, we posit that SPT affects two of the foundational building blocks of TSRs (i.e., perceptions and interactions). Because these pathways are continually activated throughout any given school day, interventions improving SPT could be felt many times a day – thereby magnifying an intervention’s impact. Furthermore, because the development of these relationships involve recursive social processes (see Cohen, Garcia, Purdie-Vaughns, Apfel, & Brzustoski, 2009), small improvements in interactions early in the school year may ultimately yield big dividends through a positive spiral of subsequent interactions. Third, improving teachers’ SPT can help them overcome their cognitive biases, enhancing their perception of, and interactions with, students.

Some SPT interventions might make

participants more accurate perspective takers, while other interventions might make participants engage in the process more frequently. However, still other SPT interventions might simply serve to remove the haze of biased perceptions as perceivers try to understand a target. As these processes unfolded we anticipated that the nature of teachers’ SPT attempts might change by seeing students less as members of an outgroup (Hewstone, Rubin, & Willis, 2002), and more as someone who was similar to themselves (Ames, 2004). By better understanding how their students experience a situation, teachers can recognize that their preconceived explanations are not the only rational explanation, and are not based on the only set of applicable evidence.

The Present Study This study grew out of a professional development exercise designed to improve teachers’ insights into the behaviors of a hard-to-understand focal student. Because past participants reported anecdotal benefits from the insights they gained, we wanted to systematically evaluate whether the SPT induction actually improves TSRs through a field experiment. The 90-minute training walked teachers through the SPT induction with a partner while focusing on a focal student whose behavior they did not understand. During the induction teachers: 1) Identifies a puzzling behavior or key incident that they would like to better understand. 2) Share the detailed stories of the focal student’s behaviors with a partner.

Mascio, McIntyre & Gehlbach

7

Seeing The Classroom Through Students’ Eyes

3) Assume the role of the focal student and retell the same stories from this new perspective (in the first-person). 4) Present background information about the focal student and themselves to provide their partner with additional insights about the situation. 5) With their partner’s help, develop multiple hypotheses for why the focal student engages in the behaviors. 6) With their partner’s help, identify concrete action steps they can take to “test” the plausibility of different hypotheses. We anticipated that the induction would improve teachers’ understanding of their problematic students, by overcoming many of the cognitive biases that may impede SPT. Actively taking the perspective of their student, through role-taking (step 3) should help ameliorate naïve realism. We hoped that identifying relevant contextual information (step 4) would help to overcome the fundamental attribution error. Identifying multiple hypotheses (step 5) further mitigates naïve realism. When combined with consciously creating a plan for assessing those hypotheses (step 6), we hoped these hypotheses would decrease confirmation bias. Working with a partner should reinforce all of these efforts and enhance the accuracy of teachers’ perceptions (Herzog & Hertwig, 2009). Increasing teachers’ motivation and accuracy for understanding their students should facilitate a corresponding improvement in TSR (Gehlbach, 2010). As detailed in our Statement of Transparency (see Appendix A), we expected final teacher surveys to show that, compared to teachers in the control group, teachers in the treatment group would: a) Report more positive and less negative TSRs with their focal student. b) Report putting more effort into taking the perspective of their focal student. c) Report greater perceived similarity with their focal student. For the focal students of teachers in the treatment group (as compared to the focal students of control-group teachers) we expected to: d) See fewer disciplinary incidents at the time of final data collection. (Use of disciplinary incidents as an outcome was obtained for students in 5th – 8th grade). e) Report more positive TSRs. f) Have greater Academic Achievement, as measured by Trimester Exams and Subject Competencies, in the class taught by the teacher who identified them.

Mascio, McIntyre & Gehlbach

8

Seeing The Classroom Through Students’ Eyes

In accordance with recent recommendations (Cumming, 2014; Simmons, Nelson, & Simonsohn, 2011) we pre-registered our study and the above pre-specified hypotheses at Open Science Framework: https://osf.io/6ca4m/?view_only=86f1736b62ee49a7b3f6e6c97715c6d4

Methods Participants We conducted our study with a charter school network in the Northeast that includes grades K-9. All teachers in the system were eligible to participate. Of the 106 participating teachers, 55 were randomly assigned to the treatment group and 51 to the (waitlisted) control group. (These control-group teachers received the same training during a second session in April 2015). Because co-teaching occurs frequently at the schools, co-teaching elementary school teachers were randomly assigned to condition in pairs, allowing us to maintain random assignment while mitigating the possibility that teaching partnerships would contaminate the control group. Co-teaching arrangements within grades 5-9 were more fluid, precluding our ability to account for this practice, so we used simple random assignment for these teachers.

These teaching

partnerships in grades 5-9 spend less concentrated time together and thus have less opportunity to cause cross-treatment contamination. If any contamination did occur, we presume that it only diminished the effects of our intervention. Central to our study, both control and treatment teachers identified a focal student whose behaviors they struggled to understand. Notably, these focal students were demographically distinct from the rest of the student body at their participating schools (see Table 1). Race and gender predicted which students were selected as focal students. African American males represent 13% of the focal students, while they only make up 6% of the non-focal students (đ?‘? = .047, đ??šđ?‘–đ?‘ â„Žđ?‘’đ?‘&#x; ′ đ?‘ đ??¸đ?‘Ľđ?‘Žđ?‘?đ?‘Ą đ?‘‡đ?‘’đ?‘ đ?‘Ą)1. Likewise, Hispanic males make up 45% of the focal students, and yet comprise only 24% of the non-focal students (đ?œ’ 2 (1) = 12.23, đ?‘? < .001) . Additionally, special education students constitute 20% of the focal students but only 10% of nonfocal students (đ?œ’ 2 (1) = 5.22, đ?‘? = .022). Finally, the 75% of focal students who qualify for free

1

Based on the marginal proportions of students who were African American boys and who were designated focal students, the expected number of African American boys who were designated focal students was only 3.83. This is too small for a đ?œ’ 2 test to give approximately correct p-values. For this test, we instead used a Fisher’s exact test.

Mascio, McIntyre & Gehlbach

9

Seeing The Classroom Through Students’ Eyes

and reduced lunch, is significantly greater than the 62% of lower income students among the rest of the student population (đ?œ’ 2 (1) = 5.23, đ?‘? = .022)2. Table 1: Student characteristics, identifying the proportional difference between those students who behaviors were identified as being difficult to understand, and those who were not Student Characteristics Total # % Among % Among Non-Focal Focal Students Students DEMOGRAPHICS Male 414 48 73 White 285 37 20 ELL 83 11 10 Free & Reduced Lunch 484 62 75 Special Education 75 10 20 Been Retained 11 1 7 GENDER x RACIAL INTERACTIONS Asian Female 9 1 2 Asian Male 13 2 0 African American Female 49 6 8 African American Male 53 6 13 Hispanic Female 196 25 1 Hispanic Male 210 24 45 American Indian Female 1 2 2 American Indian Male 2 0 0 White Female 162 21 5 White Male 135 16 15 Several teachers identified the same students -- thus, our 106 teachers identified a total of 85 focal students. Of these students, we matched school records to 75 of the students – the decrease was primarily due to teacher error in inputting students’ ID numbers when originally selecting their focal student. We were able to utilize student survey results from 69 of the focal students. The attrition of these final six students stemmed largely from the fact that the survey was not developmentally appropriate for, and thus not administered to any kindergarten and first grade students even though we included their teachers in the study.

Measures While we assessed multiple constructs, our pre-registered hypotheses center on the measures of TSR, SPT, and similarity (for a complete list of teacher measures, see the Statement

2

We explicitly compare focal students to non-focal students (as opposed to comparing them to the overall student body) to avoid double-counting the focal students.

Mascio, McIntyre & Gehlbach

10

Seeing The Classroom Through Students’ Eyes

of Transparency in Appendix A). Appendix B lists the complete versions of each of our main assessment scales. We assessed TSRs through Gehlbach et al.’s (2012) scale, which assess the positive and negative aspects of the relationship in separate subscales. We use these two separate scales because the positivity and negativity of a relationship are separate dimensions (Cacioppo & Berntson, 1994). For example, the absence of warm feelings towards a student is not the same as the presence of hostility. In the present study, we find the two scales to be negatively, but only moderately correlated – see Tables 2 and 3. The TSR positivity scale consisted of nine items to assess the positivity of perceptions of the relationship (sample item: “How often do you say something encouraging to <student name>?”).

The TSR-negativity subscale consisted of five

items to assess the negative perceptions of the relationship (sample item: “How often does <student name> put you in a bad mood during class?”). In both scales, items are rated on 5-point response options tailored to individual questions (e.g., “Almost never” through “Almost all the time” for items asking about frequency). We assessed SPT through two separate scales: one for confidence and one for effort. The Table 2: Student Scales including mean, standard deviations, and inter-scale correlations Pearson correlation coefficients Scale Mean SD TSRpos TSRneg SPTE TSR positivity 4.12 0.93 1.00 TSR negativity 1.98 0.87 -.44 1.00 SPT Effort 3.50 0.99 .69 -.21 1.00 Social Perspective Taking Confidence scale (adapted from Gehlbach, Brinkworth & Wang, 2012), Table 3: Teacher scales in November and January including mean, standard deviations, and inter-scale correlations Correlations Scale Mean SD TSR pos TSR neg SPTE Similarity November TSR pos (Nov) 2.97 0.75 1.00 TSR neg (Nov) 2.78 0.54 -.30 1.00 SPTE (Nov) 3.19 0.71 .53 -.03 1.00 Similarity (Nov) 2.40 0.74 .51 .04 .33 1.00 January TSR pos (Jan) 3.35 0.72 1.00 TSR neg (Jan) 2.28 0.55 -.45 1.00 SPTE (Jan) 3.47 0.63 .53 -.27 1.00 Similarity (Jan) 2.51 0.69 .34 .04 .33 1.00 consisted of 6 items to assess the teachers’ perceptions of their ability to take their focal student’s

Mascio, McIntyre & Gehlbach

11

Seeing The Classroom Through Students’ Eyes perspective (sample item: “When you disagree with <student name>, how confident are you that you can figure out what he/she is thinking?”). Each item was rated on a 5-point Likert-type scale ranging from “Not at all confident” to “Extremely confident”. The Social Perspective Taking Effort scale (Gehlbach, et al., 2015) consisted of 6 items to assess the teachers’ perceptions of the amount of effort they put into taking the perspective of their focal student (sample item: “How much effort have you put into figuring out what <student name>'s goals are?”). Each item was rated on 5-point response options tailored to individual questions (e.g., “Almost no effort”, through “A tremendous amount of effort”). The Similarity scale (Gehlbach, Brinkworth, & Harris, 2012) consisted of 6 items to assess the teachers’ perception of their shared characteristics with their focal student (sample item: “How interested do you think you and <student name> are in the same activities?”) using 5-point response options such as “Not at all interested”, through “Extremely interested”. In addition to teacher surveys, we also utilized school records that included studentreported TSR, discipline, student competencies, and test scores (for a complete list of student measures, see our Statement of Transparency in the appendix). The student-reported TSR scale was administered by Panorama Education as part of a larger network-wide student survey, and includes parallel items to the teacher-reported TSR scale. Discipline methods and records differed by school. Only the middle school used an electronic demerit system that could be used for analysis purposes. In the participating schools, instead of grades and report cards each trimester, teachers assess students on specific competencies in the given subject (e.g., students’ abilities to convert from decimals to fractions) – the scoring requirements are standardized across teachers, grades and schools. The scores are subjective in the sense that they are assessed by an individual teacher and are vulnerable to human biases. However, they are more objective than typical grades, in that they do not include common subjective components of grades such as effort and participation. The teachers are collectively trained to assess these individual competencies as objectively as possible. Finally, school leaders reduce subjectivity in this system by focusing on individual skills, such as adding fractions, rather than creating a potentially weighted composite grade of loosely related skills within a subject. In addition to these student competencies, the school provided standardized test scores for each student.

Mascio, McIntyre & Gehlbach

12

Seeing The Classroom Through Students’ Eyes

Procedures At the beginning of the 2014-2015 school year, all teachers attended a professional development session on SPT and the cognitive biases that may impede a teacher’s capacity to ‘read’ his or her students. This initial session remained at a conceptual level; no explicit strategies were provided to take the perspective of others and teachers were not “trained” to do anything in particular. Providing background information so that both the treatment and control group teachers had equivalent baseline levels of knowledge about biases, allowed us to more precisely gauge the impact of the SPT induction activity itself. The training that represents the intervention took place in early November 2014 during a system-wide professional development day. At the time of the training, all participants – treatment and control – identified a focal student whose behaviors they struggled to understand, and then completed a survey about their relationship with, and perceptions of, that student. The data collection sequence described below can be seen in Table 4. Day of the treatment.

Approximately two months after the initial training, all

participating teachers in both the treatment and waitlisted control groups viewed a short Table 4: Data collection chronology for treatment and control teachers Date Treatment Teachers Control Teachers  Meet at central location  Receive individually emailed link  Watch video with reminder and instructions  Watch video with reminder and  Identify student instructions  Complete first 3 survey scales Early November  Identify student  Participate in training  Complete all survey  Complete remaining survey scales scales  Provide hypotheses and proposed  Provide hypotheses action steps and proposed action steps Receive follow-up survey via email asking which proposed steps have Early December been taken, and what understanding was gained Late January

Receive post survey via email survey with all survey scales

February

Schools conduct an all-student survey (grades 2-9), that included perceptions of both control and treatment teachers

May

Schools provide student data including students’ competency scores, attendance, discipline, test scores, and survey results

Mascio, McIntyre & Gehlbach

13

Seeing The Classroom Through Students’ Eyes

(approximately 10-minute) video that reviewed the SPT content from the previous training, and then instructed teachers to identify a focal student whose behaviors they struggled to understand. They completed a survey about their relationship with and views of that student. At the end of the survey, teachers identified hypotheses for the reasons behind their focal student’s puzzling behaviors, and proposed action steps they could take to better understand their focal student. Members of the control group watched the video and took the survey at their regular schools. After completing the survey, these teachers spent the rest of the day doing professional development activities with their administrators (i.e., a “business as usual” control group). Teachers assigned to the treatment condition met at a central location for the training. After watching the video and completing the TSR and SPT survey scales on their focal student, these teachers participated in our intervention training before completing the remaining survey scales and proposing their hypotheses and action steps. At the conclusion of the training, treatment teachers: a) completed the remaining survey scales, b) recorded their hypotheses about their focal student’s motivation/behavior, and c) proposed what action steps they might take to better understand their focal student. The surveys taken by the treatment and control teachers contained all of the same components, including their hypotheses and action steps. We embedded these components as products of the induction activity for the treatment teachers. Meanwhile, the control teachers were also asked to create multiple hypotheses about their focal student’s motivation/behavior, as well as propose actions they could take to better understand their focal student. This allowed us to isolate the treatment to the induction activity, and include the control teachers in the follow-up communication. Follow-up. All participants – both treatment and control teachers – were emailed at the beginning of December 2014 to remind them of the action steps they had identified, and asked to report on the results of those steps that they had proposed. Final data collection. At the end of January 2015, all teachers were asked to complete a survey which included the same scales as were taken at the time of the intervention. In February of 2015 in collaboration with Panorama Education, the charter network conducted student surveys in all schools – this included all students other than kindergarteners and first graders, who schools deemed too young to participate. At the end of the school year, we collected student data including students’ competency scores, attendance, discipline, test scores, and survey results.

Mascio, McIntyre & Gehlbach

14

Seeing The Classroom Through Students’ Eyes

Analytic Approach We conducted our analysis in accordance with the pre-registered methods in our Statement

Results of Transparency. For observed variables, we tested our pre-specified hypotheses using generalized linear regression3. For latent variables we used scale scores. When analyzing students’ subject competencies, we standardized each student’s competencies relative to all other students in the grade on that particular competency. Because test scores were reported as percentiles, we converted them into normal scores by assigning each student a score equal to the value of a standard normal distribution associated with her or his percentile. For example, if a student had a score equal to the 75th percentile, we assigned her a transformed score of 0.67, which is the 75th percentile of the normal distribution. We did this because we suspected that the percentile scale was not equal interval. The difference between a score at the 98th percentile and a scores at the 99th percentile probably represented a much greater difference in true ability than the difference between a score at the 50th percentile and a score at the 51st. If the true distribution of ability is approximately normal, then our method will have the effect of putting the scores onto an equal interval scale, which will make our estimates more precise and interpretable. For student survey outcomes we used hierarchical linear models with random teacher intercepts. For the other outcomes we used linear regression models, equivalent to t-tests with pooled standard deviations. For student survey outcomes and course competencies, where models were fit to a dataset including focal and non-focal students, we included gender and grade level as covariates to improve the precision of our estimates. For teacher outcomes, the sample size was so small that we omitted these student covariates. Across our main findings we adhere to Cumming’s (2014) recommended practice of reporting only confidence intervals (not p-values). In addition to the numerous benefits articulated by Cumming, this approach seems particularly appropriate given our modest sample size. In other words, rather than simply reporting a significant or non-significant result, the point estimate and confidence intervals allow readers to see where the range of plausible values falls.

3

We found a great deal of between-teacher variability in the number of disciplinary incidents. Therefore, to attempt to control for teacher tendencies to identify disciplinary infractions, we defined this outcome as the difference in the number of incidents assigned to a targeted student by a targeting teacher after the intervention as compared to before the intervention.

Mascio, McIntyre & Gehlbach

15

Seeing The Classroom Through Students’ Eyes

Findings Teacher outcomes. In regards to the impact of the intervention on teachers, our data were congruent with several of our hypotheses. We were specifically looking for improvements in teacher-reported TSR, the effort they put into taking the perspective of their focal student, and their sense of similarity with their focal student. Both treatment teachers and control teachers experienced growth in the positivity of their relationships with their focal students (TSR-Pos), but the amount of growth was higher for treated Ě‚ đ?‘‡đ?‘†đ?‘…−đ?‘ƒđ?‘œđ?‘ = 0.55) than for control teachers ( Δ Ě‚ đ?‘‡đ?‘†đ?‘…−đ?‘ƒđ?‘œđ?‘ = 0.28) , and this difference teachers ( Δ

5

Treatment Teachers 4.5

Control Teachers

TSR-Positive Scale Score

4

3.5

3.49 3.18

3.02 3 2.89 2.5

2

1.5

1 November

January Timepoint

Figure 1. Teacher-Student Relationship Positive as reported by teachers at time of intervention and at final data collection. This measures the positive aspects of their relationship with the focal student whose behaviors they find difficult to understand.

Mascio, McIntyre & Gehlbach

16

Seeing The Classroom Through Students’ Eyes was moderate and statistically reliable (đ?‘€đ?‘‘đ?‘–đ?‘“đ?‘“đ?‘’đ?‘&#x;đ?‘’đ?‘›đ?‘?đ?‘’ = .27, đ??śđ??ź: [0.03, 0.50], đ??śđ?‘œâ„Žđ?‘’đ?‘›â€™đ?‘ đ?‘‘ = 0.52); (see Figure 1). Similarly, both groups decreased their negative feelings of their teacher-student relationship (TSR-Neg), but the decrease was larger for treated teachers Ě‚ đ?‘‡đ?‘†đ?‘…−đ?‘ đ?‘’đ?‘” = −0.61) than for controls (Δ Ě‚ đ?‘‡đ?‘†đ?‘…−đ?‘ đ?‘’đ?‘” = −0.32). Once again, the difference was (Δ statistically reliable and moderate in size (đ?‘€đ?‘‘đ?‘–đ?‘“đ?‘“đ?‘’đ?‘&#x;đ?‘’đ?‘›đ?‘?đ?‘’ = .29, đ??śđ??ź: [−0.53, −0.05], đ??śđ?‘œâ„Žđ?‘’đ?‘›â€™đ?‘ đ?‘‘ = 0.52); (see Figure 2). These differences indicate that the training caused teachers to increase the positive perceptions of their relationship with their most difficult-to-understand student, and

5

Treatment Teachers Control Teachers

4.5

TSR-Negative Scale Score

4

3.5

3 2.78 2.77 2.5

2.42 2.14

2

1.5

1 November

January Timepoint

Figure 2. Teacher-Student Relationship Negative as reported by teachers at time of intervention and at final data collection. This measures the negative aspects of their relationship with the focal student whose behaviors they find difficult to understand.

reduce the negative aspects.

Mascio, McIntyre & Gehlbach

17

Seeing The Classroom Through Students’ Eyes 5

Treatment Teachers

Control Teachers 4.5

SPT-Effort Scale Score

4 3.62

3.5

3

3.31

3.22 3.16

2.5

2

1.5

1 November

January Timepoint

Figure 3. Social Perspective Taking Effort as reported by teachers at time of intervention and at final data collection. This measures how hard they work at understanding the focal student whose behaviors they find difficult to understand.

Also from the teacher surveys, we found that treatment teachers increased more on Social Ě‚đ?‘†đ?‘ƒđ?‘‡âˆ’đ??¸ = 0.44 and 0.19, respectively), Perspective Taking Effort (SPT-E), than controls (Δ although the confidence interval for this difference does include 0, (đ?‘€đ?‘‘đ?‘–đ?‘“đ?‘“đ?‘’đ?‘&#x;đ?‘’đ?‘›đ?‘?đ?‘’ = .25, đ??śđ??ź: [−0.05, 0.55], đ??śđ?‘œâ„Žđ?‘’đ?‘›â€™đ?‘ đ?‘‘ = 0.37); (see Figure 3). In a related post-hoc, exploratory finding, we found that the treatment teachers increased more on Social Perspective Taking Confidence Ě‚đ?‘†đ?‘ƒđ?‘‡âˆ’đ??ś 1.02 and 0.57, respectively), and this difference was moderate (SPT-C) than the controls (Δ and statistically reliable (đ?‘€đ?‘‘đ?‘–đ?‘“đ?‘“đ?‘’đ?‘&#x;đ?‘’đ?‘›đ?‘?đ?‘’ = .45, đ??śđ??ź: [0.14, 0.76], đ??śđ?‘œâ„Žđ?‘’đ?‘›â€™đ?‘ đ?‘‘ = 0.67);

Mascio, McIntyre & Gehlbach

18

Seeing The Classroom Through Students’ Eyes

5

Treatment Teachers

Control Teachers 4.5

SPT-Confidence Scale Score

4

3.5 3.33 3

2.5

2.95

2.44 2.38

2

1.5

1 November

January Timepoint

Figure 4. Social Perspective Taking Confidence as reported by teachers at time of intervention and at final data collection. This measures the degree to which they feel correct in their understanding of the focal student whose behaviors they find difficult to understand.

(see Figure 4). Thus, the training might bolster teachers’ efforts to work harder to understand their focal student, and appears to give them more confidence that they do so accurately. In contrast to these findings, treatment and control teachers did not differ in their respective sense of perceived similarity to their focal student ( đ?œ‡Ě‚ đ?‘Ąđ?‘&#x;đ?‘’đ?‘Žđ?‘Ą = 2.49, đ?œ‡Ě‚ đ?‘?đ?‘œđ?‘›đ?‘Ąđ?‘&#x;đ?‘œđ?‘™ = 2.52, đ?‘€đ?‘‘đ?‘–đ?‘“đ?‘“đ?‘’đ?‘&#x;đ?‘’đ?‘›đ?‘?đ?‘’ = .03, đ??śđ??ź: [−0.34, 0.28], đ??śđ?‘œâ„Žđ?‘’đ?‘›â€™đ?‘ đ?‘‘ = 0.04). Note that for similarity we compare post-intervention mean scores rather than growth because the November similarity measure was taken after the intervention. At that time the scores of the treatment teachers were slightly higher than those of the control teachers, although the difference does include 0 and may well have

Mascio, McIntyre & Gehlbach

19

Seeing The Classroom Through Students’ Eyes occurred by chance ( đ?œ‡Ě‚ đ?‘Ąđ?‘&#x;đ?‘’đ?‘Žđ?‘Ą = 2.45, đ?œ‡Ě‚ đ?‘?đ?‘œđ?‘›đ?‘Ąđ?‘&#x;đ?‘œđ?‘™ = 2.34, đ?‘€đ?‘‘đ?‘–đ?‘“đ?‘“đ?‘’đ?‘&#x;đ?‘’đ?‘›đ?‘?đ?‘’ = .11, đ??śđ??ź: [−0.17, 0.39], đ??śđ?‘œâ„Žđ?‘’đ?‘›â€™đ?‘ đ?‘‘ = 0.11) Student outcomes. In regards to student impact, we hypothesized improvements in discipline, student-reported TSR, and student achievement. When we compared the focal students of treatment versus control teachers, we found no statistically reliable difference in the students’ perception of their teacher-student relationship (For TSR-Pos, focal students of treatment teachers were

slightly

more

positive

(

đ?œ‡Ě‚ đ?‘Ąđ?‘&#x;đ?‘’đ?‘Žđ?‘Ą = 4.64, đ?œ‡Ě‚ đ?‘?đ?‘œđ?‘›đ?‘Ąđ?‘&#x;đ?‘œđ?‘™ = 4.41,

đ?‘€đ?‘‘đ?‘–đ?‘“đ?‘“đ?‘’đ?‘&#x;đ?‘’đ?‘›đ?‘?đ?‘’ =

.23, đ??śđ??ź: [−0.21, 0.66], đ??śđ?‘œâ„Žđ?‘’đ?‘›â€™đ?‘ đ?‘‘ = 0.28). For TSR-Neg, focal students of treatment teachers (

đ?œ‡Ě‚ đ?‘Ąđ?‘&#x;đ?‘’đ?‘Žđ?‘Ą = 2.17, đ?œ‡Ě‚ đ?‘?đ?‘œđ?‘›đ?‘Ąđ?‘&#x;đ?‘œđ?‘™ = 2.28, đ?‘€đ?‘‘đ?‘–đ?‘“đ?‘“đ?‘’đ?‘&#x;đ?‘’đ?‘›đ?‘?đ?‘’ = −.11,

đ??śđ??ź: [−0.56, 0.34], đ??śđ?‘œâ„Žđ?‘’đ?‘›â€™đ?‘ đ?‘‘ = 0.13) . )

The change in the number of disciplinary actions

were

slightly

less

negative

Ě‚đ?‘Ąđ?‘&#x;đ?‘’đ?‘Žđ?‘Ą = assigned before and after the treatment was similar for treatment teachers as for control (Δ Ě‚đ?‘?đ?‘œđ?‘›đ?‘Ąđ?‘&#x;đ?‘œđ?‘™ = 5.9, đ?‘€đ?‘‘đ?‘–đ?‘“đ?‘“đ?‘’đ?‘&#x;đ?‘’đ?‘›đ?‘?đ?‘’ = 4.1, đ??śđ??ź: [−6.9, 15.0], đ??śđ?‘œâ„Žđ?‘’đ?‘›â€™đ?‘ đ?‘‘ = 0.25) 10.0, Δ

,

and

the

confidence interval was extremely wide. In examining student achievement, we separately look at student standardized test scores and student competency scores. While the test scores provide only suggestive results, the differences in competency scores are particularly promising. All students in second grade and higher take standardized tests in both math and reading at the beginning of the year and then three more times throughout the year, approximately at the end of each trimester. These tests are norm-referenced to a national sample, and the data reported in Table 5 are standard deviations from average. The test scores of both groups of focal students were slightly below average in reading at the start of the school year as shown in Table 5. However, by the end of the second trimester the students identified by treatment teachers had grown almost twice as much in their reading test scores

Ě‚treat = 0.22, Δ Ě‚đ?‘?đ?‘œđ?‘›đ?‘Ąđ?‘&#x;đ?‘œđ?‘™ = 0.12, đ?‘€đ?‘‘đ?‘–đ?‘“đ?‘“đ?‘’đ?‘&#x;đ?‘’đ?‘›đ?‘?đ?‘’ = .10, đ??śđ??ź: [−0.22, 0.42], đ??śđ?‘œâ„Žđ?‘’đ?‘›â€™đ?‘ đ?‘‘ = (Δ

0.17). In math, both groups of focal students (as well as the rest of the student body) actually declined from the start of the school year to the end of the second trimester, but the focal students Ě‚treat = of treatment teachers did so by approximately half as much as those of control teachers (Δ Ě‚đ?‘?đ?‘œđ?‘›đ?‘Ąđ?‘&#x;đ?‘œđ?‘™ = −0.21, đ?‘€đ?‘‘đ?‘–đ?‘“đ?‘“đ?‘’đ?‘&#x;đ?‘’đ?‘›đ?‘?đ?‘’ = .09, đ??śđ??ź: [−0.20, 0.37], đ??śđ?‘œâ„Žđ?‘’đ?‘›â€™đ?‘ đ?‘‘ = 0.19). −0.12, Δ We did find a difference in student competencies between focal students of treatment versus control teachers. We examine growth between the end of the first trimester (approximately the time of intervention) and the end of the second trimester (when final data Mascio, McIntyre & Gehlbach

20

Seeing The Classroom Through Students’ Eyes

Table 5: Mean achievement for students not selected by teachers, students identified by treatment teachers, and students identified by control teachers Measurement Non-Focal Focal Students of Focal Students of Students Treatment Teachers Control Teachers (n= 1080) (n= 56) (n= 26) + Competencies Non-Focal Classes Trimester 1 2.55 2.27 2.17 Non-Focal Classes Trimester 2 3.00 2.55 2.63 Focal Classes Trimester 1 n/a 2.18 2.24 Focal Classes Trimester 2 n/a 2.63 2.30 ++ Test Scores Reading Start of Year 0.07 -0.07 -0.03 Reading Trimester 1 0.11 0.04 -0.03 Reading Trimester 2 0.43 0.15 0.10 Reading growth from Start of 0.37 0.22 0.12 Year to Trimester 2 Math Start of Year 0.32 0.22 0.24 Math Trimester 1 0.42 0.25 0.15 Math Trimester 2 0.15 0.10 0.02 Math growth from Start of Year -0.17 -0.12 -0.21 to Trimester 2 Notes: + Competency scores are reported on a scale of 1 – 5 ++ Test Scores are reported in standard deviations

collection occurred). In the course taught by the teacher who had identified them, the focal students of the treatment teachers grew 0.46 standard deviations more than the focal students of Ě‚ đ?‘Ąđ?‘&#x;đ?‘’đ?‘Žđ?‘Ą = 0.07, the control teachers on their course competencies (see Table 6); ( đ?‘”đ?‘&#x;đ?‘œđ?‘¤đ?‘Ąâ„Ž Ě‚ đ?‘?đ?‘œđ?‘›đ?‘Ąđ?‘&#x;đ?‘œđ?‘™ = −0.39, đ?‘€đ?‘‘đ?‘–đ?‘“đ?‘“đ?‘’đ?‘&#x;đ?‘’đ?‘›đ?‘?đ?‘’ = .46, đ??śđ??ź: [0.05, 0.89], đ??śđ?‘œâ„Žđ?‘’đ?‘›â€˛ đ?‘ đ?‘‘ = 0.77). đ?‘”đ?‘&#x;đ?‘œđ?‘¤đ?‘Ąâ„Ž It is helpful to understand the focal students’ competency scores (which are rated from 1 to 5) by benchmarking them against the competencies of the rest of the student body, as shown in Figure 5. The mean competency scores of the non-focal students (i.e., most students at the schools) grew from 2.55 in first trimester to 3.00 in second trimester. In the class that is taught by the teacher who had identified them, the focal students of treatment teachers had an average competency score of 2.18 in first trimester, and grew to 2.63 by the end of the second trimester. This growth rate is similar to the rest of the student body, and is reliably greater than for students who were identified by control teachers, who only grew from a 2.24 to a 2.30 over the same time

Mascio, McIntyre & Gehlbach

21

Seeing The Classroom Through Students’ Eyes Ě‚ đ?‘Ąđ?‘&#x;đ?‘’đ?‘Žđ?‘Ą = 0.45, đ?‘”đ?‘&#x;đ?‘œđ?‘¤đ?‘Ąâ„Ž Ě‚ đ?‘?đ?‘œđ?‘›đ?‘Ąđ?‘&#x;đ?‘œđ?‘™ = 0.06, đ?‘€đ?‘‘đ?‘–đ?‘“đ?‘“đ?‘’đ?‘&#x;đ?‘’đ?‘›đ?‘?đ?‘’ = .39, đ??śđ??ź: [0.05, 0.89], đ??śđ?‘œâ„Žđ?‘’đ?‘›â€™đ?‘ đ?‘‘ = (đ?‘”đ?‘&#x;đ?‘œđ?‘¤đ?‘Ąâ„Ž 0.77)4. Table 6: Effect sizes (and confidence intervals) between treatment and control groups at time of final data collection Measure Effect Size Effect Size in Standard in Scale Score Points Deviation Units on a 5-point scale (95% Confidence (95% Confidence Interval of Effect Interval of Effect Size) Size) TEACHER REPORTED MEASURES Teacher-Student Relationship Positive 0.45 0.31 (0.00, 0.91) (0.00, 0.63) Teacher-Student Relationship Negative+ -0.54 -0.29 (-0.99, 0.09) (-0.53, 0.05) Social Perspective-Taking Confidence 0.62 0.37 (0.17, 1.07) (0.10, 0.64) Social Perspective-Taking Effort 0.49 0.31 (0.06, 0.92) (0.04, 0.58) Similarity -0.04 -0.03 (-0.45, 0.37) (-0.34, 0.28) STUDENT MEASURES Teacher-Student Relationship Positive Teacher-Student Relationship Negative+ Subject Competencies Math Second Trimester Exams Reading Second Trimester Exams Student Discipline

0.24 (-0.22, 0.69) -0.12 (-0.61, 0.37) 0.46 (0.05, 0.89) 0.06 (-0.27, 0.39) 0.05 (-0.29, 0.40) 0.25 (-0.42, 0.91)

0.23 (-0.21, 0.66) -0.11 (-0.56, 0.34) n/a n/a n/a n/a

Note: + Because of the valence of the TSR-Negative scale a negative treatment effect for TSR-Negative is desirable (i.e., TSRs that are less negative).

4

Because of the small number of students used in this analysis, we were concerned that a t-test would not be reliable. Therefore, we computed the 95% confidence interval using a case-resampling bootstrap.

Mascio, McIntyre & Gehlbach

22

Seeing The Classroom Through Students’ Eyes

5

Treatment Students Control Students

4.5

Non-Focal Students

Student Competency Scores

4

3.5

3

2.5

3 2.63

2.55

2.3

2.24 2.18 2

1.5

1 Trimester 1

Trimester 2 Timepoints

Figure 5. Academic achievement as assessed by student competency scores (on a 1 to 5 scale) at the end of the first and second trimester. Data for Treatment and Control students come from the class that is taught by the teacher who identified them. Data for non-focal students is averaged across all classes.

Mascio, McIntyre & Gehlbach

23

Seeing The Classroom Through Students’ Eyes

Discussion Scholars have theorized that SPT is an important part of a teacher’s capacity to relate to their students – a vital foundation for teaching and learning to occur (Gehlbach & Brinkworth, 2008). Even skilled teachers are likely to have at least one student whose behaviors they find difficult to understand. Frequently, these students then consume an inordinate amount of teachers’ time and energy. Our professional development session attempted to bolster teachers’ SPT skills and motivations, while they focused specifically on a perplexing student. Despite the robust literature on the impotency of teacher professional development programs (Kennedy, 2016) this professional development training clearly impacted the teachers involved. Several months after the training, the treatment teachers had – in comparison to the control teachers – better (both more positive, and less negative) perceived relationships with their focal students. Like any relationship, a TSR is essentially comprised of how each party interacts with and perceives the other. We cannot disentangle whether our intervention changed teachers’ perceptions, interactions, or both. Although, our findings that teachers improved their confidence and effort in reading their focal students suggest that part of the power of the intervention may have been in altering teachers’ perceptions. We do, however, find that students’ perceptions remain unchanged. The small number (N=51) of identified students who we were able to link to survey responses may have contributed to this null finding; While the teachers’ sense of TSR could be improved through either a shift in perception or interaction, the student could not possibly sense an improved TSR without overtly recognizable changes in teacher behavior. Our hypothesis regarding disciplinary actions was based on the rationale that the students whose behaviors were difficult to understand were also likely to receive more disciplinary actions from the teachers who identified them as such. We believed that once teachers were equipped with other tools to understand their focal students, they would find more effective means of interacting and intervening with those students and thus use less punitive discipline. Unlike another recent study (Okonofua, Paunesku, & Walton, 2016), we did not, find this difference either in total second trimester disciplinary actions, or change in number of disciplinary actions, between the treatment and control teachers with their focal students. We suspect that the finding is caused in part by the large inter-student variation in receiving demerits and the large inter-teacher variation in the use of the demerit system. Some teachers hardly used the computerized demerit system at Mascio, McIntyre & Gehlbach

24

Seeing The Classroom Through Students’ Eyes all – with focal or non-focal students – while others relied on it prolifically. For a sample of this size, it may have been unrealistic for any treatment effect to be seen through the great variation. Of course, it may also be that the intervention was insufficiently potent to change teachers’ approaches to discipline – if these habits are deeply ingrained, they may be hard to break. A particularly promising finding is the treatment effect on student achievement. Although our findings with respect to test scores are plausibly explained by chance, the differences we find on students’ competency scores appear statistically reliable. Relative to the overall student body, the students identified as hard to understand appeared to have lower competencies. From this lower starting point, the control teachers’ focal students grew little in their competencies between the first and second trimester. By contrast, students of treatment teachers grew substantially, approximating the growth of the rest of the student body. One possible explanation for the treatment effect on student competencies is that the training has caused the treatment teachers to assess the focal students in a more generous or lenient manner – this is a common criticism of findings based on grades. While this explanation remains a possibility, it is complicated by the important difference between grades and the competencies in question. The participating school network purposefully standardizes their competency score requirements across teachers, grades and schools, and trains teachers on how to assess these competencies in an unbiased fashion.

These efforts of standardization make their process

qualitatively different than how schools assign grades, and decreases (though presumably does not eliminate) the amount of teacher bias inherent in the scores themselves. With an intervention designed to enhance a complex, multifaceted aptitude such as SPT in a field setting, researchers rarely learn exactly which aspect of an intervention causes which specific outcomes. While other explanations are certainly plausible, our best guess is that the effects of the intervention were around mitigating teachers’ biases (e.g., the fundamental attribution error, confirmation bias, and naïve realism). Specifically, because teachers identified focal students whose behaviors they struggled to understand, we worried that those teachers might develop misunderstandings that would reinforce negative biases towards those students. After our intervention, the control group teachers continued to have poorer TSRs with their focal students, likely contributing to those students falling further behind their classmates. We think the training may have helped treatment teachers to alleviate their negative biases towards their focal students. The training improved teachers’ perceptions of their TSRs, presumably through their perceptions

Mascio, McIntyre & Gehlbach

25

Seeing The Classroom Through Students’ Eyes

of and/or interactions with these students. As a downstream consequence of the intervention, students of treatment teachers received higher scores on their trimester competencies.

Implications Teachers and administrators commonly lament the ineffectiveness of professional development workshops. The fact that this 90-minute workshop had a statistically reliable effect despite the small number of teachers and their students is important in its own right. Furthermore, the effectiveness of this intervention falls outside of the more typical professional development approaches of increasing either teachers’ technical content knowledge or their pedagogical content knowledge (Shulman, 1987). Rather than instructing teachers in what to do with their students, the training provides them with tools to improve how they think about their students and what they do with them. Improving this kind of thinking is key to developing a teacher’s decisional capital, one of the three central components to a teacher’s professional capital (Fullan, Rincón-Gallardo, & Hargreaves, 2015). By scaffolding a thought-based induction that is directly linked to psychological theory, but is immediately applicable to their daily practice, this training may further empower teachers to grow professionally. Important practical and theoretical implications also come from considering the demographics of the focal students, being over-represented by African-American and Hispanic males, as well as students in special education and on free and reduced lunch. These populations commonly suffer from an achievement gap, and thus any theoretically-grounded training that can help teachers form relationships with and work with these students more effectively is valuable. Future studies may investigate whether teachers find specific student populations more difficult to understand, and how these teacher-level challenges affect student-level achievement gaps.

Limitations Several limitations affected this study. First, the small number of teachers and students impeded our ability to reach statistically reliable findings. Hopefully, similar studies in the future can retain greater statistical power. Second, we know little regarding the fidelity in teachers’ use of the training in the months after the intervention. Often in field experiments, researchers cannot evaluate the difference between what teachers are shown to do during a professional development session, and then what they in fact do once back in their classrooms. Our follow-up email several weeks after the Mascio, McIntyre & Gehlbach

26

Seeing The Classroom Through Students’ Eyes treatment was meant to address fidelity in two ways: 1) it would act as a “booster shot” to remind teachers of the action steps they had proposed, and 2) they would report to us whether they had in fact taken those steps, thus allowing us to control for fidelity in our analysis. Unfortunately, very few teachers responded to the follow-up email, which decreased any potential impact it could have and denied us the ability to statistically control for fidelity. Last, we know little about the mechanisms or pathways through which the intervention affected our outcomes of interest. Perhaps the training functioned primarily to mitigate teachers’ biases; perhaps teachers became more motivated to take the perspective of their chosen focal student; perhaps teachers’ SPT accuracy improved; perhaps all three are true. Likewise, we know little about whether the intervention primarily affected teachers’ perceptions or interactions with students or both. Future studies – which may need to be based in more highly controlled laboratory settings – may be able to tell us more about how the intervention impacts these outcomes.

Mascio, McIntyre & Gehlbach

27

Seeing The Classroom Through Students’ Eyes

Conclusion The relationship between teachers and students matters critically for students and their learning. At the crux of these relationships is a teacher’s ability to understand their students; students who are difficult to understand may be hard to connect with and to teach. In this study, the students identified as such also had the demographic characteristics who are often poorly served by schools – regarding race, gender, SES and special education status (Fruchter et al., 2012). This same population is rightfully the focus of many current reform efforts (Hargreaves et al., 2010).

Teachers and schools expend a great deal of energy and resources in remediating

achievement gaps between these students and their higher achieving peers, but commonly these expenditures lead to disappointing results and mixed reception by the teaching community (Donnell & Gettinger, 2015; Holme & Rangel, 2012). While there is no silver bullet to improve schooling, our intervention suggests a potentially valuable path forward. By uniting theory and practice, our professional development induction enhances teachers’ ability to understand those who are both most difficult and most in need of being understood. Rather than focusing on a single skill to be used in a narrow context, we have provided teachers with a way of thinking that can be used with all students across multiple contexts and help them build stronger TSRs. Instead of dictating what teachers should or should not do in their practice, we hope our intervention empowers teachers by sharpening their understanding of challenging students and allowing them to use their considerable professional capital.

The authors would like to acknowledge the technical support provided by Zoe Yang.

Acknowledgements Funding from Panorama Education supported this research. The conclusions reached are those of the investigators and do not necessarily represent the perspectives of the funder.

Mascio, McIntyre & Gehlbach

28

Seeing The Classroom Through Students’ Eyes

References Ames, D. (2004). Strategies for social inference: A similarity contingency model of projection and stereotyping in attribute prevalence estimates. Journal of Personality and Social Psychology, 87(5), 573–585. Barge, S., & Gehlbach, H. (2012). Using the theory of satisficing to evaluate the quality of survey data. Research in Higher Education, 53(2), 182-200. doi: 10.1007/s11162-0119251-2 Batson, C. D., Early, S., & Salvarani, G. (1997). Perspective taking: Imagining how another feels versus imagining how you would feel. Personality and Social Psychology Bulletin, 23(7), 751-758. Bissonnette, V. L., Rusbult, C. E., & Kilpatrick, S. D. (1997). Empathic accuracy and marital conflict resolution. In W. J. Ickes (Ed.), Empathic accuracy (pp. 251-281). New York, NY, US: Guilford Press Brinkworth, M. E., & Gehlbach, H. (2015). Perceptual barriers to teacher-student relationships: Overcoming them now and in the future. In C. Rubie-Davies & J. M. Stephens (Eds.), The Social Psychology of the Classroom International Handbook. Cacioppo, J. T., & Berntson, G. G. (1994). Relationship between attitudes and evaluative space: A critical review, with emphasis on the separability of positive and negative substrates. Psychological Bulletin, 115(3), 401–423. doi:10.1037/0033-2909.115.3.401 Cialdini, R. B. (2009). Influence: Science and practice (5th ed.). Boston, MA: Pearson. Cohen, G. L., Garcia, J., Purdie-Vaughns, V., Apfel, N., & Brzustoski, P. (2009). Recursive processes in self-affirmation: Intervening to close the minority achievement gap. Science, 324(5925), 400-403. doi: 10.1126/science.1170769 Corcoran, K. O., & Mallinckrodt, B. (2000). Adult attachment, self-efficacy, perspective taking, and conflict resolution. Journal of Counseling & Development, 78(4), 473-483. Cornelius-White, J. (2007). Learner-centered teacher-student relationships are effective: A metaanalysis. Review of Educational Research, 77(1), 113-143. doi: 10.3102/003465430298563 Corno, L., Cronbach, L. J., Kupermintz, H., Lohman, D. F., Mandinach, E. B., Porteus, A. W., & Talbert, J. E. (2002). Remaking the concept of aptitude: Extending the legacy of Richard E. Snow. Mahwah, NJ: Lawrence Erlbaum Associates. Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25(1), 7-29. doi: 10.1177/0956797613504966 Davis, M. H., Conklin, L., Smith, A., & Luce, C. (1996). Effect of perspective taking on the cognitive representation of persons: A merging of self and other. Journal of Personality & Social Psychology, 70(4), 713-726. Davis, M. H. (1996). Empathy: A social psychological approach. Boulder: Westview Press.

Mascio, McIntyre & Gehlbach

29

Seeing The Classroom Through Students’ Eyes

Davison, A. C., & Hinkley, D. V., (1997). Bootstrap Methods and their Application, New York, NY: The Cambridge University Press. Donnell, L. A., & Gettinger, M. (2015). Elementary school teachers' acceptability of school reform: Contribution of belief congruence, self-efficacy, and professional development. Teaching and Teacher Education, 51, 47-57. Eyal, T., & Epley, N. (2010). How to seem telepathic: Enabling mind reading by matching construal. Psychological Science, 21(5), 700-705. Fruchter, N., Hester, M., Mokhtar, C., & Shahn, Z. (2012). Is Demography Still Destiny? Neighborhood Demographics and Public High School Students' Readiness for College in New York City. A Research and Policy Brief. Annenberg Institute for School Reform at Brown University (NJ1). Fullan, M., Rincón-Gallardo, S., & Hargreaves, A. (2015). Professional capital as accountability. Education Policy Analysis Archives, 23(15), 1-22. Gable, S. L., Reis, H. T., & Downey, G. (2003). He said, she said: A quasi-signal detection analysis of daily interactions between close relationship partners. Psychological Science, 14(2), 100-105. Galinsky, A. D., Ku, G., & Wang, C. S. (2005). Perspective-taking and self other overlap: Fostering social bonds and facilitating social coordination. Group Processes & Intergroup Relations, 8, 109-124. Galinsky, A. D., Maddux, W. W., Gilin, D., & White, J. B. (2008). Why it pays to get inside the head of your opponent: The differential effects of perspective taking and empathy in negotiations. Psychological Science, 19(4), 378-384. Galinsky, A. D., & Moskowitz, G. B. (2000). Perspective-taking: Decreasing stereotype expression, stereotype accessibility, and in-group favoritism. Journal of Personality and Social Psychology, 78(4), 708-724. doi: 10.1037//0022-3514.78.4.708 Gehlbach, H. (2010). The social side of school: Why teachers need social psychology. Educational Psychology Review, 22(3), 349-362. doi:10.1007/s10648-010-9138-3 Gehlbach, H., & Brinkworth, M. E. (2008). Motivated thinkers and the mistakes they make: The goals underlying social cognitions and their consequences for achievement. In M. L. Maehr, S. Karabenick, & T. Urdan (Eds.), Advances in motivation and achievement: Social psychological perspectives. (Vol. 15, pp. 119-144). Bingley, UK: Emerald. Gehlbach, H., & Brinkworth, M. E. (2012). The social perspective taking process: Strategies and sources of evidence in taking another’s perspective. Teachers College Record, 114(1). Gehlbach, H., Brinkworth, M. E., & Harris, A. D. (2012). Changes in teacher-student relationships. British Journal of Educational Psychology, 82, 690-704. doi:10.1111/j.2044-8279.2011.02058.x Gehlbach, H., Brinkworth, M. E., King, A. M., Hsu, L. M., McIntyre, J., & Rogers, T. (2016). Creating Birds of Similar Feathers: Leveraging Similarity to Improve Teacher–Student Relationships and Academic Achievement. Journal of Educational Psychology. doi:dx.doi.org/10.1037/edu0000042 Gehlbach, H., Brinkworth, M. E., & Wang, M.-T. (2012). The social perspective taking process: Mascio, McIntyre & Gehlbach

30

Seeing The Classroom Through Students’ Eyes What motivates individuals to take another’s perspective? Teachers College Record, 114(1), 29. Gehlbach, H., Marietta, G., King, A., Karutz, C., Bailenson, J. N., & Dede, C. (2015). Many ways to walk a mile in another’s moccasins: Type of social perspective taking and its effect on negotiation outcomes. Computers in Human Behavior, 52, 523-532. doi:10.1016/j.chb.2014.12.035 Gehlbach, H., & Robinson, C. (in press). Commentary: The foundational role of teacher-student relationships. In K. R. Wentzel & G. Ramani (Eds.), Social Influences on SocialEmotional, Motivation, and Cognitive Outcomes in School Contexts. United Kingdom: Informa. Gilbert, D. T. (1995). Attribution and interpersonal perception. In A. Tesser (Ed.), Advanced Social Psychology (pp. 98–147). New York: McGraw-Hill. Goodenow, C. (1993). Classroom belonging among early adolescent students: Relationships to motivation and achievement. The Journal of Early Adolescence, 13(1), 21-43. doi:10.1177/0272431693013001002 Griffin, D. W., Dunning, D., & Ross, L. (1990). The role of construal processes in overconfident predictions about the self and others. Journal of Personality and Social Psychology, 59(6), 1128–1139. Gruhn, D., Rebucal, K., Diehl, M., Lumley, M., & Labouvie-Vief, G. (2008). Empathy across the adult lifespan: Longitudinal and experience-sampling findings. Emotion, 8(6), 753765. Hall, J. A., Andrzejewski, S. A., & Yopchick, J. E. (2009). Psychosocial correlates of interpersonal sensitivity: A meta-analysis. Journal of Nonverbal Behavior, 33(3), 149– 180. Hargreaves, A., Lieberman, A., Fullan, M., & Hopkins, D. (Eds.). (2010). Second international handbook of educational change (Springer international handbooks of education ; v. 23). Dordrecht ; London: Springer. Herzog, S. M., & Hertwig, R. (2009). The wisdom of many in one mind: Improving individual judgments with dialectical bootstrapping. Psychological Science, 20(2), 231-237. Hewstone, M., Rubin, M., & Willis, H. (2002). Intergroup bias. Annual Review of Psychology, 53(1), 575-604. Holme, J. J., & Rangel, V. S. (2012). Putting School Reform in Its Place Social Geography, Organizational Social Capital, and School Performance. American Educational Research Journal, 49(2), 257-283. Ickes, W. J. (Ed.). (1997). Empathic accuracy. New York: Guilford. Juvonen, J. (2007). Reforming middle schools: Focus on continuity, social connectedness, and engagement. Educational Psychologist, 42(4), 197–208. Kennedy, M. M. (2016). How does professional development improve teaching? Review of Educational Research, doi:10.3102/0034654315626800 Ku, G., Wang, C. S., & Galinsky, A. D. (2010). Perception through a perspective-taking lens: Mascio, McIntyre & Gehlbach

31

Seeing The Classroom Through Students’ Eyes

Differential effects on judgment and behavior. Journal of Experimental Social Psychology, 46(5), 792-798. doi: 10.1016/j.jesp.2010.04.001 Liberman, V., Minson, J. A., Bryan, C. J., & Ross, L. (2011). Naïve realism and capturing the “wisdom of dyads”. Journal of Experimental Social Psychology. doi:10.1016/j.jesp.2011.10.016 Loftus, E. F. (2003). Make-believe memories. American Psychologist, 58(11), 867-873. doi:10.1037/0003-066x.58.11.867 Lord, C. G., Lepper, M. R., & Preston, E. (1984). Considering the opposite: A corrective strategy for social judgment. Journal of Personality and Social Psychology, 47(6), 1231-1243. Montoya, R. M., Horton, R. S., & Kirchner, J. (2008). Is actual similarity necessary for attraction? A meta-analysis of actual and perceived similarity. Journal of Social and Personal Relationships, 25(6), 889-922. doi:10.1177/0265407508096700 Murdock, T. B. (1999). The social context of risk: Status and motivational predictors of alienation in middle school. Journal of Educational Psychology, 91(1), 62-75. doi:10.1037/0022-0663.91.1.62 Murdock, T. B., & Miller, A. (2003). Teachers as sources of middle school students' motivational identity: Variable-centered and person-centered analytic approaches. The Elementary School Journal, 103(4), 383-399. doi: 10.1086/499732 Okonofua, J. A., Paunesku, D., & Walton, G. M. (2016, online). Brief intervention to encourage empathic discipline cuts suspension rates in half among adolescents. Proceedings of the National Academy of Sciences, 1-6. doi:10.1073/pnas.1523698113 Pianta, R. C. (1999). Enhancing relationships between children and teachers. American Psychological Association. Richardson, D. R., Green, L. R., & Lago, T. (1998). The relationship between perspective-taking and nonaggressive responding in the face of an attack. Journal of Personality, 66(2), 235256. Roorda, D., Koomen, H., Split, J. L., & Oort, F. J. (2011). The influence of affective teacherstudent relationships on students' school engagement and achievement: A meta-analytic approach. Review of Educational Research, 81(4), 493-529. doi:10.3102/0034654311421793 Rosenthal, R., & Jacobson, L. (1968). Pygmalion in the classroom: Teacher expectation and pupils’ intellectual development. New York: Holt Rinehart and Winston. Ross, L. (1977). The intuitive psychologist and his shortcomings: Distortions in the attribution process. In L. Berkowitz (Ed.), Advances in Experimental Social Psychology (Vol. 10, pp. 173–220). Orlando: Academic. Ross, L., & Ward, A. (1996). Naive realism in everyday life: Implications for social conflict and misunderstanding. In E. S. Reed & E. Turiel (Eds.), Values and Knowledge (pp. 103– 135). Mahwah: Erlbaum. Sakiz, G., Pape, S. J., & Hoy, A. W. (2012). Does perceived teacher affective support matter for middle school students in mathematics classrooms? Journal of School Psychology, 50(2), 235-255. doi: http://dx.doi.org/10.1016/j.jsp.2011.10.005 Mascio, McIntyre & Gehlbach

32

Seeing The Classroom Through Students’ Eyes

Selman, R. L. (2003). The promotion of social awareness: Powerful lessons from the partnership of developmental theory and classroom practice. New York: Russell Sage Foundation. Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1-22. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359-1366. doi: 10.1177/0956797611417632 Trötschel, R., Hüffmeier, J., Loschelder, D. D., Schwartz, K., & Gollwitzer, P. M. (2011). Perspective taking as a means to overcome motivational barriers in negotiations: When putting oneself into the opponent's shoes helps to walk toward agreements. Journal of Personality and Social Psychology, 101(4), 771-790. doi: 10.1037/a0023801 Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185 (4157), 1124–1131. Verhofstadt, L. L., Buysse, A., Ickes, W., Davis, M., & Devoldre, I. (2008). Support provision in marriage: The role of emotional similarity and empathic accuracy. Emotion, 8(6), 792802. Wason, P. C. (1960). On the failure to eliminate hypotheses in a conceptual task. The Quarterly Journal of Experimental Psychology, 12, 129–140. Wentzel, K. R., Battle, A., Russell, S. L., & Looney, L. B. (2010). Social supports from teachers and peers as predictors of academic and social motivation. Contemporary Educational Psychology, 35(3), 193-202. doi: 10.1016/j.cedpsych.2010.03.002 Wentzel, K. R. (1997). Student motivation in middle school: The role of perceived pedagogical caring. Journal of Educational Psychology, 89(3), 411-419. doi: 10.1037/00220663.89.3.411

Mascio, McIntyre & Gehlbach

33

Seeing The Classroom Through Students’ Eyes

Appendix A – Statement of Transparency Background to the present study: This study grew out of a professional development exercise that has been given to teachers and school leaders multiple times over the course of the past 8 years. The exercise consists of a social perspective taking (SPT) induction in which partners take turns describing a person from their professional work context whom they find challenging to understand. With the help of their partner, each person proceeds through several steps to try to better understand the behaviors, motives, point of view, etc. of the perplexing target person. Anecdotally, participants report feeling helped by the insights they gained about their target person – thus, providing the motivation to systematically evaluate whether the SPT induction actually improves relationships. If the induction benefits participants, the hope is ultimately to learn more about why the intervention is effective and which steps are most important. An initial pilot study assessed the overall induction, and an internet-based study was conducted to attempt to tease apart the efficacy of its components. However, the present study represents the first randomized control trial of the induction. Overview of the current study: We conducted our randomized control trial with a local charter school network in the Northeast that includes grades K-9. All teachers in the system were eligible for participation. Of these teachers, 106 agreed to participate – 55 were randomly assigned to the treatment group and 51 to the control group. Both groups had previously been introduced to SPT and the cognitive biases that may impede it at a conceptual level; neither group was given any explicit strategies to take the perspective of others during this initial training. The training that represented the intervention took place on November 4th, 2014 during a system-wide professional development day. Because a great deal of co-teaching takes place at the schools, we worried that it would have an impact on the difference found between treated and control teachers. To partially mitigate the influence of this practice, co-teaching pairs of elementary school teachers were jointly assigned to condition. Co-teaching arrangements within grades 5-9 were much more fluid, precluding our ability to account for this practice. At the time of the training, all participants – treatment and control – identified a specific student that they have difficulty understanding, and then completed a survey about their relationship with, and perceptions of, that student. Treatment Group – Teachers assigned to the treatment condition met at a central location for the training. Before the training began, teachers viewed a short (approximately 10-minute) video that reviewed the SPT content from the previous training, and then instructed teachers to identify a specific student that they have difficulty understanding. They were then given a link to, and asked to complete, a survey about their relationship with and views of that student (the first 3 measures in Table 2) on their laptops. The treatment group then participated in a 2½-hour training in which teachers focused on the particularly puzzling students they had each identified. In pairs, they walked through a step-by-step process to better understand the student’s perspective. The induction process involved describing incidents of the puzzling behavior to a partner who helped to hypothesize a variety of possible reasons why the student is exhibiting these behaviors. The Mascio, McIntyre & Gehlbach

34

Seeing The Classroom Through Students’ Eyes

induction resulted in the identification of 4 concrete steps to cultivate a better understanding of the student. At the conclusion of the training, treatment teachers: (a) completed the remaining survey scales (see Table 2), (b) recorded their hypotheses about the target student’s motivation/behavior, and (c) proposed what next steps they might take to better understand their target student. Control Group – Teachers assigned to the control condition spent the day at their regular schools, doing professional development activities with their administrators (i.e., a “business as usual” control group). They were informed that they will receive the same training as the treatment teachers during a second session in April, 2015. Control teachers were sent links to the survey and asked to complete the survey during the professional development day. This survey began with the same video viewed by the treatment teachers, asked them to identify a particularly puzzling student, and included all of the survey scales (see Table 2) completed by those teachers in the treatment condition. Similarly to the treatment survey, the control survey asked teachers to identify possible hypotheses for the student’s behaviors and steps they may take to better understand the student. Follow-up – All participants will be emailed at the beginning of December 2014 to remind them of the action steps they had identified, and asked to report on the results of those next steps that they had proposed. Final Data Collection – Approximately 3 months after the original training, final data collections will take place:  All teachers will complete a survey with the same scales as were taken at the time of the training.  We will collect school record data including students’ grades, attendance, discipline, test scores, and survey results. Table 1: Data collection chronology for treatment and control teachers Date Treatment Teachers Control Teachers  

Nov 4th

Dec 8th Jan 20th

Together in central location Watch video with reminder and instructions  Identify student  Complete first 3 survey scales  Participate in training  Complete remaining survey scales  Provide hypotheses and proposed action steps Follow-up survey emailed asking which proposed steps have been taken, and what understanding was gained Email survey with all survey scales

Mascio, McIntyre & Gehlbach

    

Individually emailed link Watch video with reminder and instructions Identify student Complete all survey scales Provide hypotheses and proposed action steps

Follow-up survey emailed asking which proposed steps have been taken, and what understanding was gained Email survey with all survey scales

35

Seeing The Classroom Through Students’ Eyes

Table 2: List of variables in the present study Measure

# of items per construct

TEACHER REPORTED MEASURES 14 Teacher-Student Relationship Social Perspective-Taking Confidence 6 6 Social Perspective-Taking Effort Student Potential 6 Naïve Realism 6 6 Similarity Factors in Past Incident 9 Responsibility in Past Incident 3 Self-Assessment of Past Incident 5 Teacher-Student Relationship from Student Perspective 14 STUDENT MEASURES 14 Student-Reported Teacher-Student Relationship Student-Reported Teacher Social Perspective-Taking Effort 6 SCHOOL RECORD DATA n/a Subject Competencies n/a Trimester Exams n/a Student Discipline Student Attendance n/a Student Test Scores n/a Demographics 8 Note: Bolded measures denote variables used as pre-specified hypotheses for this study Pre-specified hypotheses:    

Compared to the teachers in the control group, teachers in the treatment group will report more positive and less negative Teacher-Student Relationships with their target students at the time of final teacher survey. Compared to the teachers in the control group, teachers in the treatment group will report greater SPT Effort at the time of final data collection. Compared to the teachers in the control group, teachers in the treatment group will report greater perceived Similarity with their identified student at the time of final data collection. Among the middle school students, those identified by the teachers in the treatment group will have fewer Disciplinary incidents in the class taught by the teacher who identified them, as compared to students identified by control group teachers, at the time of final data collection.5

5

The recording system in grades 5 – 8 allowed for use of disciplinary incidents as an outcome. Incidents were not clearly recorded by a particular teacher in the other grades.

Mascio, McIntyre & Gehlbach

36

Seeing The Classroom Through Students’ Eyes  

Compared to the students identified by teachers in the control group, students identified by teachers in the treatment group will have higher Student-Reported Teacher-Student Relationships with the teacher who identified them, at the time of final data collection. Compared to the students identified by teachers in the control group, students identified by teachers in the treatment group will have greater Academic Achievement, as measured by Subject Competencies and Trimester Exams, in the class taught by the teacher who identified them, at the time of final data collection.

Analysis and reporting: 

Data cleaning will proceed as follows: o We will remove participant responses that show evidence of straight-line responding (Barge & Gehlbach, 2012). Specifically, sets of ten or more sequential responses on the same response anchor within the same section of the survey will be converted to missing values. o A naïve graduate assistant will check for fidelity to treatment within the treatment group, based on their responses to the follow-up email in December. Our main analyses will be conducted with all participants included; however, as an exploratory analysis we will also re-run these analyses without these participants who were flagged as being low on fidelity to treatment. Prior to conducting any comparisons between conditions, we will create and finalize composites for all 12 survey scales (i.e., the first 12 measures in Table 2). We anticipate separate subscales for the Teacher-Student Relationship scale (separate positive and negative sub-scales). For academic achievement, a composite will be created for each Subject Competencies. A separate composite will be created for the items on the Trimester Exam. Because of the co-teaching that takes place within the schools, we will compute these two academic achievement scores as follows: o For subject area teachers we will compute students’ Subject Competency score and Trimester exam score for the assigned subject area, and o For “homeroom” teachers we will compute students’ Subject Competency score and Trimester exam score for all subjects that they co-teach. For observed variables, we will test our pre-specified hypotheses using generalized linear regression. For count outcomes we will use negative binomial models, unless the outcome has a high number of 0s, in which case we will use zero-inflated negative binomial models. For latent variables we will use estimated factor scores in a linear regression framework. Some students have been selected by multiple teachers. To account for this complex nesting structure, we will take the following steps. First, for students who are identified by one treatment and one control teacher, we will conduct a within-subjects estimate of treatment effect for all teacher-level outcomes. For each outcome we identified above, we will take the treatment effect as the difference in outcomes between the treatment and control teachers. For example, we will look for a within-student difference between treatment and control teachers on Teacher-Student relationships. Second, in the broader analyses we will perform statistical inference by bootstrapping students and using a wild bootstrap with asymptotic refinement (see, e.g., Davison and Hinkley, 1997).

Mascio, McIntyre & Gehlbach

37

Seeing The Classroom Through Students’ Eyes 

We plan to use two covariates in our analyses: student gender and student grade-level. However, if random assignment fails (i.e., any of our 8 demographic variables are unevenly distributed across treatment conditions) we will also include that variable as a covariate in each of our seven pre-specified analyses. In line with Cumming (2014), results will be reported as effect sizes and 95% confidence intervals rather than focusing on test statistics and p-values. References

Barge, S., & Gehlbach, H. (2012). Using the theory of satisficing to evaluate the quality of survey data. Research in Higher Education, 53(2), 182-200. doi: 10.1007/s11162-0119251-2 Brinkworth, M. E., & Gehlbach, H. (2015). Perceptual barriers to teacher-student relationships: Overcoming them now and in the future. In C. Rubie-Davies & J. M. Stephens (Eds.), The Social Psychology of the Classroom International Handbook. Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25(1), 7-29. doi:10.1177/0956797613504966 Davison, A. C., & Hinkley, D. V., (1997). Bootstrap Methods and their Application, New York, NY: The Cambridge University Press. Gehlbach, H. (2010). The social side of school: Why teachers need social psychology. Educational Psychology Review, 22(3), 349-362. doi:10.1007/s10648-010-9138-3 Gehlbach, H., & Brinkworth, M. E. (2008). Motivated thinkers and the mistakes they make: The goals underlying social cognitions and their consequences for achievement. In M. L. Maehr, S. Karabenick, & T. Urdan (Eds.), Advances in motivation and achievement: Social psychological perspectives. (Vol. 15, pp. 119-144). Bingley, UK: Emerald. Gehlbach, H., Brinkworth, M. E., & Harris, A. D. (2012). Changes in teacher-student relationships. British Journal of Educational Psychology, 82, 690-704. doi:10.1111/j.2044-8279.2011.02058.x Gehlbach, H., Brinkworth, M. E., Hsu, L., King, A., McIntyre, J., & Rogers, T. (online, 2016). Creating birds of similar feathers: Leveraging similarity to improve teacher-student relationships and academic achievement. Journal of Educational Psychology. Gehlbach, H., Brinkworth, M. E., & Wang, M.-T. (2012). The social perspective taking process: What motivates individuals to take another’s perspective? Teachers College Record, 114(1), 29. Gehlbach, H., & Robinson, C. (in press). Commentary: The foundational role of teacher-student relationships. In K. R. Wentzel & G. Ramani (Eds.), Social Influences on SocialEmotional, Motivation, and Cognitive Outcomes in School Contexts. United Kingdom: Informa. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359-1366. doi:10.1177/0956797611417632

Mascio, McIntyre & Gehlbach

38

Seeing The Classroom Through Students’ Eyes

Appendix B – Survey Scales Used Teacher-Student Relationship (TSR) Scale We’d like to get a sense of your interactions with ___ this semester. How much do you enjoy helping _____ learn? How friendly is _____ toward you? How often does _____ ignore something you say? How often do you say something encouraging to _____? During class, how often does ________ talk when you are talking (for instance, when s/he is supposed to be listening)? How respectful is ______ toward you? How excited would you be to have _____ again next year? How motivating does ______ find the activities that you plan for class? How often do you say something that bothers ____? How caring is _____towards you How unfair are the grades that you give ________ in class?

A tremendous amount Extremely friendly Almost all the time

Not at all

Slightly

Somewhat

Quite a bit

Not at all friendly Almost never

Slightly friendly Once in a while

Somewhat friendly

Quite friendly

Sometimes

Frequently

Almost never

Once in a while

Sometimes

Frequently

Almost all the time

Almost never

Once in a while

Sometimes

Frequently

Almost all the time

Not at all respectful

Slightly respectful

Somewhat respectful

Quite respectful

Extremely respectful

Not at all excited

Slightly excited

Somewhat excited

Quite excited

Extremely excited

Not at all motivating

Slightly motivating

Somewhat motivating

Quite motivating

Extremely motivating

Almost never Not at all caring Not at all unfair

Once in a while Slightly caring Slightly unfair

Sometimes

Frequently

Somewhat caring Somewhat unfair

Quite caring Quite unfair

Almost never Extremely caring Extremely unfair A tremendous amount Almost never

How much do you like ______'s Not at all Slightly Somewhat Quite a bit personality? How often does ___ put you in a Almost Once in a Sometimes Frequently never while bad mood during class? Overall, how much does ____ learn Almost A little bit Some Quite a bit A great deal nothing from you? Note: * Blank lines were automatically filled with the initials of the student that the teacher had identified ** Items shaded in grey are for the TSR-Neg scale, while all other items are for TSR-Pos

Mascio, McIntyre & Gehlbach

39

Seeing The Classroom Through Students’ Eyes

Mascio, McIntyre & Gehlbach

40

Seeing The Classroom Through Students’ Eyes

Social Perspective Taking – Effort (SPT-Effort) Scale For the following questions, please reflect on your understanding of _________. During class, how hard do you try to understand what _____ is feeling? When talking with ____, how confident are you that you can understand his/her point of view?

Not hard at all

Slightly hard

Somewhat hard

Quite hard

Extremely hard

Not at all confident

Slightly confident

Somewhat confident

Quite confident

Extremely confident

How much effort have you put into figuring out what _____'s goals are?

Almost no effort

A small amount of effort

Some effort

Quite a bit of effort

A tremendous amount of effort

When you disagree with ____, how confident are you that you can figure out what he/she is thinking? When _____ seems to be in a worse mood than usual, how hard do you try to understand the reasons why? In general, how confident are you that you could accurately guess what motivates ______?

Not at all confident

Slightly confident

Somewhat confident

Quite confident

Extremely confident

Not hard at all

Slightly hard

Somewhat hard

Quite hard

Extremely hard

Not at all confident

Slightly confident

Somewhat confident

Quite confident

Extremely confident

Overall, how much effort do you put into figuring out what _______ is thinking?

Almost no effort

A small amount of effort

Some effort

Quite a bit of effort

A tremendous amount of effort

When ____ is acting strangely, how confident are you that you can Not at all Slightly Somewhat Quite Extremely confident confident confident confident confident figure out why s/he is acting that way? How much do you try to understand A Not at all A little bit Somewhat Quite a bit tremendous _____'s motivations as they are amount doing different classroom activities? Overall, how confident are you that Not at all Slightly Somewhat Quite Extremely you can figure out what ___ is confident confident confident confident confident feeling? How hard do you try to understand Not hard at Slightly Somewhat Extremely Quite hard all hard hard hard _____'s point of view? How confident are you that you can Not at all Slightly Somewhat Quite Extremely understand what _____ is thinking confident confident confident confident confident when you are talking with him/her? Note: * Blank lines were automatically filled with the initials of the student that the teacher had identified ** Items shaded in grey are for the SPT-Confidence scale, while all other items are for SPT-Effort

Mascio, McIntyre & Gehlbach

41

Seeing The Classroom Through Students’ Eyes

Similarity Scale For this section, please reflect on this specific student, as a person. How interested do you think you and ____ are in the same activities? How similar do you think your sense of humor is compared to ____'s sense of humor? How similar do you think your point of view is compared to _____'s point of view? How similar do you think your personality is compared to______'s? How easy is it for you to think of things you and_____ might have in common? Overall, how similar do you think you and ____ are?

Not at all interested

Slightly interested

Somewhat interested

Quite interested

Extremely interested

Not at all similar

Slightly similar

Somewhat similar

Quite similar

Extremely similar

Not at all similar

Slightly similar

Somewhat similar

Quite similar

Extremely similar

Not at all similar

Slightly similar

Somewhat similar

Quite similar

Extremely similar

Not at all similar

Slightly similar

Somewhat similar

Quite similar

Extremely similar

Not at all similar

Slightly similar

Somewhat similar

Quite similar

Extremely similar

Note: * Blank lines were automatically filled with the initials of the student that the teacher had identified

Mascio, McIntyre & Gehlbach

42

Turn static files into dynamic content formats.

Create a flipbook
Issuu converts static files into: digital portfolios, online yearbooks, online catalogs, digital photo albums and more. Sign up and create your flipbook.