Assessing Social-Emotional Learning: A Guide for the Perplexed Chase Nordengren Research Scientist, NWEA
About NWEA • Assessment solutions provider • Measures of Academic Progress (MAP) – growth, skills, reading fluency • Nine million+ students, 8500 districts • Professional Learning • Research services
What are the top socialemotional learning priorities at your school? 5 minutes with a nearby partner
What Teachers in One District Picked •Attention
•“Connection to school”
•Happiness
•(Feeling of) Safety
•Attitude toward learning •Diversity
•Intrinsic motivation
•Self-efficacy
•Awareness of one’s own
•Emotion management
•Maturation
•Self-image
emotions
•(Relationships with)
•Mental health and state
•Sense of control
•Behavior
Family members
of mind
•Social media use
•Communication
•Feeling accepted
•“Ownership and
•Stress
strategies
•(Relationships with)
accountability”
•(Relationship with)
•Confidence
Friends and peers
•Positive mindset
Teachers and adults
•Conflict resolution
•Future aspirations
•Readiness to learn
Common Threads • Important
Common Threads • Important • Lots of different themes
Common Threads • Important • Lots of different themes • Impact learning … and so much more
The field urgently requires much greater clarity about how well, at present, it is able to count some of the things that count. Duckworth, A. L., & Yeager, D. S. (2015). Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44(4), 237–251. https://doi.org/10.3102/0013189X15584327
Outline • Choices in picking an SEL assessment • Our study – Student Engagement Instrument • Best practice in choosing an SEL instrument
Choices, Choices
Big Picture Considerations • Rigor and quality • CASEL Assessment Guide • School Climate Survey Compendium
measuringsel.casel.org
Big Picture Considerations • Rigor and quality • CASEL Assessment Guide • School Climate Survey Compendium • Cost – financial, staff, time • Access to data
Who should take the assessment Students
Teachers (Observation / Opinion)
• Direct window to mindsets
• Can apply strong professional judgement
• May reveal gaps in teachers’ knowledge of students • May cause discomfort or raise parent concerns
• More complex items / assessments • Bias and equity concerns
(Duckworth & Yeager, 2015; McKown, 2017; Wallace et al., 2016)
How to measure Survey Items
Performance Task
• Simple to administer
• More “authentic” demonstration of application
• More straightforward interpretation • Subject to reference bias, social desirability bias
• May be too removed from day-today practice • Still rare and expensive (Duckworth & Yeager, 2015; West et al., 2016)
What to assess (some considerations) • District / school standards – a “crazy quilt” (McKown, 2017) • Student age and development • College and career readiness • What’s being taught
•
Who should take it? •
•
How should it measure? •
•
Students or teachers? Self-report or performance task?
What should I assess? •
School, district, community needs?
5 minutes with your partner
SEI: A Case Study
Student Engagement Instrument • Who: Students (grades 3-12) • How: Survey items (33) • What: • Cognitive engagement (feelings about school / future) • Affective engagement (relationships w/ peers, family, teachers)
Example Items • My family/guardian(s) are there for me when I need them. • Overall, my teachers are open and honest with me. • My education will create many future opportunities for me. • Most of what is important to know you learn in school.
Study Design • Given to 3800 students (grades 3-8) 3x • Interviewed 14 teachers, 8 admins • We studied: • Use as an early indicator • Variation across year • Use by teachers and administrators
Relationship to Achievement
Change over Time
Attendance
Barriers to Use by Teachers • Developmental appropriateness in lower grades • Interpreting scores • Aligning measure to curricular focus • Lack of time and tools
How did administrators use it? • Program placement • Meet mandates • (Limited) work with individual students
• “Having this data piece is just another layer of help”
• Who needs the data from my assessment? • What do I want them to do with it? • How can I support them? 5 minutes with your partner
Conclusions
Setting the right stakes • SEL assessments are not for accountability • Unlikely to predict other outcomes • Can support placement and resource allocation • Can provide formative support for SEL interventions
Meeting the right needs • What mandate(s) does the assessment need to meet? • What does our community want to know? • What data do our current programs need? • Placement? Content selection? Effectiveness?
Linking assessment to action • Can students complete the assessment accurately? • Age groups? Cognitive complexity? SpEd needs? • How will students develop skills / mindsets? • Do our teachers have the resources (time, money, training, community) to use this new data?
Chase Nordengren Research Scientist, NWEA chase.nordengren@nwea.org