Seesaw Early Literacy ESSA Level IV Report by seesawlearning

Seesaw Early Literacy

Logic Model

Study Type: ESSA Evidence Level IV

Prepared for: Seesaw

Prepared by LearnPlatform by Instructure:

Meetal Shah, Ph D , Senior Researcher

Alexandra Lee, Ph D , Senior Researcher

Christina Davis, M PA , Researcher

November 12, 2024

EXECUTIVE SUMMARY

Seesaw engaged LearnPlatform by Instructure, a third-party edtech research company, to develop a logic model for the Seesaw Early Literacy supplemental curriculum program LearnPlatform designed the logic model to satisfy Level IV requirements (Demonstrates a Rationale) according to the Every Student Succeeds Act (ESSA) 1

Logic Model

A logic model provides a program roadmap, detailing program inputs, participants reached, program activities, outputs, and outcomes LearnPlatform collaborated with Seesaw to develop and revise the logic model

Study Design for Seesaw Evaluation

Informed by the logic model, the next phase will focus on planning for an ESSA Level II study to examine the extent to which the Seesaw Early Literacy supplemental curriculum program impacts student learning outcomes

Conclusions

This study provides results to satisfy ESSA evidence requirements for Level IV (Demonstrates a Rationale) A study is currently underway and scheduled for the 2024–25 school year

1 Level IV indicates that an intervention should include a “well-speciﬁed logic model that is informed by research or an evaluation that suggests how the intervention is likely to improve relevant outcomes; and an effort to study the effects of the intervention, that will happen as part of the intervention or is underway elsewhere…” (p. 9, U.S. Department of Education, 2016).

Introduction

The Seesaw team recognizes that despite the growing emphasis on the Science of Reading, many students continue to struggle with developing foundational literacy skills Traditional curriculum tools often lack the evidence-based strategies necessary to support effective, research-based reading instruction There is a need for a supplementary curriculum tool that is developmentally appropriate for younger learners and integrates the principles of the Science of Reading to enhance literacy development The Seesaw Early Literacy supplemental curriculum program provides systematic, explicit instruction in phonological and phonemic awareness, phonics, oral reading ﬂuency, vocabulary, and reading comprehension By aligning with research-based practices, Seesaw can help educators address and improve reading outcomes for all students

The study had the following objectives:

1 Deﬁne the Seesaw logic model and foundational research base

2 Draft an ESSA Level III study design

Annotated Bibliography of Previous Research

1 Citation: Petscher, Y, Cabell, S Q , Catts, H W , Compton, D L , Foorman, B R , Hart, S A , Lonigan, C. J., Phillips, B. M., Schatschneider, C., Steacy, L. M., Terry, N. P., & Wagner, R. K. (2020). How the Science of Reading Informs 21st-Century Education. Reading Research Quarterly, 55(Suppl 1), S267–S282 https://doi org/10 1002/rrq 352

a Citation Type: Theoretical Paper

b Population: N/a

c Design Decision(s) Driven by the Research: Lessons and activities that are based in the science of reading literature

d How did the team leverage the citation to come to this design decision? To understand the diverse needs of students, especially those at risk of reading difficulties The team designed lessons that are adaptable and personalized to cater to different learning styles and needs The research highlights the significance of early identification and intervention for at-risk readers Seesaw includes formative assessment and effective early intervention strategies, incorporating structured and systematic instructional routines that have been validated by research

e If the sample varies signiﬁcantly from your population of interest, please explain why this research is still relevant to your product: N/a

f Evidence of the feature (screen shot, video, or URL): Seesaw’s Early Literacy curriculum supports early intervention strategies based on the Science of Reading such as explicit and systematic phonics instruction, strong phonemic awareness activities, ﬂuency practice, and early reading comprehension strategies.

Lessons from the Phenomenal Phonics collection provides modeling videos, audio support, formative assessments in practice mode so students can check their work, and access to Seesaw’s multimodal tools to ensure a diverse range of learners can show their learning and thinking

Formative Assessment in a Show What You Know activity on a PK lesson on the lowercase letter “ m ” shows how even the youngest learners can show their understanding Audio support on directions and text increases access

Citation: Foorman, B.R., Chen, D. T., Carlson, C., Moats, L., Francis, D.J., & Fletcher, J.M. (2003). Necessity of the alphabetic principle to phonemic awareness instruction. Reading and Writing, 16, 289–324.

a Citation Type:

b Population:

c Design Decision(s) Driven by the Research: Scope and sequence design; phonemic awareness and phonics lessons/activities

d How did the team leverage the citation to come to this design decision? The research highlights the importance and impact of incorporating phonemic awareness alongside alphabetic instruction in kindergarten curricula

e If the sample varies signiﬁcantly from your population of interest, please explain why this research is still relevant to your product: N/a

f. Evidence of the feature (screen shot, video, or URL):

g The ﬁrst 10 weeks of the K Scope and Sequence feature Fun Phonemes lessons alongside the K alphabetic principle/alphabet recognition collection, Alphabet Treasures The order of activities in Alphabet Treasures is scaffolded to begin with a phonemic awareness activity immediately following the “Introduction” in the “Listen” activity

2. Citation: Blevins, W. (2017). A fresh look at phonics: Common causes of failure and 7 ingredients for success. Corwin.

a Citation Type: Book

b Population: N/a

c Design Decision(s) Driven by the Research: Alphabet recognition and phonemic awareness lessons and activities

d How did the team leverage the citation to come to this design decision? Seesaw’s letter books draw on the importance of teaching the alphabet to students through, ‘active exploration of the relationships between letter names, the sounds of the letter names, their visual characteristics' and other Seesaw Early Literacy lessons support 'the motor movement involved in their formation’ (Bear, Templeton, Invernizzi, & Johnston, 1996) This results from direct instruction and multiple exposures to print, including the wide use of alphabet books” (p 4)

e If the sample varies signiﬁcantly from your population of interest, please explain why this research is still relevant to your product: N/a

f Evidence of the feature (screen shot, video, or URL): Lessons in the Letter Books include an audiobook where students can listen to and watch the book being read aloud Then, the next activity allows them to demonstrate or practice their learning by recording themselves reading the letter names and the words. At the corner of the word pages is a help icon students can click on to hear the word being read to them for additional support

Alphabet Treasures lessons further supporting letter name practice, sounds of the letters, and letter formation practice

Citation: Suggate, S P (2016) A meta-analysis of the long-term effects of phonemic awareness, phonics, ﬂuency, and reading comprehension interventions Journal of Learning Disabilities, 49(1), 77-96

a Citation Type: Meta-analysis

b Population: N/a

c Design Decision(s) Driven by the Research: Lessons and activities that involve phonemic awareness and comprehension strategies

d How did the team leverage the citation to come to this design decision? The team designed lessons that go beyond phonics and ﬂuency and include phonemic awareness and comprehension strategies.

e. If the sample varies signiﬁcantly from your population of interest, please explain why this research is still relevant to your product: N/a

f Evidence of the feature (screen shot, video, or URL): Numerous activities included in Seesaw’s phonemic awareness lessons including modeling videos, encouraging students to record themselves reproducing phonemes, identifying the phonemes in isolation, identifying the phonemes in words, adding sounds, changing sounds.

Passages Playhouse activities highlight examples of comprehension strategies (and phonics and ﬂuency) Over the course of a week, the comprehension tasks based on the same passage increase in rigor and go from asking students to identify the character, setting, problem/solution, story details, making an inference, retelling a story, and making a connection.

4 Citation: Burkins, J , & Yates, K (2021) Shifting the balance: 6 ways to bring the science of reading into the blanched literacy classroom Stenhouse

a Citation Type: Book

b Population: N/a

c Design Decision(s) Driven by the Research: Lessons and activities that involve high-frequency words and sight words

d How did the team leverage the citation to come to this design decision? The research cited in this book supports Seesaw’s list of 109 words in High-Frequency Words With Turtle and covers 50% of words in children’s texts The research also

supports some of the dialogue in Seesaw’s scripts and the “Analyze” section of the lessons

e If the sample varies signiﬁcantly from your population of interest, please explain why this research is still relevant to your product: N/a

f. Evidence of the feature (screen shot, video, or URL): High Frequency Words with Turtle Lesson Overview

High Frequency Words with Turtle “Analyze” Activity Example

Example video which includes dialogue in script of what the characters notice about the word GC07 Then FC3 mp4

5 Citation: Saha, N (2022, December 14 ) The Science Behind Decodable Books Metametrics, Inc https://metametricsinc com/the-science-behind-decodable-books/

a Citation Type: Online article

b Population: N/a

c Design Decision(s) Driven by the Research: Lessons and activities that include decodable texts.

d. How did the team leverage the citation to come to this design decision? Research synthesized in this article states that decodable texts help children focus on letter-sound relationships and use decoding strategies Using highly decodable texts implies that students apply letter-sound knowledge more often, are more accurate, and need less help from teachers compared to those using less decodable texts Additionally, combining phonics instruction with decodable texts is more effective for multilingual learners than phonics instruction alone

e If the sample varies signiﬁcantly from your population of interest, please explain why this research is still relevant to your product: N/a

f Evidence of the feature (screen shot, video, or URL): Passages Playhouse lesson examples Also including the lesson plan which includes suggested preview language teachers can use to provide additional support/scaffolds for students and MLLs

6 Citation: Duke, N K , Ward, A E , & Pearson, P D (2021) The science of reading comprehension instruction. The Reading Teacher, 74(6), 663-672. https://doi.org/10.1002/trtr.1993

a Citation Type: Theoretical Paper

b Population: N/a

c Design Decision(s) Driven by the Research: Lessons and activities that support reading comprehension

d How did the team leverage the citation to come to this design decision? Supporting reading comprehension requires a combination of approaches, including phonics, phonemic awareness, print awareness, and word recognition Students need to develop grapho-phonological semantic cognitive ﬂexibility (GSF), morphological awareness, and reading ﬂuency through various reading practices Comprehension instruction should start early and be integrated with these skills, along with teaching text structures and comprehension monitoring Motivation and social context in the classroom play crucial roles in enhancing reading comprehension

Engaging students in discussions and providing motivational reading interventions can signiﬁcantly improve their reading skills

e If the sample varies signiﬁcantly from your population of interest, please explain why this research is still relevant to your product: N/a

f. Evidence of the feature (screen shot, video, or URL): The Passages Playhouse lessons specifically were designed to purposefully integrate phonics, word recognition, reading fluency, and comprehension The various reading fluency activities Passages Playhouse lessons include - echo reading, choral reading, reader’s theater, help foster engaging and motivating reading interactions to help improve reading skills

7 Citation: McBreen, M , & Savage, R (2020) The impact of motivational reading instruction on the reading achievement and motivation of students: A systematic review and meta-analysis Educational Psychology Review Advance online publication https://doi org/10 1007/s10648-020-09584-4

a Citation Type: Meta-analysis

b Population: N/a

c Design Decision(s) Driven by the Research: Lessons and activities that support motivational reading instruction

Reader’s Theater Script for the same lesson: G1PP02 The Big Kick script

d How did the team leverage the citation to come to this design decision? A review of motivational intervention strategies highlighted several methods, such as self-regulatory instruction, promoting reading interest or value, and retraining students’ attributions to foster a ﬂexible mindset. About half of the studies combined multiple strategies, including autonomy-supportive practices. All these approaches signiﬁcantly improved both achievement and motivation outcomes

e If the sample varies signiﬁcantly from your population of interest, please explain why this research is still relevant to your product: N/a

f Evidence of the feature (screen shot, video, or URL): Many of the Phonics Games lessons encourage hands-on activities either with letter/word/image cards (which we provide as additional resources in the lesson plans) or other materials Seesaw’s Alphabet Garden lessons also include hands-on activities where students make a letter with physical manipulatives

Many of the lessons have a Connect activity which also encourages off-line/off-device interactive learning with a partner

In any lesson, students can choose their preferred method of sharing their learning via Seesaw’s multimodal tools Another example of how we encourage student choice is in HFW With Turtle where students can choose a chant to practice spelling a word or choose which digital manipulatives to use to create a word

Additionally, the practice mode of Formative Assessments provides process-oriented feedback Finally, some lessons include self-reﬂection opportunities for students to reﬂect on their learning, thereby focusing on the iterative and continued process and not the product of learning

For a complete review of the research base see (link)

Logic Model

A logic model is a program or product roadmap It identiﬁes how a program aims to impact learners, translating inputs into measurable activities that lead to expected results A logic model has ﬁve core components: inputs, participants, activities, outputs, and outcomes (see Table 1)

Table 1 Logic model core components

Inputs

What the provider invests

Participants Who the provider reaches

Activities

What participants do

Outputs Products of activities

Outcomes Short-term, intermediate, long-term

What resources are invested and/or required for the learning solution to function effectively in real schools?

Who receives the learning solution or intervention? Who are the key users?

What do participants do with the resources identiﬁed in Inputs? What are the core/essential components of the learning solution? What is being delivered to help students/teachers achieve the program outcomes identiﬁed?

What are numeric indicators of activities? (e g , key performance indicators; allows for examining program implementation)

Short-term outcomes are changes in awareness, knowledge, skills, attitudes, and aspirations

Intermediate outcomes are changes in behaviors or actions

Long-term outcomes are ultimate impacts or changes in social, economic, civil or environmental conditions

LearnPlatform reviewed Seesaw resources, artifacts, and program materials to develop a draft logic model Seesaw reviewed the draft and provided revisions during virtual meetings The ﬁnal logic model depicted below (Figure 1) reﬂects these conversations and revisions

Seesaw Early Literacy Logic Model Components Seesaw invests several resources into their Early Literacy supplemental curriculum program embedded within the Seesaw platform including: 2 600+ Lessons with 3,600+ activities grounded in the Science of Reading, 50 + Letter and Word Book lessons, 100+ high-frequency word lessons, premium literacy tools (e.g., Flexcards, Read-with-Me, Reading Fluency Assessment, and Formative Assessments), lessons plans, teacher resources with full scope & sequence and week by week activity recommendations, and professional development (PD) Lessons plans include: learning objectives, standards alignment, printable supporting resources, instructional format, teacher tips, and step-by-step guide on lesson orchestration

Ultimately, the Seesaw Early Literacy program aims to reach PK–2 students, PK–2 educators, families, and district and school administrators Using these program resources, the participants can engage with Seesaw Early Literacy through the following activities:

● PK–2 Students:

○ complete teacher-assigned EL lessons using Seesaw's multimodal learning tools;

○ receive feedback from peers, educators, and family members on lessons;

○ document oﬄine learning activities and add layers of explanation and reﬂection via photos, videos, voice recording, etc ;

○ showcase mastery of learning through their portfolio of work; and

○ complete teacher-assigned formative assessments

● PK–2 Educators:

○ assign EL lessons in Seesaw;

○ model use of seesaw multimodal tools and the learning task at hand;

○ provide personalized feedback to students in text or audio recordings;

○ differentiate instruction e.g. targeted interventions, scaffolds, and small group activities;

○ enable students to document and extend oﬄine learning activities;

○ review student engagement and performance portfolios, standards-aligned skills, and activity-based reports;

○ document mastery of learning through students' portfolio of work;

○ communicate with families about classroom happenings and student progress; and

○ assign formative assessments

● District and School Administrators:

○ support educators with targeted PD and advocate for Seesaw and EL program use;

○ virtually visit classrooms to gain insight into instruction and student progress; and

○ communicate with teachers, students, and families

● Families:

2 The Seesaw platform provides dynamic and developmentally-appropriate, multimodal learning tools and formative assessment capabilities

○ review different levels of communications from school to facilitate supporting learning at home;

○ communicate with school leaders and teachers about student needs;

○ see their child’s learning in the Seesaw Journal and post encouraging comments; and

○ review and celebrate mastery of learning through their child's portfolio of work

Seesaw can examine the extent to which core activities were delivered and participants were reached by examining the following quantiﬁable outputs:

● PK–2 Students:

○ Number and nature of:

■ EL lessons completed and multimodal tools used

■ feedback received from and provided to peers

■ feedback received from educators and family members

■ posts of oﬄine activities

○ Number of:

■ portfolio views

■ formative assessments completed

■ sessions/logins (total)

● PK–2 Educators:

○ Number and nature of:

■ EL lessons & activities assigned

■ teacher-created activities assigned

■ times an activity has been presented/modeled

■ feedback provided to students

■ differentiated instruction via individual or group lessons/activities assigned

■ messages sent to families

■ posts on classroom happenings

■ PD opportunities attended

○ Number of:

■ of portfolio and report views

■ of PDF reports sent to families

■ of formative assessments assigned

● District and School Administrators:

○ Number and nature of PD opportunities provided for educators;

○ Number of class-, school-, or district-level report views; and

○ Number and nature of messages sent to families

● Families:

○ Number of school communications opened/read;

○ Number and nature of feedback given to their child; and

○ Number of portfolio views and engagement with posts

If implementation is successful, based on a review of program outputs, Seesaw can expect the following short-term outcomes: students will have increased proficiency with phonological and phonemic awareness, phonics, oral reading fluency, vocabulary, and/or reading comprehension; have greater opportunities to connect with and learn from peers, educators, and family; increased self-awareness, including strengths and areas for growth; have a growth mindset in learning; and increased proficiency on formative reading assessments Educators’ use of instructional resources tailored to meet the needs of early literacy learners and formative assessments will increase Educators will also report greater access to resources for planning, instruction, and interventions and feeling increased partnerships with families Administrators will have access to data to support decision-making while families will report having greater visibility and understanding of classroom happenings

In the intermediate term, students' value for learning activities in the classroom will increase and their confidence and self-efficacy as readers and content-based skills will improve Meanwhile, educators' ability to deliver data-driven and effective instruction will improve and their ability to differentiate instruction will become more effective Administrators will support the ongoing use of Seesaw through PD, curriculum, and school-wide communication and proactively leverage engagement and assessment data to address inequities Families will report having greater confidence and resources to support their students' learning progress Ultimately, in the long term, Seesaw expects students to report greater belonging to their community, have increased scores on standardized reading assessments, have a more successful elementary school experience, and be better prepared for future learning/middle school coursework. Educators' well-being and retention will improve and they will have sustained engagement with the Seesaw platform and other supporting edtech Educators will be better positioned to close gaps in student learning outcomes and administrators will be able to enact a high-quality, equitable curriculum across classes

Study Design for the Seesaw Early Literacy Supplemental Curriculum Program Evaluation

To continue building evidence of effectiveness and to examine the proposed relationships in the logic model, Seesaw has plans to conduct an evaluation to determine the extent to which its program produces the desired outcomes Speciﬁcally, Seesaw has plans to begin an ESSA Level II study to answer the following research questions:

Implementation Questions

1 To what extent did students use the Seesaw Early Literacy supplemental curriculum program during the 2023–24 school year?

○ What is the average number of Seesaw Early Literacy lessons that students completed on the Seesaw platform?

2. Among teachers, what were the usage patterns of Seesaw?

○ On average, how much feedback did students receive on Seesaw Early Literacy lessons?

Outcome Questions

3 After controlling for students’ prior literacy achievement levels and other student demographic characteristics (e g ELL status, FRL status, gender, and race/ethnicity), how was the use of the Seesaw Early Literacy supplemental curriculum program related to students’ performance on end-of-year standardized literacy assessments?

4. What was the impact of the Seesaw Early Literacy supplemental curriculum program on end-of-year standardized literacy assessments for students who used the program compared to those who did not?

Conclusions

This study satisﬁes ESSA evidence requirements for Level IV (Demonstrates a Rationale) Speciﬁcally, this study met the following criteria for Level IV: ✅ Detailed logic model informed by previous, high-quality research ✅ Study planning and design is currently underway for an ESSA Level I, II or III study