Action Research by Lauren Teather

Understanding by Design and

Action Research 1. Identify an issue or question.

8. Plan next steps â&#x20AC;&#x201C; actions and/or further research.

2. Form an hypothesis.

7. Summarize the findings. Report the conclusions.

3. Check existing research on this issue/ question.

6. Analyze the data. Look for patterns or trends.

4. Identify multiple data sources.

5. Collect, organize presented by and represent the data.

Jay McTighe Educational Consultant 6581 River Run Columbia, MD 21044-6066 (410) 531-1610 e-mail: jmctigh@aol.com

Understanding by Design

Stage 1 Introduction

Stage 2

Stage 3 Resources

UbD and Action Research: Something To Think About Directions: Select one of the following statements with which you agree. Explain why you agree, and, if possible, give an example to illustrate the idea.

thoughts... 1. “Why do we do research lessons? I don’t think there are any laws [requiring it]. But if we didn’t do research lessons we wouldn’t be teachers.” Japanese Elementary School Teacher

2. “Teachers are trained in schools of education as consumers of this educational research, not as future producers of knowledge, and are taught that the experts in K-12 teaching are not K-12 teachers.” Miles Myers 3. “Although they may work in the same building with other teachers and use the same materials and follow the same schedule, teachers rarely turn to one another during the school day except during their 30-minute lunch period, where informal norms often forbid any kind of professional talk.” Richard Sagor 4. “There’s good news and bad news about action research. The good news-I love trying to find out what’s working. The bad news - the more questions I ask, the more questions I have .”

Teacher - Baltimore, Maryland

5. “Professional satisfaction for members of a dynamic organization requires collective acceptance of and responsibility for the continuous cycle of measuring progress and taking action. Such progress builds individual and organizational efficacy: We can change things.” Emily Calhoun 6. “It is irresponsible for a school to mobilize, initiate, and act without any conscious way of determining whether such expenditure of time and energy is having a desirable effect.

Carl Glickman

______________________ ______________________ ______________________ ______________________ ______________________ ______________________ ______________________ ______________________ ______________________ ______________________ ______________________ ______________________ ______________________ ______________________ ______________________ ______________________ ______________________ ______________________ ______________________ ______________________ ______________________ ______________________ page 2

Understanding by Design

One-page Template

INTASC Standards – Principle 8: understands and uses formal and informal assessment strategies Principle 9: reflects on teaching

Standard(s)

Desired Results Understandings

s t a g e 1

• Student achievement is enhanced when teachers act as reflective researchers – regularly collecting evidence of learning, analyzing the results and adjusting their practice accordingly. • Understanding by Design offers a framework for collecting evidence of learning and adjusting practice. • Prior knowledge can enhance or inhibit new learning.

Essential Questions

• What counts as sound evidence of understanding, and what evidence is misleading? • What does it mean to “think like an assessor”? • What misunderstandings may exist and how we uncover them?

d s t a g e 2

Knowledge & Skill

• 3 stages of backward design • action research process • formal and informal data collection/ assessment strategies • techniques for quantifying, representing, and interpreting data • UbD design standards

Assessment Evidence

e-g

1. Design and implement an action research project for your classroom or school. Your research should be guided by an appropriate UbDrelated question, use a variety of formal and informal assessment strategies to collect data, analyze the data, and develop an action plan based on your findings. 2. Maintain a reflection journal throughout the process to document

Learning Activities s t a g e

-Review key ideas of Understanding by Design (UbD) - backward design, design template, and design standards -Reflect on link between UbD and action research -Presentation on action research process - question formulation, hypothesis generation, data collection/analysis, reporting results -Analyze action research examples - key features and design flaws -Apply the action research process for your own situation

page 3

data collection, quantification and interpretation

Stage 1 Introduction

• includes both quantitative and qualitative data • gathers data from multiple sources (uses triangulation) • searches for context-specific patterns and anomalies (not necessarily statistically significant) to inform actions • provides data for new questions

• relies on quantitative methods (“let the numbers speak for themselves”) • accepts only ”verifiable” evidence/no anecdotes • seeks statistically significant results to enable generalizations • provides data for new questions

Action Research

research design/ methodology

Traditional Scientific Research

• focuses on specific questions • typically focuses on questions or or issues for a particular issues related to general populations or settings to enable generaliz- population/setting to inform classroom and school actions ability • follows a structured procedures • follows strict scientific procedures (e.g., question/hypothesis), but (e.g., use of null hypothesis and isoseldom uses control groups lation of key variables) • emphasizes validity and • seeks validity, reliability, and reliability (not generalizability) generalizability

Research Process

Comparing Traditional Research and Action Research

Understanding by Design Stage 2 Stage 3 Resources

page 4

Understanding by Design

Stage 1 Introductions

Stage 2

Stage 3 Resources

Action Research Process 1. Reflect on the students you have in your classroom this year and in past years. List issues, problems, concerns related to teaching/learning. 2. Select one issue that is most interesting/puzzling to you. [Note: It is often difficult to select a single issue. Working with a teaching colleague, a coaching partner, or critical friend can be beneficial throughout the action research process.] 3. Once you have selected the issue, generate a research question. For example Issue: It is difficult to get 8th graders to understand how they are influenced by what they see and hear. Question: How can I develop a sequence of learning activities using a variety of text that will result in a deep understanding of persuasive text among my 8th grade students? 4. Given your question, form an hypothesis. For example Hypothesis: Using a variety of text and the strategies of think alouds and cooperative groups, students will develop a deep understanding of how authors use persuasive language to influence readersâ&#x20AC;&#x2122; beliefs and actions. 5. Given the hypothesis, detemine what data you will collect. Remember that selecting data from a variety of sources (triangulation) will enable more valid conclusions. It is helpful to create a Data Chart specifying: a) types of data, b) how/when collected, c) reason for selection 6. For each type of data you collect, consider how you will quantify the data. Being able to quantify the data will help you interpret it. 7. Collect, organize and represent the data. 8. Analyze the data. Look for patterns. Look for anomalies. Interpret the data: What does it mean? 9. Summarize your findings. What do you know now? What do the results tell you? 10. Given what you now know, what actions will you take? What new research might you do? ÂŠ2000 Grant Wiggins and Jay McTighe

page 5

Understanding by Design

Stage 1 Introduction

Stage 2

Stage 3 Resources

Action Research Process 1. Identify an issue or question.

8. Plan next steps â&#x20AC;&#x201C; actions and/or further research.

2. Form an hypothesis.

7. Summarize the findings. Report the conclusions.

3. Check existing research on this issue/ question.

6. Analyze the data. Look for patterns or trends.

4. Identify multiple data sources.

5. Collect, organize and represent the data.

Note: Effective teachers are natural researchers, constantly questioning the why and how of teaching and learning. Action research offers a more formal means by which we can investigate our craft. While there is a general logic to action research, the process is not rigidly sequential. Rather, action research is recursive, similar to the writing process. For example, teachers will frequently return to refine the initial research question as they begin to collect data.

ÂŠ2000 Jay McTighe and Marcy Emberger

page 6

Understanding by Design

Introductions Stage 1

Stage 2

Stage 3Resources

Generating Possible Research Questions Use the following worksheet to generate possible action research questions. Then, check your questions against the four criteria at the bottom of this page.

How can textbooks be used most effectively in teaching for understanding?

How can I better engage my students while still using the textbook?

topic, problem, issue:

My curriculum requires the use of a textbook. My students are rarely engaged by the textbook, and some can’t even read it. Also, the text emphasizes ‘coverage’, not depth.

What strategies can I use to move beyond the textbook for ‘uncovering’ important content ideas?

Is your proposed research question: o significant – at the “heart” of teaching and learning? o feasible – within your control to research? o transferable – applicable in different contexts? o compelling – you really care about finding the answer? ©2000 Grant Wiggins and Jay McTighe

page 7

Understanding by Design

Introductions Stage 1

Stage 2

Stage 3Resources

Generating Possible Research Questions Use the following worksheet to generate possible action research questions. Then, check your questions against the four criteria at the bottom of this page.

topic, problem, issue:

page 8

Understanding by Design

Introductions Stage 1

Stage 2

Stage 3Resources

Generating and Sharpening Research Questions Use one or more of the following questions in the filter to sharpen and refine your action research question. Then, check your question against the criteria at the bottom of this page. inital question

• What’s puzzling? • What’s frustrating? • Why is this a problem? • So what? • Why does it matter? • If the issue/problem is a story, what’s the “moral”? • What do you think causes ________? • What does research say about _________? • How might we define the problem/issue in different ways?

• How will better understanding help?

Refined research question:

Is your proposed research question: o specific – A question at the “heart” of teaching/learning? o measurable – A question that can be researched? o transferable – A question applicable in different contexts? o compelling – You really care about finding the answer? ©2000 Grant Wiggins and Jay McTighe

page 9

Understanding by Design

Stage 1 Introductions

Stage 2

Stage 3 Resources

Collecting Sufficient Data Effective data collection requires multiple sources of data – a photo album, not a single snapshot. Use the following worksheet to brainstorm possible sources of data related to your research question.

▼

Data source #2

▼

Data source #1

▼

Topic, issues, problem: _________________________________________________

Research Question:

Data source #4

▼

▼ ▼

▼

Data source #3

▼

Hypothesis:

page 10

Understanding by Design

Stage 1 Introductions

Stage 2

Stage 3 Resources

Linking Research Questions, Hypotheses and and Ideas for Data Collection Research Question: How can I predict likely student misunderstandings in an upcoming unit? How might I avoid/overcome the identified problem areas? Hypothesis: There are predictable student misunderstandings inherent in many topics. On-going assessments (diagnostic and formative) are needed to help identify these problem areas and to inform the instruction needed to address them. Data Collection Ideas: a. Pre-test – Develop a diagnostic assessment (pre test) to check for prior misconceptions about the topic. b. On-going Assessments – Develop formative (on-going) assessments to check for understanding “along the way” during instruction. c. Summative Assessment – Develop a culminating performance assessment to check for understanding. Re-administer the pre test (as a post test). d. Journal – Keep a log of observations following instruction that has been planned to overcome misconceptions. Target your observations to those students who have been identified (through formative assessments) as having the greatest difficulty. e. Other: ____________________________________________ ____________________________________________________ f. Other: ____________________________________________ ____________________________________________________ ©2000 Grant Wiggins and Jay McTighe

page 11

Understanding by Design

Stage 1 Introductions

Stage 2

Stage 3 Resources

Linking Research Questions, Hypotheses and and Ideas for Data Collection Research Question: To what degree are the WHERE instructional principles reflected in my teaching and what are the effects on my students? Hypothesis: When the WHERE instructional principles are regularly and effectively applied, students will be more engaged, more mindful of the major learning goals and how their work will be evaluated, and better able to evaluate their own work. Data Collection Ideas: a. Structured Observations – Shadow a student for a day. Develop an observation list of observable indicators based on WHERE. b. Student Interviews – Randomly interview students, asking the following questions: W Why are you doing this?, How will learning this help you?, How is what you are doing today related to yesterday? ...to tomorrow?. How will you show you’ve learned it?, how will your work be evaluated? H Is this work interesting? engaging? boring? Did any activities “hook” you into considering the big ideas? E Do you have opportunities to explore the ideas and their value? Are you prepared (equipped) for the culminating performance task(s)? R Have any activities cause you to rethink and revise? E Are you asked to self evaluate your work? Is there an opportunity to reflect on (make meaning of) your work and the unit? c. Journal – Keep a journal of reactions to the results of various teaching activities. Ask 1-2 colleagues to do the same. d. Other: ____________________________________________ ©2000 Grant Wiggins and Jay McTighe

page 12

Understanding by Design

Stage 1 Introductions

Stage 2

Stage 3 Resources

Linking Research Questions, Hypotheses and and Ideas for Data Collection Research Question:

Hypothesis:

Data Collection Ideas:

❏

❏ ❏

❏

page 13

Understanding by Design

Introductions Stage 1

Stage 2

Stage 3Resources

Types of Data

How Collected?

When?

Reason for selection

Data Collection Plan

ÂŠ2000 Grant Wiggins and Jay McTighe

page 14

Understanding by Design

Stage 1 Introductions

Stage 2

Stage 3 Resources

Questions for Analyzing and Interpreting Data Use one or more of the following questions to analyze and make meaning of the available data.

❍ What does the data show?

❍ What anomolies exist?

❍ What patterns are evident? ❍ Are there different possible explanations for these results?

❍ Are these the results I/we expected?

data ❍ Is the data from different sources consistent? ❍ Are any data from different sources in conflict? What accounts for the differences? ❍ What action(s) are needed? ❍ What new research questions does this data suggest? ❍ other: ____________________________________________? ©2000 Grant Wiggins and Jay McTighe

page 15

Understanding by Design

Stage 1 Introductions

Stage 2

Stage 3 Resources

Action Research Design Template (directions) Topic, issues, problem: _________________________________________________ Research Question: • What issues, problems, questions, or concerns do you have related to learning? Select one issue, problem, or concern to use in formulating an action research question. Data Collection Methods:

Hypothesis: • What do you hope/predict you will find? Generate an hypothesis.

Quantification Method(s):

• What data is needed? • How will I collect valid, reliable, and sufficient data?

• How will you quantify the data in order to effectively represent it?

Identify a variety of formal and informal assessments to gather needed data.

Identify the scales, rubrics, observable indicators, etc. that you will use to quantify the data.

Data Analysis – data representation(s) and pattern search, • How are the data best represented? • What does the data show? • What patterns exist? Select appropriate methods for representing the data to help you and others interpret and understand it. Identify any patterns. Research Results • What does the data mean? • What did you learn? • What were your conclusions, insights, surprises, disappointments? • What new understandings do you have? Summarize the results. Identify the insights you have gained from your research. ©2000 Grant Wiggins and Jay McTighe

page 16

Understanding by Design

Stage 1 Introductions

Stage 2

Stage 3 Resources

Action Research Design Template Peer Tutoring - 5th Grade Mathematics Topic, issues, problem: _________________________________________________ Research Question: Can peer tutoring help 5th grade students improve achievement in mathematics, including solving multi-step problems?

Data Collection Methods: 1. Pre/Post: Harcourt Brace mathematics inventory 2. Pre/Post: Teacher-designed attitude survey on peer tutoring) 3. Journal: Teacher/students complete logs following peer tutoring sessions 4. Weekly mathematics test scores

Hypothesis: Students will demonstrate improvement in problem solving and in their interpersonal skills through a consistent use of peer tutoring in mathematics instruction. Quantification Method(s): 1. Pre/post statistical comparison on Harcourt Brace inventory 2. Pre-post comparison of responses on peer tutoring survey 3. Rubric for weekly logs rated on positive/negative/neutral attitudes

4. Plot scores on weekly mathtests.

Data Analysis – data representation(s) and pattern search, • Pre and Post graphs representing individual Harcourt Brace scores • Pre and Post graphs illustrating responses on peer tutoring survey • Tally of weekly log entries (% of positive, negative, neutral responses) • Graph of weekly individual math scores and whole class scores Research Results All students participating in the peer tutoring project improved on the Harcourt Brace inventory (average gain 25.6%). This gain correlated with growth in weekly mathematics quizzes. Students showed a gain in positive attitudes toward peer tutoring. I plan to expand my use of peer tutoring to include spelling and vocabulary. ©2000 Grant Wiggins and Jay McTighe

page 17

Understanding by Design

Stage 1 Introductions

Stage 2

Stage 3 Resources

Action Research Design Template Calculators in Primary Classrooms Topic, issues, problem: _________________________________________________ Research Question: Can the use of hand calculators increase primary students competency in basic math facts?

Data Collection Methods: 1. Baseline kindergarten and composite records & pre/post test on math facts 2. Teacher/students logs following calculator lessons 3. Individual interviews with selected students & bi-monthly quizzes. 4. Observation checklists used during lessons

Hypothesis: Given instruction and repeated practice, hand calculators will increase primary students competency in basic math facts.

Quantification Method(s): 1. Comparison of pre/post data on individual students 2. Rubrics to examine student/ teacher logs for indication of calculator impact 3. Tallies showing responses to interviews and quizzes 4. Rubric to evaluate observations

Data Analysis – data representation(s) and pattern search, • Correlation of pre/post data on math facts • Graphs of results of rubrics from teacher/student log entries • Graphs of interview results and quizzes over semester • Items on checklist graphed to show patterns Research Results No correlation was found between initial baseline data and final assessment scores on math facts. Results of the interviews showed that while students could explian the concepts, that understanding seemed to have little impact on mastery of those facts. Observation checklists showed that calculators appeared to be a distraction to learning. Other manipulatives may be more beneficial at the primary level. ©2000 Grant Wiggins and Jay McTighe

page 18

Understanding by Design

Stage 1 Introductions

Stage 2

Stage 3 Resources

Action Research Design Template Note: This is a flawed example.

Use of Textbooks and Teaching for Understanding Topic, issues, problem: _________________________________________________ Research Question: How can textbooks best be used in teaching for understanding? What strategies and resources can I use to move beyond the textbook for greater ‘uncovering’ important content ideas and engage my students? Data Collection Methods: 1. Parent survey on their opinions related to quality and use of textbook. 2. Student survey on their opinions related to quality and use of textbook. 3. Faculty survey on their opinions related to quality and use of textbook. 4. Student interviews on their use of textbooks and their opinions about the value of textbooks

Hypothesis: Given the use of the textbooks and additional classroom strategies, students will be able to develop a deeper understanding of the content as well as becoming more engaged in their learning. Quantification Method(s): 1. Tallies of each of the three surveys 2. Tallies of similarities and differences among responses on the three surveys. 3. Tallies of responses of student interviews and comparisons with surveys.

Data Analysis – data representation(s) and pattern search, • Graphs showing total responses for each of the three surveys • Graph showing similarities and differences among three surveys • Graph showing student interview responses • Graph showing similarities and differences between student surveys and interviews Research Results The survey data showed some similarities among student and teacher opinions concerning the quality of the textbook being used. Parents overall showed a higher regard for textbooks than either students or teachers. No conclusions could be made about the value of the textbooks in relation to developing understanding.

page 19

Understanding by Design

Stage 1 Introductions

Stage 2

Stage 3 Resources

Action Research Design Template Topic, issues, problem: _________________________________________________ Research Question:

Hypothesis:

Data Collection Methods:

Quantification Method(s):

Data Analysis – data representation(s) and pattern search,

Research Results

page 20

Understanding by Design

Stage 1 Introductions

Stage 2

Stage 3 Resources

Assessment Task Blueprint UbD-related action research What understandings/content standards will be assessed through this task?

• Student achievement is enhanced when teachers act as reflective researchers – regularly collecting evidence of learning, analyzing the results and adjusting their practice accordingly. • INTASC Standards – Principle 8: understands and uses formal and

informal assessment strategies Principle 9: reflects on teaching

Through what authentic performance task will particpants demonstrate understanding?

1. Design and implement an action research project for your classroom or school. Your research should be guided by an appropriate UbDrelated question and use a variety of formal and informal assessment strategies to collect data. Analyze the data and develop an action plan based on your findings. 2. Maintain a reflection journal throughout the process to document

What products/performances will provide evidence of desired understandings?

1. action research plan including research question(s), data collection methods, data analysis, and action plan 2. reflection log By what criteria/indicators will task-derived student products/performances be evaluated? (Consider criteria which refer to desired task content, process, and results)

• appropriate UbD-related research question(s) • data/evidence of understanding collected using a variety of appropriate assessment methods • thorough analysis of data • action plan reflects insights from research findings

page 21

Understanding by Design

Stage 1 Introductions

Stage 2

Stage 3 Resources

Ideas for Action Research Shadow a student for a day. Pick a student at random and follow that student for a day. As you “walk in their shoes” consider questions such as, Is their schoolwork engaging? …boring? Do the learners see purpose in what they are learning? Are they exploring big ideas? What are your impressions of their school experiences? Take notes and report on your experiences at the next faculty meeting. A Place Called School – reprise. Repeat the classic John Goodlad survey as to which courses students see as most engaging (and why), as most worthwhile (and why); as most and least challenging, etc. Share your findings with the rest of the faculty. Questioning Strategies Monitor the use of classroom questioning. What percentage of my questions require factual recall?, …application?, …synthesis or evaluation? What are the results of asking different types of questions? What happens when I use various follow-up strategies; e.g., “wait time”?, probes?, play devil’s advocate? Visit other teachers’ classrooms and take note of their questioning strategies. Then, share your findings. Survey Graduates Contact recent high school graduates who are now at institutions of higher education and employed. Ask them to describe the extent to which their K-12 schooling prepared them for future study and the world of work. In what ways were they well prepared? In what ways might their schools have prepared them better? Present and discuss survey results with teachers and administrators. Do students understand the goals and priorities? What will students say if you ask them “why are you doing what you are doing?” “How does yesterday’s lesson relate to today’s?” What do you predict we will be doing tomorrow?” What is your long-term goal for this unit? “How will your learning be judged?” Compare your findings with other teachers and discuss the implications of the finding. Reverse normal sequence. Using two classes (one as a control), alter the normal sequence for a unit. In one class, immerse students in examining essential questions around provocative issues or problems, and only teach the “basics” on an as needed basis. In the control class, “cover” the basics using a textbook. How do the two classes compare in terms of student engagement and interest in the topic? Use the same assessment for both groups. What are the results? Grading and reporting inquiry. Survey students and parents regarding the current grading and reporting system. To what extent do they think that grades and reports are understandable? … consistent among teachers? … fair? … accurately communicate student performance, progress and work habits? Compile and report on your findings and discuss the implications for current practice. ©2000 Grant Wiggins and Jay McTighe

page 22