Hands-on workshop: Creating rubrics for evaluating speaking and writing in ESL Daniela Bramwell IIDEA – Instituto de Investigación, Desarrollo y Excelencia Educativa Quito, 13 de septiembre de 2009
Today’s program • Context: reviewing the three basic steps in planning (objectives; evaluation; activities) • Why evaluate? • Subjectivity in evaluation • Rubrics: An instrument for making qualitative evaluation more objective
Context: reviewing the three basic steps in planning (objectives; evaluation; activities) (Backward Design model) 1. OBJETIVES Identifying the desired results
2. EVALUATION Determining the acceptable evidence
3. ACTIVITIES Planning learning experiences
Source: Wiggins & McTighe, 2005 cited in Tokuhama-Espinosa, 2008
Evaluation • How will we know if and to what degree students have reached the desired results? • What will we accept as evidence that they have learned what was planned?
Source: Wiggins & McTighe, 2005 cited in Tokuhama-Espinosa, 2008
Why evaluate? • Two principal types of functions: ▫ Social ▫ Pedagogical
• ACTIVITY: ▫ Brainstorming
Source: Barriga & Hernández, 2002
Subjectivity in evaluation • Is evaluation subjective? • Is that a problem? Why is or isn’t it? • What forms of evaluation are more subjective? Which are less so?
Research: subjectivity in evaluation • In 1912, Starch and Elliot conducted a research project to determine whether grades were a credible indicator of student learning. • Two student essays were sent to 200 high schools in USA in order to receive grade. One essay received grades ranging from 64 to 98 over 100. The other, grades from 50 to 97 over 100. • Guskey (1996) showed that this type of problem still exists. Source: Starch & Elliot, 1912 cited in Guskey, 1996; Guskey, 1996
An instrument for making qualitative evaluation more objective • A qualitative and quantitative evaluation instrument • A way of establishing clear expectations (and raising the bar) • A way to reduce subjectivity
Rubrics
• ACTIVITY: ▫ ▫ ▫ ▫
Who can remember the 3 steps in planning? Objective: learn to make the best bubble gum balloon In groups of 3, determine who made the best bubble How did you decide who had the best bubble? What criteria did you use? What was the acceptable evidence that the person had made the best bubble?
Types of rubrics • Analytic or holistic ▫ Analytic – Lists levels of performance for each criterion. It is possible to show the student the specific areas that need improvement ▫ Holistic – Gives general feedback (does not list levels of performance of each criterion)
• General or specific ▫ General – can be used to evaluate all similar activities (e.g. one rubric for every class debate) ▫ Specific – is used to evaluate one particular activity (e.g. one class debate on one specific topic) Source: Wiggins & McTighe, 2005 cited in Tokuhama-Espinosa, 2008; Mueller, 2008
Example of an analytic rubric CRITERIA
1
3
5
Number of Sources
1-4
5-9
10-12
Historical Accuracy
Lots of historical inaccuracies
Few inaccuracies
No apparent inaccuracies
Organization
Cannot tell from which Can tell with difficulty Can easily tell which source information where information came sources info was drawn came from from
Bibliography
Bibliography contains very little information
Source: Mueller, 2008
Bibliography contains most relevant information
All relevant information is included
Example of a holistic rubric 3 - Excellent Researcher •
included 10-12 sources
•
no apparent historical inaccuracies
•
can easily tell which sources information was drawn from
•
all relevant information is included
2 - Good Researcher •
included 5-9 sources
•
few historical inaccuracies
•
can tell with difficulty where information came from
•
bibliography contains most relevant information
1 - Poor Researcher •
included 1-4 sources
•
lots of historical inaccuracies
•
cannot tell from which source information came
•
bibliography contains very little information
Source: Mueller, 2008
Rubrics
• ACTIVITY:
▫ Groups of 3: Find two people who teach a similar level as you do. ▫ Decide whether you want to develop an analytic rubric for evaluating speaking or writing. ▫ Create the rubric (you can use the provided format) ▫ Questions?
For more information Daniela Bramwell IIDEA – Instituto de Investigación, Desarrollo y Excelencia Educativa, Universidad San Francisco de Quito Edif. Galileo, Oficina #101 Telf.: 297-1700 ext. 1338 dbramwell@usfq.edu.ec www.educacionparatodos.com
References • Barriga, F y Hernández, G. (2002). Estrategias docentes para un aprendizaje significativo. Madrid: McGraw-Hill. • Covey, S. (2004). The Seven Habits of Highly Effective People. New York: Free Press • Guskey, T. (1996). Communicating student learning. ASCD Yearbook. • Mueller, J. (2008). Authentic assessment toolbox. Retrieved on September 1st, 2009 from http://jonathan.mueller.faculty.noctrl.edu/toolbox/rubrics.htm • Tokuhama-Espinosa (2008). Formas de evaluación para un mejor aprendizaje: El uso de rubricas. Retrieved on August 23rd, 2009 from http://www.educacionparatodos.com/pdf/Evaluation, %20Valoracion%20y%20Retroalimentacion.pdf • Wiggins, G, y McTighe, J. (1998/2005). Understanding by design. Alexandria, VA: Association for Supervision and Curriculum Development.