Data Collection and Analysis In order to embark on school improvement you need to know where your school is right now in the areas identified for school‐improvement. To do this you need to collect data – but which data? The most important decision in identifying the data to be collected is asking the right questions. Data has to be specific to a specific question and meaningful enough to enable you to identify the effective interventions you need to implement. SCHOOL SELF REVIEW‐ ASKING THE RIGHT QUESTIONS Good school self‐review can improve student outcomes and teacher performance. Teacher participation in the evaluation process leads to ownership of the process, increased understanding of the process and a culture of continuous improvement. Skilled facilitation can support the school review process. School review is: Strategic: long‐term and higher‐level Regular: on‐going data‐gathering and monitoring Emergent: spontaneous The aim is to establish evaluative and investigative questions and provide a guide to the collection of evidence. These questions focus schools on the data that is relevant and assist schools in the development of their own questions. DATA
INFORMATION
EVIDENCE
The first level of analysis is the gathering The second level of analysis takes The third level of analysis of raw data data and turns it into information uses this information as evidence to support Quantitative It describes “what is significant” judgment :how well/to what Qualitative This material extent” describes “what is happening” or “what is “If this is so, then we need to…” Outcomes Indicators Process Indicators Direct signs Factors which contribute to the result Specific measures of the impact of certain actions A guide to what might have caused a Student results can be directly linked to teaching and school particular outcome decisions Factors that contribute to student engagement, progress and achievement EVALUATION QUESTIONS Key questions to unpack the domain and/or the sub‐section of the domain Evaluative prompts A focus on more specific aspects
Sample Indicators Statements of measurable outcomes or observable behaviour
Possible sources of evidence Sources of data and ways of gathering this to assist in making judgments
The following example questions relate to three elements on the Matrix for school improvement. For the purpose of these examples, it is assumed that the collection of data is to facilitate improved school/student outcomes. Shared Values in Practice First level analysis What do we mean by shared values in practice? What are our current shared values? How do we know? How do our shared values contribute to improved student outcomes? How do we know? What evidence do we need to collect? How will we collect it? Who will collect it? Possible Information Sources Parent, staff, student surveys and focus groups; observation in class and school grounds; records of parental communication and complaints; student behaviour data Second level analysis What is the information we have collected telling us about the effectiveness of how we embed shared values in the school? What further data do we need to collect? What is the data telling us about how embedding shared values improves student outcomes? Are there differences between year levels and genders? Staff? Parents? Why is this so? Is there anything else significant the data is telling us? Third level analysis To what extent are shared practices known and understood in the school? What do we need to do at the school level to improve understanding of our shared values? How do we link shared values and student outcomes so the school community understands the link?
Intentional Teaching in Literacy First level analysis What does this element mean? What do we understand by Intentional Teaching? What are the demonstrated characteristics of intentional teaching? Why is intentional teaching important? How does intentional teaching impact on student learning in literacy? What models of Intentional Teaching are being used? How often? How do we know? How well are teachers currently performing in their use of intentional teaching to improve literacy? How do we know? What data could we collect to confirm this? How will we collect it? Who will collect it? What are we trying to achieve? Possible Information Sources Education research, classroom observations, staff discussion, student work samples, student in‐class test results, NAPLAN, teacher survey, student survey. Second level analysis What is significant about what we see in the data? What does an analysis of the data tell us about how well our teachers deliver intentional teaching in literacy? How well is effective intentional teaching improving student learning in literacy? Third level analysis To what extent is intentional teaching impacting on student learning? How do we know it is? What do we need to do to further improve?
Data collection for assessment First level analysis Why do we want to collect assessment data (what is our purpose)? What do we want it to tell us? What data do we currently collect? How does data collection for assessment impact on student learning? What data do we need to collect to facilitate our purpose? How will we know the data we collect will give us the information we need? How should we collect it? Who will collect it? Possible Information Sources Student performance on NAPLAN and national tests, in class assessment results, Years 11 and 12 moderation, outcomes, any other assessment data available at the school Second level analysis What is significant about what we see in the data? What connections can we make between the data we collect and student learning outcomes? What is missing? What proportion of students is making satisfactory progress? What do we consider is ‘satisfactory’ progress? Are there any trends over time? What variations exist between subjects? Are there significant differences in achievement in different cohorts; e.g. male and female, ESL, indigenous etc.? Are there any indications that particular year levels are performing at a higher or lower level? Are there any indicators with regard to the adequacy of current resources – physical, personnel? Third level analysis How will we use the information? – short term, longitudinal, whole school etc? What interventions do we need to implement? What Professional Development is required? What accountability measures will be applied?