The EDI-Informing community planning, evaluation and policy for ECD outcomes Sally Brinkman: Chief Scientific Officer – Int. EDI Consortium and National Australian EDI Epidemiologist. Eric Duku: Chief Statistician – Int. EDI Consortium and Chief Statistician – Canadian EDI
“A large number of children at a small risk for school failure may generate a much greater burden of suffering than a small number of children with a high risk�
(Based on Rose 1992, Offord et al. 1998)
What is the EDI? • The Early Development Instrument (EDI) is a community measure of young children’s development. • The EDI Checklist consists of around 100 questions completed by the child’s teacher based on their knowledge of the child
• It is designed for use with whole populations of children (based on geographical or administrative boundaries) • The checklist was originally designed in Canada to measure child development between 4 and 6 years of age.
What is the EDI • The EDI measures 5 developmental domains: – Physical health and well-being – Social competence – Emotional maturity – Language and cognitive development. – Communication skills and general knowledge. The EDI is a measure of how well the community has raised their children before school. It is an index of the outcomes of early child development and of children’s ability to take advantage of the learning environments offered to them by schooling
Why a community or population approach? • It is important to understand and think about the social and environmental influences on child development • Moving the focus of effort from the individual to the community can make a bigger difference • Allowing communities the opportunity to “shift the curve” of a whole population and therefore improve outcomes for many children
Number of Children
EDI Population curve Children developmentally Vulnerability cut-off vulnerable
Children performing well
Average score
Low
Outcome domain
High
Universal strategies-entire population moves
% of Children
Targeted strategies decrease numbers of vulnerable Gap narrows
Low
Outcome measure
High
Uses of the EDI •
To report on the early childhood development outcomes for whole of populations of children, small geographic areas and special populations
•
To measure progress over time in improving early childhood development outcomes
•
To facilitate community mobilisation around early childhood development through the AEDI implementation process and results
•
To enable schools and teachers to look back at the development of children before entering school and look forward to children’s needs once at school
•
To make state and territory, national and international comparisons
How the AEDI findings are used Local
• Stimulate discussions about early child development among teachers, parents, schools, community groups, and policymakers
Regional
• Identify communities and neighborhoods where children may be at risk developmentally • Plan and evaluate ECD initiatives
State National
• Document the effect, efficacy, and cost-effectiveness of ECD programs.
• Leverage better-informed ECD policies • Match programs with investment opportunities.
Case Studies: Australia Indonesia Canada Jamaica Mozambique Kosovo
Australia
Understanding Early Childhood Development in Australia • Before the AEDI we had no way of consistently understanding how children were developing across Australia as they came to the end of their pre-school years • 2002-2003 North Metro Perth EDI Study (2002 n=200, 2003 n=4500) • 2003 National meeting of experts to consider whether Australia needed an individual measure or a population measure of ECD. Resounding endorsement for a population measure • 2004-2007 Development and piloting of the AEDI, overwhelming success for communities (n=40,000) • 2007-2008 Agreement by COAG for the AEDI as a national progress measure of ECD for Australia • 2009
National AEDI Census in recognition of the need for all communities to have local ECD information to improve outcomes for children
National AEDI Results • 24% of Australian children surveyed were “developmentally vulnerable” on one or more domains of the AEDI 11.9% of Australian children surveyed were “developmentally vulnerable” on two or more domains of the AEDI 67.9% of Australian children surveyed were “performing well” on one or more domains of the AEDI
% Vulnerable on at least one area of ECD
Australia – AEDI Children 5-6 yrs.
30
20
10 Highest 10%
10% -25%
25% -50%
50% -75%
SEIFA disadvantage
75% -90%
Lowest 10%
“Janus” communities – exceptions to the rule (healthy deprived areas & unhealthy wealthy areas)
66th %ile
Social disparity increases over time As children move from year 3 to year 5, the disparity among those meeting literacy standards grows 100 Socioeconomic Status:
Proportion Meeting Standard (%)
90 80 The Wide ning Gap
70 High
60
Low
50 40 30 20 10 0 Year 3
Year 5
WALNA Data, DoE 2006
Map 15: Level of disadvantage according to the SEIFA index
Coolbellup Coolbellup Hamilton Hamilton Hill Hill
Bibra Bibra Lake Lake Spearwood Spearwood
South South Lake Lake
SEIFA Index of disadvantage 1. Most disadvantaged 2. 3. 4. 5. Least disadvantaged
Coogee Coogee
Yangebup Yangebup
Beeliar Beeliar Success Success
SEIFA is derived indicators such as low income and educational attainment, high unemployment, jobs in relatively unskilled occupations and variables that reflect disadvantage. The state quintile categories are referenced to state. City of Cockburn, WA
Atwell Atwell
Prepared by: AEDI National Support Centre Source: ABS Census 2001 Basic Community Profile
Proportion of children vulnerable on one or more domains
Coolbellup Coolbellup Hamilton Hamilton Hill Hill
Spearwood Spearwood
Bibra Bibra Lake Lake
South South Lake Lake
Proportion of children vulnerable
Coogee Coogee
N=Percent 49.1 40 33.3 18.9 11.5
Yangebup Yangebup
to to to to to
53.3 49.1 40 33.3 18.9
Beeliar Beeliar Success Success Atwell Atwell
City of Cockburn, WA
Prepared by: AEDI National Support Centre Source: AEDI Communities Data 2006
Participation in preschool in the year before entering school
Coolbellup Coolbellup Hamilton Hamilton Hill Hill
Bibra Bibra Lake Lake Spearwood Spearwood South South Lake Lake
Preschool participation N=Percent 38.6 48.1 64.3 83.9 90.9
Coogee Coogee
to to to to to
48.1 64.3 83.9 90.9 96.3
Yangebup Yangebup
Beeliar Beeliar Atwell Atwell Success Success
Proportion of children that attended preschool program including in a day care centre . City of Cockburn, WA
Prepared by: AEDI National Support Centre Source: AEDI Communities Data 2006
AEDI Summary Table Community report – summary table
Proportion of children developmentally vulnerable Suburb
No
Phys
Atwell
121
4.3
Beeliar
57
Bibra Lake
Soc
Average scores
Emo
Lang
Com
Vul 1
Vul 2
Phys
Lang
Com
12.1
9.5
9.5
9.5
20.7
10.3
9.09
8.96
8.65
8.72
8.75
16.4
29.1
20.0
26.0
29.1
49.1
32.7
8.50
8.13
7.12
7.02
6.25
55
1.9
3.8
3.8
11.3
1.9
18.9
3.8
9.55
8.96
8.60
7.69
8.75
Coogee
28
11.5
3.8
0.0
11.5
3.8
11.5
11.5
9.77
9.06
8.65
7.31
10.00
Coolbellup
17
26.7
20.0
13.3
20.0
26.7
53.3
20.0
7.73
8.96
8.46
7.31
6.25
Hamilton Hill
42
23.1
17.9
17.9
15.4
12.8
33.3
20.5
9.09
8.75
7.31
8.08
7.50
South Lake
50
8.3
8.3
14.6
27.1
12.5
35.4
20.8
9.09
9.17
8.46
7.12
8.44
Success
74
2.9
10.0
7.1
13.0
10.0
21.4
11.4
9.09
8.75
7.50
7.69
8.75
Yangebup
62
10.0
11.7
16.9
25.0
21.7
40.0
28.3
8.64
7.60
7.69
6.92
6.88
Abbreviations used: Phys Physical health and wellbeing Soc Social competence Emo Emotional maturity Lang Language and cognitive skills Com Communication and general knowledge Vuln1 Vulnerable on one or more domains Vuln 2 Vulnerable on two or more domains No Total number of children surveyed
Soc
Emo
Prepared by: AEDI National Support Centre Source: AEDI Communities Data 2006
Mobilizing community action for children
The AEDI community planning process 1. Identifying areas of particular need
2. Assessing the local distribution of children’s developmental vulnerability
e.g. Mission Australia funds 3 year play group, language program & mums group at school
3. Community asset mapping
4. Mobilising community action
Governance – National AEDI Census COAG
National AEDI Partnership (CCCH/TICHR) Ultimate responsibility for the AEDI Program
DEEWR
McMaster Int. EDI Consortium
National AEDI Support Centre (NSC) Mapping, analytical, database (ACER), overall project management and support to states State AEDI Coordinators Key tasks: Provide state wide leadership, engage key stakeholders and support communities, schools and teachers to successfully implement the AEDI Resources: State AEDI Coordinator Training and ongoing support from AEDI NSC
Local AEDI Coordinators Key tasks: Raise awareness of the AEDI in the community Resources: Community Preparation & Implementation Guide Training for communities Community Strategies Toolkit
Schools and teachers Key tasks Teachers complete AEDI Training (1 hour) and AEDI Checklist Principals invoice for Teacher Relief Resources: AEDI Guide for Teachers AEDI Teacher Training AEDI Guide for Principals
Indonesia
ECED Project Challenges • 12.7 Million (US$) – 2013 (loan) • Beneficiaries: 738,000 children ages 0 to 6 in 6,000 poor communities (dusuns) in 3,000 villages within 50 districts • an average of 123 children per dusun
Study Objectives •
• • •
•
To what extent the ECED project improves children's development, attendance and readiness for school To what extent the ECED project improves parents awareness and practices To what extent the ECED project increases the availability and utilisation of ECED services How do these impacts differ by gender, wealth, and level of service delivery at baseline What factors contribute to success and failures of the ECED intervention
Challenges • Community driven project • Research design Three components: – Increasing the delivery of ECED services in targeted poor communities using a Community Development district based approach – Developing a sustainable system for ECED quality – Establishing effective program management, monitoring, and evaluation
• Village characteristics • Household survey:
Data to be collected
– Household economy – Parent background – Parent knowledge, attitudes, beliefs child rearing
• Child outcome data : indicators of child development across domains
Instruments • Philippines ECD study (developmental milestones, anthropometrics, nutrition) • IFLS (Rand) • SDQ • EDI
Canada
“EDI data in conjunction with other data can be used to create, maintain, and monitor community support for programs and policies affecting young children…. Analysis can increase public understanding of the factors which contribute to early child development, inspiring a commitment to fundraising, policy development and other initiatives.” Source: Canadian Association of Principals – Student Readiness to Learn and the School Ready to Teach: an Internet Essay and Collection of Selected Documents: www.schoolfile.com/cap_start/schoolready.html (2003 )
Canadian Normative Data Collected from 2004/5 to 2006/7 (n=176,621) • Percent of children scoring low on 1 more domains = 27% • Percent of children scoring low on 2 or more domains = 13.6%
Factors increasing the vulnerability risk Odds ratios • • • • • • •
Child health (low) Gender (boy) Income (low) Family status (not intact) Age (younger half) Literacy (looking at books) Parent smoking Source: Janus & Duku 2007
2.35 2.32 2.02 1.83 1.36 1.35 1.29
Important neighbourhood variables • Parent-reported neighbourhood quality (includes playground, safety, health, transport, presence of families with children etc.) • Parent-reported frequency of contacts with neighbours (talking, visiting)
Source: NLSCY/UEY/EDI 1998/9
Canada: % vulnerable by SES 60
50
%
40
31.9
29.1 23.1
30
13.7
20
10
0 very poor
poor
Source: NLSCY/UEY 1999-2000; EDI 1999-2000
not poor
well-off
Toronto
Survey of response to EDI results in Ontario and Manitoba • 68.4% report changes have been made • Areas of most frequent change: Areas of least frequent change: Parenting Programs non-profit recreation behaviour programs in schools programs child and family centres ESL programs for children library programs speech/language programs • Educational programs most likely to be implemented • Low school readiness associated with higher percentage of Child & Family Centers, and higher percentage of programs aimed at Aboriginal children with special needs
Kosovo
Kosovo • In collaboration with World Bank • ECD Programme Evaluation Objectives • Assessing using the EDI, whether children who attended ECD centres were more advantaged developmentally over those who did not attend ECD centres
• To gather information on the households of children in the main survey for the assessment of correlates of EDI domains and outcomes
Kosovo Data Collection • 1st wave – Children from 9 municipalities – Teachers filled out forms – Test-retest on smaller sample of 51 ECD children from 2 communities
• 2nd wave – 111 children selected at random using proportional allocation to ensure representativeness from all 11 communities
Kosovo Instruments used • EDI (teacher filled): short version developed in collaboration with the World Bank; translated into Albanian • Teacher demographics • Household Form (2nd wave) • Parent interview
Kosovo – sample descriptives group
Total (n=460)
Variables
categories
ECD (n=219)
non-ECD (n=241)
Gender
Females
42.9%
50.2%
46.7%
Attend pre-school
Yes
3.7%
0.0%
1.7%
Class child attends
One
95%
100%
97.6%
Age
based on EDI
6.71 (0.59)
6.96 (0.60)
6.84 (0.61)*
Kosovo: % vulnerable by SES 60
56.3 43.2
50
40 30.8
%
40 30 20 10 0
Cat 1
Cat 2
Cat 3
Cat 4
Jamaica
Jamaica • In collaboration with World Bank • Part of process for establishing a national monitoring system of children from pre-school through Grade 3 • Over 2,000 schools in Jamaica • School selection using stratified randomization, based on the government’s determinations of rural/urban and small/medium/large. • All 4 year olds attending the selected schools were included. • Parental consent eventually determined who participated in the study.
Jamaica Instruments
• Teacher assessments: the EDI (full 2003/4 version) • Parent assessment: Vineland Social-Emotional Early Childhood Scales • Direct assessments : McCarthy Scales of Children’s abilities and PPVT • SES – demographic and household information from parental interview
Jamaica • • • • •
Sample description Sample size, n = 151 % females = 49 # schools = 18 Mean age at completion 4.8 (0.4) Good internal reliability and concurrent validity
JAMAICA - validity
PPVT McCarthy-verbal -perceptual -quantitative -cognitive memory -cognitive general
Language & Communication Cognitive Dev & Gen Knowledge .396 .331 .361 .401 .443
.318 .346 .308
.371 .448
.312 .383
Jamaica % vulnerable by SES 60
50
47.4 34.2
%
40
30
21.1
18.9
20
10
0 Q1
Q2
Q3
Q4
Mozambique
Mozambique • • • • • •
In collaboration with World Bank Pre-pilot Full version of EDI adapted for use Translation into Portuguese Test retest Items removed because…
Mozambique – sample descriptives • • • • • •
Sample size, n = 78 Mean age = 6.3 years % females = 55.1 # schools = 4 Teachers Good test-retest reliability
Early Development Instrument (EDI) Versão em Português para Moçambique Portuguese version for Mozambique Numero de identificação da escola-Número do aluno Copie o número do aluno segundo a marcação após a questão ST20 no questionário para o professor P[__][__] D[__][__] PA[__][__] L[__][__]APC[__][__] UR[__]AE[__][__][__]NAF[__][__][__] ESCOLA [__][__] ALUNO [__][__] Nome da escola: _____________________________________________ Nome do professor: __________________________________________ Nome da criança: ____________________________________________ Data de nascimento da criança: Dia _____/ Mes______/Ano Sexo da criança: M F Data em que completou o questionário : Dia _____/ Mes______/Ano A criança possui necessidades educativas especiais ? Sim Não Se sim, por favor explique: _________________________________________________________________ _________________________________________________________________ ______________ Língua Materna da criança Changane Chope Ronga Outra? ______________________ A criança se comunica bem em sua língua materna? Sim Não Não sabe
Reliability and Validity
Reliability and validation studies • Teacher to parent inter rater reliability • Teacher to teacher aid inter rater reliability • Repeat testing intra rater reliability • Construct and concurrent validity • Predictive validity • Rasch psychometric property analyses • Indigenous and minority culture validation studies Publications downloadable from: www.australianedi.org.au and www.offordcentre.com
Adaptation Process
Adaptation • Ethnographic scan (age of schooling etc etc)
• Content validity • Translation / back translation • Translation of the teacher guide • Adaptation to local language, context, deletion/addition of items (cultural relevance, behaviour constructs) • Pilot implementation (inter-rater reliability, test-retest, validity assessment with other instrumentation/assessment) • License agreement with McMaster Uni, Offord Centre.
EDI vs. Traditional Evaluation tools
EDI • Impacts at multiple levels of the ecological model • Can be used as an outcome and predictor • The data is not to be owned by one organisation and as information is aggregated it can be freely shared amongst varying sectors
• When the data is mapped it provides an easily understandable dissemination strategy that facilitates community engagement and thus mobilisation • Communities use the results together with other information such as national and local statistics, community, family, and service provider feedback to develop strategic plans.
EDI
• The data prompts service providers to review existing services and programs and to consider new initiatives that might be needed. • Prompts universal service provision with the aim to better all children while also reducing future inequality.
• Provides a means to evaluate community development initiatives. • As more countries utilise the EDI, in the future we will be able to make international, national, jurisdictional and local level comparisons • Quick, cheap • Can be completed by teachers, child care workers or parents - as not a diagnostic does not require referral to allied/medical professionals
• Eric Duku – duku@mcmaster.ca • Sally Brinkman – sallyb@ichr.uwa.edu.au
• www.offordcentre.com/readiness • www.australianedi.org.au • http://www.councilecd.ca/internationaledi/