EVALUATING KA ORA, KA AKO’S IMPACT ON KIWI KIDS

In Term 1 2020 an interim evaluation of the Healthy School Lunch programme was commissioned to assess the early impact of the pilot programme based on the priority outcomes of food availability, consumption, hunger reduction, wellbeing, and attendance.

Selected schools and kura from three regions – Hawke’s Bay/Tairāwhiti, Bay of Plenty/Waiariki and Otago/ Southland – took part in the evaluation.

The aim of the interim evaluation was to ensure our decisions for the future of the programme are based on evidence.

A second evaluation of the expanded programme was completed in 2022. The Impact Evaluation is an independent evaluation focussed on wellbeing in secondary school students, impacts on attendance across all year levels, and providing case studies on stories of greatest change for whānau.

The evaluation findings show significant positive impacts overall in terms of wellbeing, food security, and the nutritional value of Ka Ora, Ka Ako meals.

Food Security in New Zealand

Food insecurity in New Zealand is a real issue for may households, with the latest 2020/21 New Zealand Health Survey indicating approximately 15 percent of children live in households with moderate to severe food insecurity.

Ka Ora, Ka Ako | New Zealand Healthy School Lunches pilot evaluation found that before the lunch programme was introduced, 21 percent of primary and intermediate school-aged kids were hungry after consuming the lunch available to them. This finding suggested inadequate food quantities were available, the larger concern was the type of food available to them.

Data on Kiwi children’s dietary intake during school hours indicates that lunch

(between 12pm and 2pm) provides approximately 20 percent of their daily energy and macronutrient intake, and approximately 15 percent of daily calcium, iron, and vitamin A intake.

School-Based Food Programmes

School-based food programmes have demonstrated effectiveness across several pertinent outcomes and in many international settings. One systematic review of 216 programmes in 52 low-and-middle-income countries found that school food programmes are one of the few education-relevant interventions that show a positive impact on school enrolment, attendance, and learning.

These same broad benefits, however, are not always replicable, particularly when comparing these results with those from trials conducted in high-income countries in the short term.

In one systematic review of universal free school meals in economically developed countries, 47 robust studies highlighted mixed results. While nearly all studies examining universal free school meals found positive associations with school meal participation (diet quality, food security, and academic performance), the research findings shows only some significant effects for attendance and only for specific subpopulations rather than overall.

Ka Ora, Ka Ako

The New Zealand-based healthy lunches programme was first announced in 2019. Rather than targeting individual students (like in the U.K), it targets schools and kura with the highest concentration of kids that face socio-economic barriers and provides lunches to all the students within the school.

The universal approach rather than the targeted approach was intended to reduce the stigma associated with needing food and ensure those most in need took up the school lunches. But this could also be a possible reason as to why food waste has been reported by some schools within the programme.

The Evaluation

The evaluation assessed the impact of the programme on secondary students’ wellbeing and the impact of the programme on attendance among all age groups. It extends upon the earlier evaluation of the Ka Ora Ka Ako pilot that focused on the programme’s impact on wellbeing and attendance among primary and intermediate-aged students involved in the initial roll out.

The evaluation serves an accountability purpose for the New Zealand Government (funders) and the Ministry of Education (MOE) (responsible for implementation) and provides the New Zealand public with an understanding of what has been achieved by the taxpayer funded programme.

The evaluation, approved by the MOE and agreed with Cabinet, asses the non-monetary value of the programme for New Zealand. Of particular interest was to estimate the programmes impact on the wellbeing of students who rarely would otherwise have enough to eat, this evidence was intended to inform decisions about any future roll out of the programme.

Putting things in Perspective

In December 2018, the Government passed into law the Child Poverty Reduction Act 2018 and the Children’s Amendment Act 2018, with the goal to ensure successive governments retain a commitment to reduce child poverty and improve child wellbeing.

New Zealand’s first Child and Youth Wellbeing strategy was launched in 2019. The Ka Ora, Ka Ako programme is part of this strategy.

Since 2019, Ka Ora Ka Ako has grown and now reaches approximately 215,000 students in schools and kura. The pilot programme was delivered in 2020 and released in tranches across three regions. In response to the COVID-19 pandemic, the programme was expanded, covering more regions in New Zealand.

By 5 September 2022, there were 981 schools and kura enrolled in the programme.

The context of New Zealand schools and kura is that governance is a local responsibility bestowed onto their Board of Trustees. The local decision-making means that schools and kura decide which programmes they take part in and how they take part, reflecting their existing policies, priorities, facilities, and activities.

Nearly all the invited schools and kura (97.3 percent) agreed to take up the programme. Most of these participating schools and kura (715) are providing lunches through the external delivery model. Fewer schools and kura (231) hire staff and/or use school facilities to provide lunches to their students, or work in partnership with other schools and kura who have hired staff – the internal delivery model.

COVID-19 was a significant factor influencing programme delivery. The effects are clearly evidenced in student attendance rates, comparing 2019 with 2020, 2021, and 2022.

Evaluation Approach and Methods

The evaluation was commissioned to help the MOE’s Policy and Special Projects team to understand the effectiveness of the programme and to inform the Government and the public about the impact of the programme.

The evaluation made us of a mix of methods including:

• Survey of wellbeing among secondaryaged students

• MOE’s school attendance data

• Case studies exemplifying how the programme benefitted some students, schools, providers and family using a strengths-based, storytelling approach

The value of the programme was determined according to the measured impacts among students, and more aptly, how widely these impacts were felt among the student population.

Findings And Supporting Evidence

Food Security Among Secondary School-Aged Students

The findings in this section were derived from the responses to the learner wellbeing questionnaire. Secondary students (from both schools within the programme and those not in the programme) were asked to complete two sets of questions asking them about the frequency at which they ate different type of food at home, and separately at school.

Key Findings:

• About 7.3 percent of secondary students within schools and kura with moderate disadvantage didn’t have sufficient food at home on any day in the previous week

• More than half of secondary students (54 percent) in Ka Ora, Ka Ako schools had sufficient food in school everyday compared to the 40 percent of their peer respondents without the healthy lunch programme.

• The pilot was found to significantly improve satiety among those students most in need with students feeling 3.8 percent fuller on average than those who were not receiving the healthy school lunches.

• More secondary students are eating vegetables at Ka Ora, Ka Ako schools and kura while fewer are eating snacks and sweets.

Ka Ora, Ka Ako Resulted in Better Wellbeing

The evidence for the pilot evaluation conducted with primary and intermediate students demonstrated mixed results in terms of wellbeing. This was noted as unsurprising given the early stage of the pilot initiative.

One year later, the evaluation considered the wellbeing effects of the programme on secondary students using similar wellbeing measures – effects above and beyond what would have been achieved without the school lunches (measured by similar students not in the programme).

The evaluation demonstrated that the programme exceeded expectations in terms of wellbeing with large and significant benefits for those most underserved students.

Key Findings:

• Better mental wellbeing of secondary students, with those underserved students within the programme realising a 14 percent advantage in mental wellbeing than their underserved peers in other schools.

• This benefit appeared to spill over into the broader population of students with a 6.4 percent advantage in mental wellbeing for the average Ka Ora, Ka Ako secondary students.

• The programme resulted in improved energy and ability to undertake a range of physical activity with the programme contributing to a 16 percent physical functioning advantage for the most underserved students.

• Improved ability to function in school with the programme contributing a 20 percent school functioning advantage for the most underserved students.

• Improved emotional and social functioning with the programme contributing to a 9-12 percent advantage for the most underserved.

• The programme resulted in an overall better health quality of life for secondary students.

• An 8.4 percent reduction in secondary students designated ‘at risk’ of impaired health quality of life.

No Contribution to Overall Attendance

Regular attendance rates have been gradually declining in New Zealand since 2015. The recent drop, between 2020 and 2021, would be likely attributed to the COVID-19 pandemic.

While there has been mixed evidence about universal lunch programmes effects on attendance in developed countries, the MOE expected that providing food to students would result in an increase in attendance.

The evidence across the MOE attendance data demonstrated no change in relation to attendance for the overall student population.

What do the Results Mean?

The average primary and intermediate student made small but significant gains in emotional and physical functioning, and overall health quality of life. Secondary students benefited even more broadly in terms of wellbeing. Collectively the results demonstrated happier and healthier students. No Evaluation of the Broader Impacts of the Programme on Suppliers, Businesses and Society

Unfortunately, no evaluation has been done on the effects of the programme on suppliers, contributing businesses, employment, or larger social impacts such as waste (from food and packaging). H