Eg rce report final

Final summative evaluation of the Engagement Gateway Project within the Regional Competitiveness and Employment area October 2012

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

Report prepared by: Richard Brooks Any queries or questions about this report should be referred in the first instance to Richard Brooks at Wavehill: t: 01545 571711 | e: richard.brooks@wavehill.com Client contact: Tessa White, Engagement Gateway and Grants Manager, WCVA t: 029 2043 1755| e: TWhite@WCVA.org.uk Report version: Final

ŠCopyright Wavehill 2012

1

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

Contents 1.

Introduction ....................................................................................................................... 3 1.1.

2.

Overview of the RCE Engagement Gateway project and the purpose of the evaluation . ............................................................................................................................................ 5 2.1.

2.2.

2.3.

2.4.

3.

Recession in Wales....................................................................................................... 17 The Policy Context ....................................................................................................... 20

Analysis of outcomes and impact.................................................................................... 23 4.1.

4.2.

4.3.

4.4.

4.5.

5.

The project .................................................................................................................... 5 Funding ....................................................................................................................... 15 Performance indicators ................................................................................................ 16 Re-profiling.................................................................................................................. 16

Economic and policy context of Engagement Gateway ................................................. 17 3.1.

3.2.

4.

The remainder of the report is set out as follows: ............................................................ 4

Introduction ................................................................................................................ 23 The data and analysis ................................................................................................... 24 Characteristics of Participants....................................................................................... 24 Employability and Employment Outcomes .................................................................... 26 Analysis by Local Authority ........................................................................................... 32

Conclusions and Recommendations ............................................................................... 39

Š Copyright Wavehill Ltd 2012

2

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

1. Introduction Engagement Gateway was a 4-year1 European funded project aimed at supporting people who, through a range of disadvantages, are economically inactive or unemployed. The project started in April 2009, entered a 12 to 15 month procurement phase, and has been working with participants since January 2011. The project covers the Priority 1 East Wales Regional Competiveness and Employment Operational Programme (RCE) area of Wales, which consists of the local authorities of Cardiff, Flintshire, Monmouthshire, Newport, Powys, Vale of Glamorgan, and Wrexham.

This is the final independent evaluation report of the project, which has been undertaken by social research and evaluation specialists Wavehill. The report has been prepared at the end of the programme, in September and October 2012, three months after Engagement Gateway delivery stopped at the end of June 2012.

Engagement Gateway is a pan-Wales programme which is part funded by European Commission ESF funding. The funding is divided between two areas of Wales, the first is named the Convergence area that consists of 15 local authority areas, and the second is the RCE area that consists of seven.

The purpose of this report is to produce a final summative evaluation report that assesses the impact and results of the Engagement Gateway project in the RCE area. The evaluation has also been asked to help explain the apparent difference in the number of project participants working with the Engagement Gateway project in the RCE area, which is lower than originally anticipated compared to the project in the Convergence area in Wales.

The evaluation method includes document review, consultation with stakeholders and some survey work with training providers and the County Voluntary Council (CVC) and Volunteer Centres across the RCE area and those responsible for the administration of Engagement Gateway within WCVA. The main analysis has been undertaken using monitoring and outcome data collected by the Engagement Gateway administration team.

The evaluation and its limitations The principal aim of the evaluation of Engagement Gateway RCE is to determine whether the project has achieved its objectives, including its effectiveness, impact and sustainability.

WEFO offers guidance for evaluators of ESF projects and asks that project-level evaluations will address the following evaluation questions, as appropriate to the individual project: • • •

1

How many (net) participants have entered employment as a result of ESF assistance?

How many of these participants are still in employment 12 months after receiving ESF assistance?

How many participants have received part qualifications as a result of ESF assistance?

October 2008-June 2012 + 3 months for Audit and Evaluation to October 2012

© Copyright Wavehill Ltd 2012

3

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

• •

How many participants who had, or were at risk of developing a work limiting health condition, have remained in employment as a result of ESF assistance?

How many of these participants are still in employment 12 months after receiving ESF assistance?

The evaluation method used in this report answers all but the last question regarding how many participants have remained in employment. The reason is that we did not have the budget or the timescale to perform a survey to inform this indicator. A survey was undertaken by a previous contractor (ERS Research) but they did not release the data for this report.

The data used for this report was based on project monitoring returns gathered by the WCVA Central Team.

1.1.

• • • •

The remainder of the report is set out as follows:

Section 2 provides an overview of the RCE Engagement Gateway project and the purpose of the evaluation; Section 3 the economic and policy context of Engagement Gateway; Section 4 an analysis of outcomes and impact; Section 5 includes conclusions and recommendations.

© Copyright Wavehill Ltd 2012

4

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

2. Overview of the RCE Engagement Gateway project and the purpose of the evaluation This section provides the background and context for the remainder of the report by introducing the RCE Engagement Gateway project together with the aims and objectives of the evaluation.

Section summary: • • • •

Engagement Gateway RCE was match and grant funded to £5 million but spent £4.2m in the end.

Wrexham and Cardiff, both urban areas, reached proportionately 3 – 4 times more participants than the more rural areas of Powys and Monmouthshire.

Originally 7740 participants were planned for, but 3142 were finally engaged, making the final participation rate 41% of target (3142/7740).

The number of contractors identified by CVCs within LA areas were quite variable and this may have contributed to differences in performance by local authority area.

2.1.

The project

Engagement Gateway is a £34.4 million European Commission (EC) funded project, with the EC ESF funds contributing £18.1 million of grant funding. It is delivered under the EC Strategic ESF Framework Themes 1 and 2 of the Convergence ESF programme and Priority 1 of the RCE ESF programme. In the RCE area the project was planned to deliver £5m of grant and match funding making the Convergence project around 6 times larger than the RCE element.

Figure 1: A map of Wales highlighting the RCE area

This evaluation deals with the RCE area of Engagement Gateway, Wales, which consists of the local authorities of Cardiff, Flintshire, Monmouthshire, Newport, Powys, Vale of Glamorgan, and Wrexham.

The aim of Priority 1 in the RCE area is to raise levels of employment and economic activity, and secure higher participation in the labour market in the East Wales region. It achieved this by helping people acquire the skills needed for sustainable employment and addressing specific barriers faced by individuals within each group.

© Copyright Wavehill Ltd 2012

5

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

The Engagement Gateway model provided training contracts and grants using a multi-layered funding facility through three tiers of ESF funding from less than £25,000 through to £150,000.

Examples of the range of Engagement Gateway activities included developing IT skills, helping obtain minibus driving licences, hair and beauty training, tree management, woodwork skills, horticulture skills, woodland management, theatre and arts activities, crime prevention services and encouraging women back to work.

Project examples: •

•

•

Innovate Trust works mainly with people with learning disabilities and delivers a wide range of projects and contracts for local authorities. Their Gateway project at Park View Café delivered catering and employment skills training to 40 individuals with learning disabilities, helping them prepare to progress to paid employment. Participants learned these skills whilst engaged within a Social Enterprise Café.

In Monmouthshire Youth Service’s ‘Time 2 Engage Plus’ Engagement Gateway project enabled young people to gain soft skills through team building activities and outdoor pursuits such as mountain biking. They also completed qualifications such as Level 1 Health and Safety and many progressed onto the ILM project being run by Melin Homes.

Community Justice Interventions Wales’ ‘Circles of Support and Accountability’ project in Wrexham was an initiative designed to enhance the safer integration of offenders into the community. For example, by booking training, helping with travel and supporting with issues of low confidence and depression to enable participants to achieve a Client Contractor National Safety Group card to qualify for employment in the engineering industry.

2.1.1 Project management and contracting of providers The structure of ESF funding is that the Welsh Government and its department the Welsh European Funding Office (WEFO) agrees a strategic framework of priorities and themes against which requests for project proposals are made. A public procurement process is used and WEFO makes open calls for project proposals from the public, private and third sectors.

In the case of the Engagement Gateway project, WCVA made two bids as a lead sponsor, one for the Convergence area and one for the RCE area. Both were successful and accepted for funding and contract.

The unique element of the WCVA bid was that it has the network of County Voluntary Councils and Volunteer Centres. It supports these in a central hub and spoke arrangement, where WCVA acts as the hub providing resources and strategic direction and CVC/VC development, and the CVCs/VCs interact and support the network of volunteers and voluntary groups within their particular area.

With 19 CVCs and two independent volunteer centres across Wales this network can access many small voluntary and training organisations effectively.

Hereafter the CVC/VC network is referred to collectively as the CVC network.

© Copyright Wavehill Ltd 2012

6

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

Figure 2: a schematic design of the contract between WEFO, WCVA, and service providers.

WEFO

WCVA

GAVO

Voluntary Community Services (Cardiff)

Flintshire Local Voluntary Council

Powys Assoc of Voluntary Orgs

Vale Volunteer Bureau

Assoc of Vol Orgs in Wrexham

Provider Network = 77 contracts in RCE area

Participants

In this format, WCVA was the lead organisation and held the project contract between WEFO and WCVA to run the Engagement Gateway project. This is beneficial for WEFO as it does not need to create many contracts to deliver its obligations under the ESF priorities, but can enter into just one agreement, with WCVA. This ‘one not many’ contracts arrangement significantly reduces the administrative overhead for WEFO and passes the administration over to the contract lead, in this case WCVA, which is formally named as the lead sponsor.

The administrative overhead required a team of around 24 full-time equivalent administrators at WCVA, (pro-rata for the RCE project) including roles such as advisors, claims handlers and contract bid and grant assessors, as shown in the diagram below.

© Copyright Wavehill Ltd 2012

7

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

Figure 3: The Engagement Gateway existing staff team

Project Manager

Procurement & Gates Manager

Assessor x 5

Administrator

Development and Support Manager

Adviser x 5

Local Advisers

The WCVA project team works across both Convergence and RCE projects with costs allocated as per hours worked (approximately a 7:1 ratio).

Compliance Manager

Organiser

Compliance Adviser x 3

Administrator x4

Administrator

WCVA as lead sponsor then undertook a sub-round of contract procurement for the Engagement Gateway, to recruit employability providers to deliver services to the target group of individuals who needed support to gain employment or to gain the skills to get closer to and compete in the employment market.

In the RCE area, Engagement Gateway recruited 45 organisations to manage 77 contracts to deliver services to participants. Some organisations were awarded multiple contracts; see the table below for contractors awarded three or more times.

Table 1: a list of contractors awarded three or more contracts Organisation The Prince's Trust Seal Enterprises (Training Division) Innovate Trust Rathbone Cymru Caia Park Partnership Ltd Solas Cymru Ltd Clybiau Plant Cymru Kids' Clubs The Women's Workshop, Cardiff Training Centre

Contracts awarded 9 6 4 3 3 3 3 3

Source: WVCA management data

Š Copyright Wavehill Ltd 2012

8

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

2.1.2 Costs of administrating Engagement Gateway Of course, the administrative burden also requires revenue funding to cover the administrative costs.

The Engagement Gateway required 24 administrators. The proportion of Convergence to RCE administration costs is 85% (£2.8m) and 15% (£0.6m) of total budget. The table below shows that the Convergence CVCs received proportionally more funding than for their RCE peers. However this is also due to the different ratios of match funding allowed within the convergence area compared to the RCE area.

Table 2: Administration costs of the Convergence and RCE areas compared.

Costs Proportion

Convergence costs £2.77m 85%

Convergence CVC costs of £2,247,013 £149,800 per CVC 75%

RCE costs £0.50m 15%

RCE CVC costs £732,813 £104,687 per CVC 25%

In summary, the RCE area received less funding for its CVC network than in Convergence, proportionally 43% less or in real terms around £45,000, which represents at least one full time member of staff. However, with fewer projects and participants to manage the number of staff required was also less with approximately four staff assisting the RCE administration.

2.1.3

The contribution of CVCs to the procurement process

This section uses interviews with four CVCs and two VCs to help inform the synthesis.

The CVCs act as the local activists for Engagement Gateway in each of the seven RCE local authority areas.

By having a strong level of local knowledge about voluntary and third sector organisations and training providers that provide services to unemployed people in their specific area each CVC was given the role to help reach potential local providers and to encourage them to bid for provision contracts. Later on, from 1st April 2011, the contracts were changed to grants and again the role of the CVCs was to reach providers, advise them of the changes and support them in developing appropriate Gateway applications that delivered employability training to participants using the grant funding as revenue.

Interviews with CVCs indicate some tension and misunderstanding between the CVCs and the tendering process. CVCs also mention the ‘long delays’ in receiving the approved supplier list from WCVA post PQQ, described as a period of 6-8 months depending on the interviewees’ recollection.

However, as the table below shows the tender rounds opened and closed frequently and lists of PQQ outcomes were published 8-12 weeks after the round had opened. This lack of understanding of the tender process may have reduced the number of successful applicants and may have affected the number of participants attending EG projects.

© Copyright Wavehill Ltd 2012

9

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

Tender and Grant Timetable Tender round 1 opened Jan 2010 Round 2, April 2010 Round 3, July 2010 Round 4, October 2010

Round 5, December 2010 Round 6, March 2011 Grant rounds 7, 8 and 9 were opened in April 2011

Our interviews with CVCs also show that targets for the number of providers or the potential demand for services were not given to CVCs, however, this local intelligence was also partly the role of the CVC / VC network.

WCVA did however provide a SLA target of 20 providers for each area. It may have helped establish the extent of CVC area provider recruitment if WCVA had given forecast demand intelligence to the CVCs. With a CVC level demand estimate, the CVCs may have been able to recruit providers slightly more strategically, essentially linking expected demand to supply. Different areas have different population densities and unemployment rates and it may have been useful to provide a more granular analysis of where the services were predicted to have been in demand.

Management data shows that 77 contracts were let across 43 contractors, which is an average (mean) of 11 contracts per each LA area across approximately 6.5 contractors. This suggests that the WCVA – CVC system identified fewer contracts and contractors than may have been anticipated when the business plan and service level agreements were drawn up.

Other issues also arose during interviews with CVCs. CVCs noted that the change in ESF eligibility criteria (economically inactive only vs registered unemployed) reduced the pool of potential participants and that “far more people could have been recruited.” However, our analysis below shows that only Wrexham managed to provide Engagement Gateway services to more than 10% of those people registered as economically inactive but wanting a job in their respective LA areas, so this point may have been used to deflect low participation rates. Also the timing of the changes only affected round 6, and the three grant rounds post April 2011.

Another quite serious issue (for WEFO et al) that was raised by CVCs was the culture of providers being ‘scared’ of entering into EC/ESF grants and contracts, with informal discussions about contracts being withdrawn and handed back affecting provider confidence. Also, there was some narrative evidence of contractors being dissuaded by the administrative overhead required to manage ESF contracts.

Furthermore two CVCs noted that there was too much duplication or overlap of similar projects “swamped with these types of projects” and that Engagement Gateway created a tranche of projects with overlapping or similar aims.

The design of the hub and spoke hierarchy led to the CVCs being empowered to identify service providers, who were then encouraged to tender for service delivery. However, this system also led to the potential for CVCs to miss out on identifying enough providers if their networks in their locale were not inclusive.

© Copyright Wavehill Ltd 2012

10

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

This may have been the case in Powys where the CVC was working with providers to encourage them to bid for a second round of funding having found the first round less subscribed. However the second round did not emerge and so the provision across Powys was not as extensive as it could have been if the funding had reached a second application stage. Again this demonstrates some confusion as the PQQ stages were limited to two rounds (these are stages to qualify to apply for tenders) and not the tenders that, as the table above shows, had 7 tender rounds and three grant rounds.

It should also be noted that according to the Wales Rural Observatory, Powys is a difficult county to provide services in as it is rural, has a low population density and only one town with a population greater than 10,000 people (Newtown) and access to public transport and road communications in general are limited. This results in social exclusion for some people due to access to services, poor transport links and a lack of administrative centres and civil society structures.

2.1.4

Who can receive assistance in the Engagement Gateway project?

Using NOMIS data finds that in total across the RCE area there are an estimated 184,540 people who are economically inactive but not retired. Of course, not all of these people are looking for employment and it is worth considering who of those listed as Economically Inactive but Wanting to Work, may have been Engagement Gateway participants.

The Labour Force Survey is the main government survey for analysis of the workforce, in terms of both the jobs people do and the characteristics of the people themselves. LFS also includes data about all the adults in a household and can therefore be used to analyse the overall work status of households (e.g. numbers of children who are in workless households).

The table below shows categories for people who are economically inactive but wanting to work, and therefore could be representative of Engagement Gateway participants.

Table 3: showing categories of economic activity and inactivity from the Labour Force Survey Value 1 2 3 4 5 6 7 8 9 10 11

Work status Employee Self-employed Government employment and training programmes Unpaid family worker ILO unemployed Inactive - seeking, student Inactive- seeking, looking after family home Inactive - seeking, temp sick or injured Inactive - seeking, long-term sick, disabled Inactive - seeking, unavailable, other reason Inactive - seeking, unavailable, no reason given

Grouping Working Working Working Working Unemployed Economically Inactive Economically Inactive Economically Inactive Economically Inactive Economically Inactive Economically Inactive

Source; (www.poverty.co.uk)

Š Copyright Wavehill Ltd 2012

11

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

The LFS asks a question to those who are economically inactive, if they would like to work or find a job. Some answer, yes, but are not receiving JSA benefits and the graph below shows the proportions of people classified as economically inactive, but who wish to work. This is a useful number as it allows us to estimate the number of people across the RCE area who would be potential participants in the Engagement Gateway project, and from that, potential (latent) demand.

Graph 1: Proportion of people recorded economically inactive who wish to find a job

21 22

% of economically inactive females who want a job

26

% of economically inactive males who want a job

27 23 24

% of economically inactive who want a job 0 Wales

5

10

15

20

25

30

United Kingdom

Source: annual population survey; ONS Crown Copyright Reserved [from Nomis on 27 November 2012]

The graph shows that around 21% of females and 26% of males wish to find a job. The difference is likely contributed to by retired females who live longer than males and females who look after children and who act as unregistered carers.

Extrapolating this data back into the RCE and Engagement Gateway context we can suggest that from the 184,590 individuals who are recorded as economically inactive (excluding-retired), approximately 47,500 would like to find work. This is shown in the table below.

The original major target group of participants for the RCE Engagement Gateway was participants that are ‘economically inactive’. Economic inactivity is defined in the UK as people aged 16 and over without a job who have not sought work in the last four weeks and/or are not available to start work in the next two weeks. The main economically inactive groups are students, people looking after family and home, long term sick and disabled, temporarily sick and disabled, retired people and discouraged workers. Also to claim Job Seekers Allowance (JSA) recipients must be ‘looking for work, able to work, available to work’ (DWP 2012) and so the term economically inactive excludes those claiming JSA.

2

https://www.gov.uk/jobseekers-allowance/overview

© Copyright Wavehill Ltd 2012

12

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

Using data from NOMIS in the chart below we can see the proportion of citizens in the RCE area local authorities that are classified as economically active. Note that all are either equal to, or higher than, the total for all Wales (61%). This means that the RCE has a greater proportion of citizens who are economically active than for Wales, and is an important difference between the RCE region compared to the Convergence region.

Graph 2: Economic activity proportions by local authority area in RCE Wales 70% 68%

68% 66%

65%

64%

66%

66%

66%

63%

63% 61%

62% 60% 58% 56%

Economically active Wales

Cardiff

Newport

Wrexham

Vale of Glamorgan

Monmouthshire

Powys

Flintshire

Note – horizontal red line denotes value for All Wales.

The opposite perspective of economic activity is to consider economic inactivity, which is the true focus of the Engagement Gateway intervention. It should be noted that some people classified as economically inactive are through choice, rather than lack of employment opportunities, people who have taken early retirement or those who remain at home caring for young families, for example.

Graph 3: Economic inactivity proportions by local authority area in RCE Wales 27%

26% 24%

25% 23%

21%

21%

23%

20%

19%

18%

19%

19%

17% 15% Economic Inactive (Excluding Retired) Wales

Powys

Flintshire

Š Copyright Wavehill Ltd 2012

Monmouthshire

Vale of Glamorgan

Wrexham

Newport

Cardiff

13

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

The data above shows that Powys, Flintshire, Monmouthshire and Vale of Glamorgan may be the local authority areas with the least potential beneficiaries accessing the Engagement Gateway project as there are fewer classified as economically inactive. We have excluded retired citizens from this data as people tend to retire permanently and not re-enter the workforce. Only Cardiff has more people classified as economically inactive than the proportion for all Wales. This means that Cardiff may have received a higher demand for its Engagement Gateway services.

The table below shows the number of people who are economically inactive and the number of those who state that they would like to work. The table then compares the proportion for each local authority against the number who have been through the Engagement Gateway project. This gives some proportion or scale to the participation levels in the project.

Table 4: Number and proportion of people recorded economically inactive (excluding retired) who wish to find a job, by Local Authority (LA) and the proportion who have been on the Engagement Gateway project

Cardiff Flintshire Monmouthshire Newport Powys Vale of Glamorgan Wrexham Column total

No. economically inactive who want a job 15,000 6,300 3,000 4,900 3,800 4,000 4,100 47,500

No. Economically active population 67,500 22,000 11,500 22,200 17,000 18,500 18,600 207,700

% of economically inactive who want a job 22 29 26 22 22 21 22 23

proportion serviced by Engagement Gateway 10% 3% 4% 9% 4% 6% 23%

Source: annual population survey; ONS Crown Copyright Reserved [from Nomis on 27 November 2012]

In terms of the proportions of people who have accessed the Engagement Gateway project, the table above shows that the LA areas of Wrexham (23%) and Cardiff (10%) have performed relatively better in terms of the proportion of participants engaged as a proportion of those economically inactive but wanting to work. Both of these are urban areas with a relatively high-density population, and possibly this is a factor for consideration. This notion is strengthened by the proportions of people serviced by the Engagement Gateway project from the less densely populated rural counties of Powys (4%) and Monmouthshire (4%). Although it is not true that Flintshire (3%) is classified as a rural county, other factors must be contributing to this lower service proportion.

The Engagement Gateway RCE project was given a target 70% to work to of participants classified as Economically Inactive and 30% classified Unemployed.

In summary, the RCE LAs have more people economically active than the aggregated Wales figure (approximately 24%), suggesting a more robust labour market than in the convergence area.

The proportions of people serviced by the Engagement Gateway project varies widely once the data is disaggregated by LA and shown as a proportion of the population who are economically inactive and want to work (ex. retired) (3% in Flintshire > 12% in Wrexham). Rurality may be a barrier to take up of Engagement Gateway services as rural areas significantly underperform non-rural areas.

Š Copyright Wavehill Ltd 2012

14

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

2.2.

Funding

As shown in the table below, Engagement Gateway is part funded by the European Social Fund (ESF) programme in Wales.

Table 5: Engagement Gateway project costs, ESF and match-funding by value Income Total project amount: ESF grant Targeted match funding: In kind match funding from suppliers: In kind match funding from CVCs: Expenditure Total project amount: Contracts and grants, including match funding: CVCs local advisor service, including match funding: WCVA administration costs:

Target £5,000,000 £2,000,000 £2,200,000 £630,279 £169,721

Actual £4,219,617 £1,185,695 £2,200,000 £569,602 £264,320

% of Target 84% 59% 100% 90% 156%

£5,000,000 £3,771,025 £732,813 £496,162

£4,219,617 £2,943,935 £788,938 £472,021

84% 78% 108% 95%

Source: Engagement Gateway Team

The RCE area has been awarded the second level of financial support under the European Union Structural Funds round 2007–2013.

The European Social Fund (ESF) programme will utilise approximately 47% of the resources from the total Regional Competitiveness and Employment funding. This means an ESF contribution of €63,597,452 of which the Engagement Gateway project accounts for approximately 2.6% (£1.185m).

Engagement Gateway is funded under Priority 1, Theme 1 of the ESF programme. The aim of Priority 1 (Increasing employment and tackling economic inactivity) is to raise levels of employment and economic activity, and secure higher participation in the labour market. It will do this through the delivery of two themes. Theme 1 (under which Engagement Gateway is funded) aims to increase employment and tackle economic Inactivity by bringing more people into employment. Theme 2 is focused on helping people to remain in work.

© Copyright Wavehill Ltd 2012

15

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

2.3.

Performance indicators

The table below shows the targets that were set for the Engagement Gateway project. The targets are split into outputs (the activities to be undertaken) and results (the immediate or direct results of the activities).

Table 6: Engagement Gateway project headline performance indicators RCE Participants Positive outcome Gaining a qualification Entering volunteering Entering further learning Entering employment

Target 7740 5418 1935 no target 1935 774

Actual 3142 2640 1172 479 492 329

% Achieved 41% 49% 61% 25% 43%

% of Participants 84% 37% 15% 16% 10%

Source: Engagement Gateway Management Data

2.4.

Re-profiling

Engagement Gateway has been subject to a re-profiling exercise in 2010 and 2011. These re-profiles were caused by changes to the context within which the project was being delivered including the recession and introduction of the Department for Work and Pensions Work Programme and some systemic lower participation rates in the RCE area.

These re-profiling exercises affected the project timeline, extending the project by 6 months to September 2012 from March 2012 and changing from match funded tenders to grant awards, with more funding for CVC activities.

Overall expenditure and targets were unchanged both times.

Š Copyright Wavehill Ltd 2012

16

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

3. Economic and policy context of Engagement Gateway This section seeks to place the Engagement Gateway project within a policy and economic context which have, or may have, an impact on the Engagement Gateway programme, or the way it is reviewed and evaluated.

Section summary •

• •

Since the original business plan for the Engagement Gateway project was prepared, the UK has been through a period of two recessions, which has had a number of economic impacts that influence the Engagement Gateway project; most notably increasing unemployment and weak labour market demand.

The poor performance of the Work Programme and its comparison to ESF funded projects demonstrates that it is hard to move unemployed people into jobs.

Unemployment is disproportionately affecting younger people and longer term unemployed, both target groups of the Engagement Gateway project.

3.1.

Recession in Wales

3.1.1 Introduction As a reminder of the context in which Engagement Gateway is being delivered we have included the following section.

On the 27th November 2012, the Office for National Statistics (ONS) released initial GVA estimates suggesting that the UK economy in Q3 2012 was unchanged at 1 per cent. This growth followed three consecutive quarters of negative growth (recession) and was the highest since the third quarter of 2007.

GDP growth in the third quarter was entirely due to a contribution of 1 percentage point from services, with small and offsetting contributions from higher production and lower construction output.

The original business plans for the Engagement Gateway were approved during 2008, just before the collapse of US mortgage debt, the collapse of Lehman Bank in the US and the emergence of the recession in late 2008 – early 2009, since which time the economic and labour market situation has changed considerably.

Recession has led to a review of Government spending, which under the previous Labour administration had been calculated using only growth and expansion calculations and a reduction in the use of public and private sector debt to fund economic expansion. Economic growth in the UK and Wales remains weak (1% in Q3 2012; ONS).

© Copyright Wavehill Ltd 2012

17

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

3.1.2 Impact on the labour market in England and Wales – Q3; 2012 UK Unemployment fell by 49,000 to 2.51 million in the three months to September 2012, taking the jobless rate to 7.8% from 7.9%, almost all the 49,000 fall was due to a decline in youth unemployment. The unemployment total is now 110,000 lower than for the July-September quarter 2011. The number of people in work increased by 100,000 in the latest quarter to just under 30 million, a rise of more than half a million over the past year. (ONS;2012) However, other data showed that long-term unemployment - those out of work for over a year increased by 12,000 in the quarter to September to 894,000, while 443,000 people have been jobless for more than two years, up by 21,000. The labour market is hard for longer term unemployed people, but appears to be easing slightly for younger people.

3.1.3

Impact on Wales

Perhaps the most pertinent impact of the recession in Wales is the impact on the labour market, specifically unemployment. There are two standard measures for unemployment used in official statistics in the UK: the ILO unemployment measure and the claimant count. These are continually updated by the ONS and the latest data for each of these will now be presented.

3.1.4

Unemployment by RCE and Convergence

The graph below shows the unemployment rate rising from around 6% pre 2008 to around 8% in 2012. The unemployment rate in RCE area is consistently below that of Wales and Convergence areas.

Graph 4: ILO calculated unemployment rate for the RCE area and Convergence area in Wales 10 9.1% 8.7% 7.9%

9 8 7 6 5 4 3 2001

2002

2003

2004

2005

All Wales

2006

2007

2008

Convergence

2009

2010

2011

2012

RCE

ILO 2012

Š Copyright Wavehill Ltd 2012

18

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

3.1.5

Impacts in the programme area

The following graphs show the unemployment rate in each local authority area for younger people aged under 25 years of age and then those over 25 years of age. They are included to show general trend rather than exact values. Note the increase from 2008, when the project was designed, to 2012 which shows the approximate doubling of unemployment in the younger (under 25 years) age groups. The graphs show the effect on longer-term unemployed is greater than for younger people.

Graph 5: Claimants under 25 years in each local authority area in RCE area.2000 – 2012 14.0 12.0 10.0 8.0 6.0 4.0 2.0 0.0 2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

Cardiff %

Flintshire %

Monmouthshire %

Newport %

Powys %

Vale of Glamorgan %

2012

Wrexham % Source: NOMIS

The graph below shows claimants claiming for longer than a year. This illustrates the difficulty for those who are becoming long term unemployed of finding employment. The rate has increased from around 7% in 2008 to around 20% in 2012. The implication for the Engagement Gateway project is that it is increasingly difficult for longer term unemployed to transition from unemployment to employment.

Graph 6: Proportion of claimants who have been claiming employment benefits for more than one year 30.0 20.0 10.0 0.0 2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

Cardiff %

Flintshire %

Monmouthshire %

Newport %

Powys %

Vale of Glamorgan %

2012

Wrexham % Source: NOMIS

Š Copyright Wavehill Ltd 2012

19

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

3.2.

The Policy Context

3.2.1 Introduction This section explores the changing policy context in Wales (and the UK) since the inception of Engagement Gateway.

• • •

• • • • •

•

•

•

’Improving Specialist Disability Employment Services‘(DWP) -the aim is to help more people with disabilities to have mainstream employment opportunities.

Older People Strategy for Wales – the project will help older people remain independent through helping them into employment.

‘Fulfilled Lives, Supportive Communities’ - one of the core aims of this project is to promote independence amongst people with social care needs, through helping them engage with training and employment.

‘Wales: A Better Country - this project will help people to break out of poverty related issues by supporting them with training and moving on to employment.

‘Wales – A Vibrant Economy’ - the project will support job creation and help individuals to tackle barriers to becoming employed.

‘Skills and Employment Action Plan’- the project will contribute to; “Interventions to reduce or remove the barriers to participation in work.”

‘One Wales’- principles of social justice, sustainability and inclusivity.

‘Enhancing Skills for Jobs’ - there is a major focus in this project on supporting people from a range of disadvantaged groups to learn new skills that should enhance their prospects for moving on to employment.

‘Skills that Work for Wales’- elements of the project include developing community based learning initiatives for people from disadvantaged groups, who have often experienced difficulties in formal educational settings.

Community Cohesion Strategy - many of the participants that this project intends to help will have been out of work for a considerable period of time and not had the opportunity to contribute to their communities as active citizens.

‘Regenerating Communities’- the project will operate within some areas that have been recognised as strategic regeneration areas by the Welsh Government and contribute to the implementation of SRAs.

© Copyright Wavehill Ltd 2012

20

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

3.2.2

Work Programme

Central to the reform of welfare is the Work Programme3, which works alongside changes to welfare benefits. The Work Programme replaces all previous mainstream (government) support activity, and is, in theory a ‘black box approach’ open to all, although disabled people may get support from the Work Choice Programme4. The new provision aims to offer individualised support underpinned by an element of benefit compulsion where people not undertaking required activity can be subject to benefit sanctions and possible mandatory activity. People coming out of their Work Programme without a job may be put onto Mandatory Community Activity and could cycle between the two if they do not gain employment.

Update on the performance of the Work Programme At the time of writing (November 2012) a formative evaluation of the Work Programme has been released and this has some relevance to the performance of the Engagement Gateway and other similar ESF programmes. Only one in 28 unemployed people referred to the government's Welfareto-Work Programme has been found a job for six months – failing to meet the government's target.

None of the 18 Work Programme contractors managed to get to the target of 5.5% of unemployed people referred to the scheme a job for half a year in the 14 months until July 2012.

Ingeus (the largest Work Programme contractor) referred almost 28,000 jobless people and got 920 into sustained employment, a success rate of 3.3% to July 2012. A4e, which is the second biggest contractor to the programme, found 490 jobs for 17,650 unemployed people in the south of England, a performance rate of 2.8%. (Guardian 2012).

The Employment Minister (Mark Hoban) said the figures had to be considered "against the backdrop of much weaker than expected growth. We had been expecting growth of 2 or 2.5% a year by now."

The Centre for Economic and Social Inclusion noted “… the Work Programme has achieved job outcomes for 4.5% of referrals in its first 14 months, compared with 5.0% for the Flexible New Deal and 5.6% for the predecessor New Deal programme."

The notable point for Engagement Gateway and other ESF projects is that the DWP minister is in part blaming underperformance of job outcomes on to the conditions in the wider economy and its transfer of effects into the Work Programme. In addition, Work Programme participants are closer to the employment market, with less long-term unemployment histories than ESF Priority 1 participants, and so the ESF outcomes could be expected to be below that of the Work Programme.

3 4

http://www.dwp.gov.uk/policy/welfare-reform/the-work-programme/ http://www.direct.gov.uk/en/DisabledPeople/Employmentsupport/WorkSchemesAndProgrammes/DG_187696

© Copyright Wavehill Ltd 2012

21

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

3.2.3

The 2010 European Social Fund Leavers Survey

The 2010 ESF Leavers Survey involved a telephone survey of 7,500 people who had participated in ESF projects delivered under Priorities 2 and 3 of the Convergence Programme and Priorities 1 and 2 of the RCE Competitiveness Programme. The survey can be used to compare the outcomes from various ESF funded projects against the outcomes across the whole programme, albeit from 2010.

• • •

•

•

•

•

•

Among Priority 2 respondents, 39% were in paid employment at the time of the Wave 2 survey.

The main reason Priority 2 Convergence respondents got involved in their particular project was to get a job, followed by widening career options and to develop a broader range of skills; Despite this, a mixed outcome was reported for Convergence Priority 2 and Competitiveness Priority 1 respondents - 44% reported a positive change and 45% reported no change in their economic activity status.

However the study positively reports upon the immediate benefit of participation in these projects in that ‘a majority of transitions out of unemployment and inactivity amongst Priority 2 respondents from the Convergence Programme occur either during or immediately following their participation in an ESF project’.

The report also goes on to state that amongst those in a job at the time of the survey that was not held prior to ESF 22% thought that their course had been vital to them in obtaining their current job.

In terms of impact the study concludes from the statistical matching that the participation of unemployed people within an ESF project aimed at increasing participation in employment is associated with a 13-19% increase in the chance of gaining work compared to the wider unemployed population.

Moreover, one in five of the respondents who had been supported by projects aimed at improving participation in employment and who were not in employment at the time of the survey felt that they had more chance of finding employment in the future as a result of participating in ESF. Overall, the most commonly cited skills acquired by respondents during their ESF project were communication skills, team working skills, organisational skills and problem solving skills.

© Copyright Wavehill Ltd 2012

22

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

4. Analysis of outcomes and impact This section sets out the outcomes and impact of the Engagement Gateway as measured by the participant data collected by WCVA and its partners.

Section summary • • • •

Engagement Gateway has worked with 41% of its target of participants across the region 3142 vs 7740.

Population adjusted rates show urban areas with greater Engagement Gateway participation rates than for rural areas.

Rurality and social exclusion may explain participation rate differences across local authority areas.

Employment outcomes are 10%, which is in between the range of outcomes for the ESF Leaver Survey (39%) and Work Programme (2.3%) and proportionately in line with forecast targets.

4.1.

Introduction

The section below analyses and discusses the impact on the participants. It is useful to note that in the design of the Engagement Gateway project it was recognised that many outcomes would be soft or other outcomes, rather than hard outcomes such as employment and/or qualifications.

Extract from Engagement Gateway Business Plan – 2009 Whilst a high proportion of the targeted actions under this project will not directly impact on the ‘hard outcomes’ that are defined within the Strategic Framework, they demonstrate an investment towards these targets. The actions supported will provide the longer-term investment and support needed for participants to achieve these harder outcomes at a later date. The actions will provide the ‘soft outcomes’ that will represent the intermediary stages on the way to achieving a hard outcome.

They could include, for example, thinking skills such as improved problem-solving abilities, personal attributes such as improved self-confidence, or practical work-focused skills, such as a better appreciation of the importance of timekeeping in the workplace.

The analysis below considers the results produced by the Engagement Gateway project across the RCE area. The analysis also considers the results by local authority area, and by socio-demographic group.

One of the reasons to disaggregate the results by local authority area was to show the differences in performance by area. This is not to create a competitive league table, but to illustrate how various geographic factors may influence the rates of participation and benefit from Engagement Gateway.

© Copyright Wavehill Ltd 2012

23

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

4.2.

The data and analysis

The data we have used for this analysis section was provided by the central administration team at WCVA. The data is collected during monitoring and then loaded onto a spread-sheet database (Excel). We have cleaned the data to standardise the fields and then used Excel and an SPSS variant to create the analysis below.

In total the database recorded 3149 participants across the RCE area. However, the graphs below show slightly lower totals. These are mainly caused by missing data. A few participants were recorded against addresses outside of the RCE area, some have inaccurate data, which excludes them from analysis, one participant is recorded both as a young person and having an age of 120 years, for example. However, this is not an audit but an evaluation of outcomes and impact based on the participants database. Where possible errors have been identified and excluded from analysis.

4.3.

Characteristics of participants

Equality of inclusion and benefit is a cross cutting theme that runs across ESF funded projects across Wales, and is embedded within UK law under the Equality Act 2010.

The following section reviews the characteristics of participants.

The chart below shows the employment status of the participants as they joined the Engagement Gateway. Unsurprisingly all were categorised as unemployed (47%) or economically inactive (53%) but presumably wanting to work. It is useful to compare the long-term unemployed proportions to those in the charts in section 3. The Engagement Gateway has worked with proportionately fewer unemployed people who have been out of work and claiming benefits for more than one year; (13% vs approximately 17%).

Chart 7: Employment status at start of Engagement Gateway participation 60% 50% 40% 30% 20% 10% 0%

53% 34% 13%

Economically inactive

Long-term unemployed Unemployed (up to one (over one year) year)

Un/employment status of joining participants N = 3102

Š Copyright Wavehill Ltd 2012

24

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

The next graph shows the age profile of participants by gender. The Labour Force Survey (DWP 2011) estimates that 8% of NEETs are male and 7% female, but within Engagement Gateway RCE 34% of females are classified as NEET and 54% of males.

Graph 8: Age and gender of participants by proportion 70 59

Relative Frequencies (%)

60 50 42 40

35

30 20

Male Female

23 15 10

7

10

8

0 16-18

19-24

25-55

55+

N = 3127

NEETs is an acronym to define young people aged between 16-24 years who are not in education, employment or training. The graph below shows that in Engagement Gateway NEETs account for 44% (1386/3149) of participants. This concurs with the graph above, which shows high participation rates in those born in the 1990s.

Graph 9: Age and gender of participants by proportion 60%

56% 44%

50% 40% 30% 20% 10% 0% NEET No

Yes

N = 3149

Š Copyright Wavehill Ltd 2012

25

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

Twenty-one per cent of participants are from an ethnic heritage which is significantly higher than for Wales based on the 2010 census which is 4.5% (nomis; economic active proportions).

Graph 10: Ethnicity of participants 100% 79% 80% 60% 40% 21% 20% 0% No

Yes BAME Heritage

N = 3149

4.4.

Employability and employment outcomes

4.4.1 Introduction The impact or benefit of participation on the Engagement Gateway participants is not even. The section below reviews the employability and employment outcomes.

Employment is recorded as finding work of greater than 16 hours a week. Employability outcomes include gaining volunteering for work experience, qualifications, entering further training, but not ‘other’ or soft outcomes.

In the analysis below we have created four categories, to explore how outcomes are achieved across different groups.

1. Employment, 2. Employability, 3. Soft or other outcomes, 4. None recorded.

© Copyright Wavehill Ltd 2012

26

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

These categories are shown below in a contingency table against participant characteristics. Each category is then represented as a graph for clarity.

Table 7: Contingency table showing employment, employability, soft and no outcomes

Total outcomes recorded Male Female Economically Inactive Unemployed more than 1 year Unemployed less than 1 year NEET Yes NEET No People > 50 People < 50 BAME Yes BAME No Lone Parent – Yes Lone Parent – No Disabled – Yes Disabled – No Work Limiting Health – Yes Work Limiting Health – No

4.4.2

Employment outcomes 327 11% 9% 7% 7% 17% 10% 11% 12% 10% 10% 10% 7% 10% 3% 11% 5% 11%

Employability outcomes 842 58% 59% 62% 58% 52% 59% 58% 67% 58% 60% 58% 58% 59% 69% 57% 70% 56%

Soft outcomes 2644 84% 84% 86% 83% 81% 83% 85% 90% 83% 85% 84% 80% 85% 92% 83% 88% 83%

No outcomes 503 16% 16% 14% 17% 19% 17% 15% 10% 17% 15% 16% 20% 15% 8% 17% 12% 17%

Outcomes by scale

The following chart is used to give some context to the section below. It compares the total positive outcomes from the employment, employability, soft and no outcomes categories in one graph. It is also worth noting that there are multiple selections as the cumulative outcomes recorded (N=4836) is greater than for the Engagement Gateway RCE project.

Total outcomes recorded

Graph 11: Graph shows the scale of recorded outcomes

327 842 2644 503 0

500

Employment outcomes

1000

1500

Employability outcomes

2000 Soft outcomes

2500

3000

No outcomes

N=4836

© Copyright Wavehill Ltd 2012

27

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

4.4.3

Employment outcomes

The graph below shows the outcome categories for the employment variable. Being NEET, an older person, of BAME ethnic heritage, and male or female, or a lone parent, has little or no determination over gaining employment. However, unsurprisingly having a work limiting illness reduces outcomes by half (11% to 5%), and being disabled reduces employment outcomes by nearly four times (11% to 3%). The strongest predictor of employment outcomes is those having worked within 1 year of joining Engagement Gateway, longer term employed people have less than half the chance of employment outcomes (7% vs. 17%). Similar observations were also made in the evaluation of the WCVA ILM project.

Graph 12: Graph showing employment outcomes by participant characteristics Employment outcomes Work Limiting Health - No Work Limiting Health - Yes Disabled - No Disabled - Yes Lone Parent - No Lone Parent - Yes BAME No BAME Yes People < 50 People > 50 NEET No NEET Yes Unemployed less than 1 year Unemployed more than 1 year Economically Inactive Female Male

11% 5% 11% 3% 10% 7% 10% 10% 10% 12% 11% 10% 17% 7% 7% 9% 11% 0%

2%

4%

6%

8%

10%

12%

14%

16%

18%

N = 3147

Š Copyright Wavehill Ltd 2012

28

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

4.4.4

Employability outcomes

The graph below shows the outcome categories for the employability variable. It appears that people with work limiting illnesses and disability, and people over the age of 50 years are gaining employability outcomes but finding it harder to convert these skills into employment.

Graph 13: Graph showing employability outcomes by participant characteristics Employability outcomes Work Limiting Health - No Work Limiting Health - Yes Disabled - No Disabled - Yes Lone Parent - No Lone Parent - Yes BAME No BAME Yes People < 50 People > 50 NEET No NEET Yes Unemployed less than 1 year Unemployed more than 1 year Economically Inactive Female Male

56% 70% 57% 69% 59% 58% 58% 60% 58% 67% 58% 59% 52% 58% 62% 59% 58% 0%

10%

20%

30%

40%

50%

60%

70%

80%

N=3147

Š Copyright Wavehill Ltd 2012

29

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

4.4.5

Soft or ‘other’ outcomes

The graph below shows the outcome categories for the soft or other outcomes variable. Most participants have recorded soft outcomes and the data shows that benefit is evenly distributed across all of the participant characteristics.

Graph 14: Graph showing employability outcomes by participant characteristics Soft outcomes Work Limiting Health - No

83%

Work Limiting Health - Yes

88%

Disabled - No

83%

Disabled - Yes

92%

Lone Parent - No

85%

Lone Parent - Yes

80%

BAME No

84%

BAME Yes

85%

People < 50

83%

People > 50

90%

NEET No NEET Yes

85% 83%

Unemployed less than 1 year

81%

Unemployed more than 1 year

83%

Economically Inactive

86%

Female

84%

Male

84%

70%

75%

80%

85%

90%

95%

N=3147

© Copyright Wavehill Ltd 2012

30

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

4.4.6

No recorded outcomes

The graph below shows the outcome categories for no outcomes variable. There is not actually such a variable in the Engagement Gateway data system, but this has been created by taking out other outcomes and finding what is left. So this may represent poor monitoring data, rather than no outcomes. However, being disabled halves the rate of no outcomes (17% > 8%), and being a lone parent increases the rate to 20%.

Graph 15: Graph showing no outcomes by participant characteristics No outcomes Work Limiting Health - No Work Limiting Health - Yes Disabled - No Disabled - Yes Lone Parent - No Lone Parent - Yes BAME No BAME Yes People < 50 People > 50 NEET No NEET Yes Unemployed less than 1 year Unemployed more than 1 year Economically Inactive Female Male

17% 12% 17% 8% 15% 20% 16% 15% 17% 10% 15% 17% 19% 17% 14% 16% 16% 0%

5%

10%

15%

20%

25%

N=3147

Š Copyright Wavehill Ltd 2012

31

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

4.5.

Analysis by local authority

The analysis below shows the number of participants by local authority area. Overall, the Engagement Gateway RCE project has reached 41% (3142/7740) of its target for participation. The graph shows that Cardiff has contributed approximately half of Engagement Gateway participants.

However, Cardiff has a much larger population and this is expected.

Graph 16: Participants in Engagement Gateway by local authority 1600 1400 1200 1000 800 600 400 200 0

1510

502

418 174

113

156

227

number of participants N = 3100

The next analysis shows the proportion of participants compared to the proportion of those economically inactive but wanting to work per area. This takes into consideration the differences in population between say Cardiff and Powys, for example, which the above absolute graph does not.

The blue bars are the difference between the number of participants and the population proportion.

In Cardiff (49%) proportionately more participants have participated in the Engagement Gateway project than in Powys (5%). The blue bar compares the proportions and shows that Cardiff (27%) has proportionately worked with more participants than Powys (-17%) for example.

Š Copyright Wavehill Ltd 2012

32

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

One reason for lower than anticipated participation may be the rurality of areas5. This analysis (as above) suggests that the more rural areas attract proportionately less participants than the more urban areas of Cardiff, Newport and Wrexham.

Graph 17: 60% 50% 40% 30% 20% 10% 0% -10% -20% -30%

49% 27% 22%

29%

26%

22%

22% 13%

6%

5%

4%

Proportion

-23%

22% 16%

-6%

-9% -23%

21% 7%

-17%

% of economically inactive who want a job

-14%

Difference

N = 3100

4.5.1 Local authority by positive outcomes Positive outcomes are outcomes such as creating a CV, attending an interview, or growing in confidence. These are sometimes called soft outcomes, and are important in measuring the journey made by participants from a long distance from the labour market, to closer to the labour market.

Understanding how to present oneself, search for a job, apply for interviews, attend interviews and communicate effectively are all valid elements of employability.

The Engagement Gateway RCE project has achieved 49% (2640/5418) of its contracted target. The graph below shows that Cardiff and Wrexham have created disproportionately more â&#x20AC;&#x2DC;Other Outcomesâ&#x20AC;&#x2122; than in other LA areas. It may be easier to source job interviews, undertake relevant job search and attend interview practice sessions and so on within an urban setting rather than a rural one.

5

http://www.walesruralobservatory.org.uk/reports/english/statistical%20report1.pdf

Š Copyright Wavehill Ltd 2012

33

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

Graph 18: Other outcomes adjusted to reflect economic inactive want to work proportions 20%

17%

15% 10%

7%

5%

2%

0% -2%

-3%

-5%

-1%

-7%

-10% Cardiff

Flintshire

Monmouthshire

Newport

Powys

Vale of Glamorgan

Wrexham

N = 2693

The ESF Leavers Survey is used to compare the outcomes of ESF leavers to ESF funded projects, and is useful as a comparator for evaluation reports. In this report we will compare each indicator as below.

ESF Leavers 2010: Other Outcomes = 91% - Positive or other outcomes are described by WCVA in their guidance to projects as subjective measures of personal development across four domains (personal, social, health, practical). Projects could record positive or other outcomes if participants demonstrated progress within these areas.

Personal achievement: e.g. in decision making, personal care, management of finances, reduced offending, improved earning potential; Social achievement: e.g. improved family relationships, improving social networks, anger management; Health achievement: e.g. weight loss, pain management, drug and alcohol reduction; and Practical achievement: e.g. stable accommodation, ability to use public transport Engagement Gateway RCE = 84% (2640/3142)

The ESF and the Engagement Gateway projects are comparable in the number of participants who have achieved other outcomesâ&#x20AC;&#x2122;. It is worth noting that the methods of collecting the data differ, as the ESF Survey is answered directly by the ESF leaver, whereas the Engagement Gateway data is completed by Project Managers.

Š Copyright Wavehill Ltd 2012

34

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

4.5.2

Local authority by qualifications

Qualifications are indications that people have reached a certain level of learning and understanding and can communicate this learning within a test environment. As there are so many types and categories of qualification, we adjusted these to a simple yes for all qualifications at NVQ level 2 and above, and no for all those at below NVQ Level 2, those without any qualification outcomes, and those who took modules but without reaching the qualification stage.

Graph 19: Qualifications gained adjusted to reflect economic inactive want to work proportions 20% 15%

15%

10% 5% 5%

3% 0%

0% -2%

-5%

-2%

Yes Qualification Adjusted

-6% -10% Cardiff

Flintshire

Monmouthshire

Newport

Powys

Vale of Glamorgan

Wrexham

N=1172

In the RCE Engagement Gateway project 1172 participants gained a qualification at or above NVQ Level 2 which is 61% (1172/1935) of the targets agreed between WCVA and WEFO for this project.

Comparison to the ESF Leavers Survey shows that the Engagement Gateway RCE project has allowed proportionally far more participants to gain NVQ Level 2 (or above) qualifications than for the ESF Survey. This is a significant increase over the ESF Survey and shows some high qualify outcomes. As shown in the table above the outperformance within this category came from the LA areas of Cardiff, Newport and Wrexham. This may be due to the proximity of participants to FE and HE institutes and the relative ease of access to learning and training providers.

ESF Leavers 2010: Qualifications at NVQ 2 minimum = 16% Engagement Gateway Qualifications at NVQ 2 minimum = 37% (1172/3142)

Š Copyright Wavehill Ltd 2012

35

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

4.5.3

Local authority by volunteering

Volunteering enables people who have never worked, or have been out of work for a long period to build an understanding of the work environment, experience working in teams and workplace hierarchies, and to build office and work skills too. Also the routine of attending volunteering helps reinforce work behaviours. Volunteering often helps potential employees to understand more about applicants, and allows applicants to demonstrate experience on their CVs. The graph below shows that volunteering experience is distributed relatively more proportionally when adjusted for population than participation or qualifications. It is possible that volunteering opportunities or locations are more distributed and so is easier to access for those in rural communities. In the graph below Powys stands out as being disproportionately less than the other local authorities, but the difference is less than 10% and so is not significantly large. Across the RCE Engagement Gateway project 15% (479/3142) of participants experience volunteering.

Graph 20: Volunteering experience adjusted to reflect economic inactive want to work proportions 6%

5%

4%

4% 2%

0.5% -0.1%

0%

-1.0%

-0.6%

-2% -4% -6% -8%

Volunteering Population Adjusted -8%

-10% Cardiff

Flintshire

Monmouthshire

Newport

Powys

Vale of Glamorgan

Wrexham

N = 479

No specific target was set for volunteering and there is no comparison in the ESF Leavers Survey for this category of benefit.

Š Copyright Wavehill Ltd 2012

36

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

4.5.4

Local authority by entering further learning

Entering further learning enables participants to develop academic and vocational skills that enhance their opportunity for employment.

In terms of entering further learning Cardiff has proportionately the most outcomes when population adjusted, and Powys the lowest. As for qualifications, it may be that rurality (often called social exclusion) is a factor in participants choosing to enter further learning.

Graph 21: Entering further learning adjusted to reflect economic inactive want to work proportions 20% 15%

14%

10% 2%

5% 0% -5%

-1%

-3%

-1%

-10% Cardiff

Flintshire

Monmouthshire

Newport

-9%

Powys

Vale of Glamorgan

-3% Wrexham

Enter Further Learning Adjusted N = 492

In terms of targets, the original WEFO target was 1935 and the RCE Engagement Gateway project has reached 25% of this and a rate of 16% (492/3142) within their cohort. The ESF Leavers Survey does not create a category for entering further learning, but measures qualifications achieved or gained.

Š Copyright Wavehill Ltd 2012

37

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

4.5.5

Local authority by entering employment

The main target of RCE Priority One is ‘Increasing employment and tackling economic inactivity’.

Clearly project participants who move into employment are priority outcomes. The graph below shows that when adjusted for population size, Cardiff produced the most employment and Powys the least.

Graph 22: Entering employment adjusted to reflect economic inactive want to work proportions 20% 15%

14%

10% 2%

5% 0% -5%

-1%

-3%

-1%

-10% Cardiff

Flintshire

Monmouthshire

Newport

-9%

Powys

Vale of Glamorgan

-3% Wrexham

Enter Further Learning Adjusted N = 329

The target set by WEFO was for 774 jobs, a 10% target and the project achieved 329 jobs, by proportion also exactly 10% although only 43% of the number of employment outcomes. This shows that the project was effective in its employment target, however, the number of participants and therefore employment outcomes was low, compared to the business plan forecast.

However it could be said that this was an ambitious target and that the Engagement Gateway RCE performed well. The ESF Leavers Survey notes that 39% of their survey were in employment at the time of survey, which compares poorly to our 10%. However, the recent employment rate published by the Department for Work and Pensions (DWP) shows that around 2.3% of individuals were employed.

This is a large range of employment rate outcomes and the Engagement Gateway RCE falls between the two and so it is hard to state if the Engagement Gateway RCE employment rate is good or not.

ESF Leavers 2010: Entering Employment = 39% Engagement Gateway Employment Rate = 10% (329/3142) Work Programme Employment Rate = 2.3%

© Copyright Wavehill Ltd 2012

38

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

5. Conclusions and Recommendations The evaluation report finds that the Engagement Gateway project in the RCE area has had some wide variance in the participation rates when considered by local authority area.

The more rural LAs show proportionately lower levels of ‘engagement’ and therefore participation and the more urban better participation and benefit. Other factors that CVCs claim to have affected engagement include, PQQ and tender submission timings, changes to eligibility to ESF projects, and reluctance of providers to operate under EC/ESF administrative rules.

Recommendation 1: One potentially serious issue (for WEFO et al) that was raised by CVCs was the culture of providers being ‘scared’ of entering into EC/ESF grants and contracts with informal discussions about contracts being withdrawn and handed back affecting provider confidence. Also, there was some narrative evidence of contractors being dissuaded by the administrative overhead required to manage ESF contracts.

If Welsh training providers are to deliver ESF training provision in future, it may be necessary for WEFO to consult with the sector to ensure their full participation.

Once participants are in the Engagement Gateway however, benefit and results are more evenly distributed, however the more urban areas, notably Cardiff and Wrexham still disproportionately outperform their peers in terms of both participation and benefits.

The evaluation brief asked that the causes of participation and benefit variance should be examined and the analysis above has reviewed participation and benefits by local authority area. Our findings do not point to any clear reason, except the effect of social exclusion and access to services when one compares rural areas to more urban areas. Undertaking employability and skills development in rural areas, with poor public transport links is clearly more difficult. Individuals may find travel too costly, complex or far to attend sessions hosted by providers. Unfortunately, we did not undertake a direct dialogue or survey with participants and so these questions remain unanswered. Variance in performance is most likely therefore a combination of rural exclusion, CVC (and staff) differences, and some systemic issues due to tendering procedures and EC/ESF administration requirements.

Recommendation 2: In rural areas with exclusion issues, support and training could be enhanced by using the telephone, especially for job interview / communication / office training scenarios, where the telephone will often be the primary communication method. Future providers may wish to include a training design element that includes telephone / remote training.

© Copyright Wavehill Ltd 2012

39

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

In terms of outcomes those participants who had been unemployed for less than a year were more than twice as likely to find employment (7% vs 17%). This shows the importance of working with people who have been recently unemployed to keep them as close to the employment market as possible.

For those who are longer term unemployed, volunteering and access to work via supported (Intermediate Labour Market) jobs may be necessary. These pathways to employment should be given greater focus for longer term unemployed groups.

Recommendation 3: Longer term unemployed people are approximately 2.5 times less likely to find employment than those unemployed for less than one year. ILM models may be a better pathway choice for the longer term unemployed as they provide skills and experience that may be necessary to gain employment outcomes.

In previous evaluation studies we have noted that for those people who have never worked, or have been a long time away from work, confidence building, communications skills, team building and employment skills are required to build the confidence of individuals and prepare them psychologically for work, or the search for work. These â&#x20AC;&#x2DC;softâ&#x20AC;&#x2122; outcomes are important for individuals to transition from a low likelihood of finding employment, to a greater likelihood. Soft outcomes need to be measured alongside hard outcomes in more detail, especially as many categories of assistance are included under the label, and it is important to identify which are most effective in moving participants on into employability, and employment outcomes.

Recommendation 4: Soft outcome tracking tools should be designed into future projects from the start, and the categories of soft or other outcomes classified and defined so that the most effective types can be identified and enhanced. There are many soft outcome tools already in the market, but they do require some discipline in their use and frequency of use. (see Triangle Consulting for an example named Work Star.)

Recommendation 5: The WEFO reporting system for ESF projects could be improved by presenting back reported data in a similar style to this report. A comparison of the participant types or categories that benefit most (or least) from participation would be useful for project managers. Without reviewing the data in this way it is difficult to find that disabled people, or long term unemployed people, benefit less from finding employment, for example.

Š Copyright Wavehill Ltd 2012

40

Final Evaluation of the Engagement Gateway Regional Competiveness and Employment project 2008 - 2012

In conclusion, the Engagement Gateway has performed at a lower rate of participation than for the Convergence element of the programme. In terms of employment, the Engagement Gateway has delivered more employment outcomes (proportionately) than the Work Programme and with a harder to employ cohort, and on this measure it has been successful. However, the employment and employability outcomes are lower than for the ESF Leavers Survey 2010, and so the project has not outperformed either in the Convergence area or against its comparator against all ESF projects.

Š Copyright Wavehill Ltd 2012

41