Causality Analysis: Agreeing on the problem and causes

Causality Analysis

Agreeing on the problem and causes

7/1/2009

1

UNICEF - Evaluation Office and PGQA Section

Children in poor households are more frequently malnourished, and have more learning difficulties when entering school

7/1/2009

2

UNICEF - Evaluation Office and PGQA Section

Children in poor households are more frequently malnourished, and have more learning difficulties when entering school

No knowledge of health system support, low utilisation

7/1/2009

3

UNICEF - Evaluation Office and PGQA Section

Children in poor households are more frequently malnourished, and have more learning difficulties when entering school

No knowledge of health system support, low utilisation

7/1/2009

4

UNICEF - Evaluation Office and PGQA Section

Children looked after by people who are poorly equipped to do so

Children in poor households are more frequently malnourished, and have more learning difficulties when entering school

No knowledge of health system support, low utilisation

Lack of knowledge of adequate child care practices

Existence of harmful traditional practices

7/1/2009

5

UNICEF - Evaluation Office and PGQA Section

Children looked after by people who are poorly equipped to do so

Children in poor households are more frequently malnourished, and have more learning difficulties when entering school

No knowledge of health system support, low utilisation

Children looked after by people who are poorly equipped to do so

Single parenting by poor mothers Lack of knowledge of adequate child care practices

Father’s absence accepted as norm

Existence of harmful traditional practices No practice of preparing expectant parents for parenthood

One parent parent died

Orphanhood

7/1/2009

6

UNICEF - Evaluation Office and PGQA Section

Children in poor households are more frequently malnourished, and have more learning difficulties when Children looked No knowledge of entering schoolafter by people health system who are poorly support, low equipped to do utilisation so Single Lack of parenting by knowledge of poor mothers adequate child care practicesof Father’s Existence absence harmful accepted as No practice oftraditional One parent norm preparing practices parent died expectant parents for Orphanhood parenthood

7/1/2009

7

UNICEF - Evaluation Office and PGQA Section

7/1/2009

8

UNICEF - Evaluation Office and PGQA Section

Causality Analysis

Create a Causal Tree

Use a Conceptual Framework k As

Validate Causal Analysis

Use for making strategic choice

7/1/2009

9

UNICEF - Evaluation Office and PGQA Section

s

s r e d l o h e k ta

!

Easier To Resolve

More Sustainability

7/1/2009

10

UNICEF - Evaluation Office and PGQA Section

Causality Analysis

Create a Causal Tree

Use a Conceptual Framework

ice Po o

Use for making strategic choice

ho

od o G

ce i o Ch rC

Validate Causal Analysis

7/1/2009

11

UNICEF - Evaluation Office and PGQA Section

Logframes

Detailed Programme Design

7/1/2009

12

UNICEF - Evaluation Office and PGQA Section

Logical Aproach

Detailed Programme Design

7/1/2009

13

UNICEF - Evaluation Office and PGQA Section

Agenda

7/1/2009

Purpose and definition of the logical approach

Elements of a logical approach and template (Logframe)

How to manage critical assumptions

14

UNICEF - Evaluation Office and PGQA Section

Purpose and Definition

7/1/2009

Improve logic of a programme or project model

Ensure that interventions are sufficient to make the results chain work

Minimize influence of critical planning assumptions

Increase the likelihood of achieving the intended results.

Monitoring indicators and strategic evaluation questions

15

UNICEF - Evaluation Office and PGQA Section

Logical Approach

7/1/2009

16

UNICEF - Evaluation Office and PGQA Section

Used for:

CPR - Results Framework

MPO: PPO structure

PPO: project/subproject structure

for Project: sub-projects/activities

for PPA (annual projects)

for stand-alone donor proposal

for any other planning exercise

What is Logframe?

A Logical Approach is often expressed in form of a matrix (Logframe):

the programme Results Chains (Activities; Outputs; Outcomes and; Strategic Results) and critical assumptions. “Vertical Logic”

(b)

programme performance assessment elements: Indicators; baselines; Targets; Means of Verification; data disaggregation, critical assumption “Horizontal Logic”

Logframes visualize programme design and allow to assess the quality of programme design at a glance

7/1/2009

(a)

17

UNICEF - Evaluation Office and PGQA Section

A Logical Framework Programme: Programme Objective or longer term outcome/impact:

Key Indicators:

Data Sources:

Assumptions (concerning Risks, External Factors):

Key Indicators:

Data Sources:

Assumptions (concerning Risks, External Factors):

Project 1: [Title of Project] Project 1 Objective/Outcome:

Course of Action: (main interventions, major anticipated activities, responsible institutions and type of needed inputs) Project 2: [Title of Project] Project 1 Objective/Outcome:

Key Indicators:

Data Sources:

Assumptions (concerning Risks, External Factors):

Course of Action: (main interventions, major anticipated activities, responsible institutions and type of needed inputs)

7/1/2009

18

UNICEF - Evaluation Office and PGQA Section

Overview of a Logframe Template and Content Levels

#

Indicators

1.

Strategic Result Statement

1.1.

Outcome statement

1.1.1.

Output Statement

1.2.1 .

7/1/2009

Assumption specific to Results Chain #1

Outcomes (as defined in RF)

Output Statement

Outputs (as defined in RF) Activities

Outcome Statement

Critical Assumption

Output Statement

Output Statement

19

Geogr Focus

Strategic Results (as defined in RF)

List Activities -

1.2.2 .

MOVs

Vertical Logic

List Activities -

1.2 .

Targets

Assumption

List Activities -

1.1.2.

Baselines

UNICEF - Evaluation Office and PGQA Section

Assumption specific to Results Chain #2

Overview of a Logframe Template and Content Levels

#

Indicators

Baselines

Targets

MOVs

Geogr Focus

1.

Strategic Result Statement

Assumption

1.1.

Outcome statement

Assumption specific to Results Chain #1

1.1.1.

Horizontal Logic Output Statement

Indicators

List Activities -

1.1.2.

Baselines

Output Statement

Targets

List Activities -

1.2 .

1.2.1 .

Means of Verification (Data Sources)

Outcome Statement Output Statement

Critical Assumptions

List Activities -

1.2.2 .

7/1/2009

Geographic Scope

Output Statement

20

UNICEF - Evaluation Office and PGQA Section

Assumption specific to Results Chain #2

However:

The value of the logical approach is not in filling the blanks in a table….

……but in reviewing and revising the interventions, until they are likely to achieve the intended result

7/1/2009

21

UNICEF - Evaluation Office and PGQA Section

„ Several iterations are usually required to reformulate the project, until critical assumptions or risks are minimised „Indicators need to including those that check whether the critical assumptions are still valid 7/1/2009

22

UNICEF - Evaluation Office and PGQA Section

Managing Assumption Achievement of Results X depends on 5 intermediate results

7/1/2009

23

UNICEF - Evaluation Office and PGQA Section

Result X5 Intermediate Result A, B, C

Will be done by UNICEF and Gov.

Intermediate Result D

Will be done by other donor and Gov.

Assumption

7/1/2009

24

UNICEF - Evaluation Office and PGQA Section

Result X5

Assumption can be Resultby the Will be done by fullyIntermediate controlled UNICEF and Gov. A, B, C project.

Intermediate Result Will be done by other Remove assumptions donor and Gov. D Assumption Regular Supplies ensured Training manual developed Quality of study design ok

7/1/2009

25

UNICEF - Evaluation Office and PGQA Section

Result X5

Irrelevant Intermediate Result Assumptions A, B, C

Will be done by UNICEF and Gov.

Intermediate Result Will be done by other Remove Assumptions donor and Gov. D Assumption Political Stability No drought No disaster Stable economy No war in the region 7/1/2009

26

UNICEF - Evaluation Office and PGQA Section

Assumption should have been Result clarified X5 earlier.

Intermediate Result A, B, C Intermediate Result Survey/Research D before proceeding Assumption with the rest

Will be done by UNICEF and Gov. Will be done by other donor and Gov.

Community considers xxx a high priority Politicians and officials believe in gender equality Private sector will cooperate

7/1/2009

27

UNICEF - Evaluation Office and PGQA Section

Assumption should5be Result X clarified.

Intermediate Result A, B, C

Will be done by UNICEF and Gov.

Consider a pilot Intermediate Result Will be done by other project before scaling donor and Gov. D up Assumption

Teachers will apply what they learned Caretakers will bring child to facility Children will use latrines

7/1/2009

28

UNICEF - Evaluation Office and PGQA Section

Assumption can be Result X5 managed to a large Intermediate Result extend.

A, B, C

Will be done by UNICEF and Gov.

Intermediate Result Will be done by other Modify/add D activities donor and Gov.

and budgets Assumption

Monitoring system in place Use of guidelines enforced Good co-ordination between partners 7/1/2009

29

UNICEF - Evaluation Office and PGQA Section

Serious Assumptions, Result X5 but can be influenced. Will be done by Intermediate Result A, B, C

UNICEF and Gov.

Intermediate Resultdesign Will be done by other Modify project donor and Gov. D or objectives Assumption

Favourable policy environment Political commitment Funds available

7/1/2009

30

Demand can be mobilised UNICEF - Evaluation Office and PGQA Section

Result X5

Killer Assumptions!

Intermediate Result A, B, C

Will be done by UNICEF and Gov.

Re-think or abandon Intermediate Result Will be done by other project/programme donor and Gov. D Assumption

Improved services are matched by social movement Attitude change required Decentralisation process completed successfully

7/1/2009

31

UNICEF - Evaluation Office and PGQA Section

Result X5

Killer Assumptions!

Intermediate Result A, B, C

Will be done by UNICEF and Gov.

Re-think or abandon Intermediate Result Will be done by other project/programme donor and Gov. D Assumption The president will… + sustained highest level… + WorldBank grant … + skilled staff everywhere... + all other things in place… + etc etc...

7/1/2009

32

UNICEF - Evaluation Office and PGQA Section