II. PORTUGAL
Portugal Evaluation and Audit Unit (Gabinete de Avaliação e Auditoria - GAA), Institute for Cooperation and Language (Camões, I.P.) Evaluation Mandate In 2012 Portugal underwent a major organisational reform. The Portuguese development agency, the Portuguese Institute for Development Support (IPAD), was merged with into Camões, the Institute for Cooperation and Language (Camões, I.P.). Since the merger, Camões, I.P. is responsible for both development co-operation and Portuguese language and cultural activities. Camões, I.P. has a unit responsible for the evaluation of Portuguese development cooperation and Portuguese language and cultural promotion activities, the Evaluation and Audit Unit (Gabinete de Avaliação e Auditoria - GAA). GAA is in charge of the evaluation of development co-operation, delivered directly and through line ministries, as well as internal audit. The role and institutional setup of GAA are guided by the Evaluation Policy currently under development. The new Evaluation Policy grants GAA the mandate to evaluate all development co-operation interventions and to ensure quality of evaluations conducted by line ministries. Through evaluation, GAA aims to deepen the knowledge about the implementation of programmes and projects and their results, to support the evidencebased decision making and to promote the continuous improvement in the areas of intervention of Camões, I.P.
Responsibility and the scope of activities The evaluation of Portuguese development co-operation is centralised. There is no decentralised evaluation of development co-operation in Portugal. GAA’s duties include: •
managing and implementing internal evaluations
•
promoting, managing and monitoring external evaluations
•
collaborating in joint evaluations
•
dissemination of evaluation results and information
•
ensuring the incorporation of lessons learned into the future actions
•
ensuring the quality of the evaluations
•
providing technical support related to evaluability.
Camões, I.P. delegates some responsibilities, such as the support function for project monitoring, to the staff based in Portugal’s six partner countries, but they are not responsible for evaluation.
Organisational Structure and Reporting Lines GAA has a direct reporting line to the Board of Camões, I.P. Camões, I.P. further reports to two Secretaries of State within the Ministry of Foreign Affairs. GAA drafts a triennial work plan in collaboration with the operational units and policy makers, so that the demands to be placed on evaluation subjects and the information gap for decision making are addressed. The work plan is subsequently submitted to the Board of Camões, I.P. for approval.
EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW © OECD 2016
183
II. PORTUGAL
Central/main evaluation units
Programme/operational units Reporting line
High level policy groups or ministries
Lines of communication
Ministry of Foreign Affairs Secretaries of State
Institute for Cooperation and Language (Camões, I.P.) Board of Camões, I.P.
Evaluation and Audit Unit (Gabinete de Avaliação e Auditoria - GAA)
External Consultants / consultant database
Types of Evaluation • Thematic evaluations • Programme evaluations • Country evaluations • Project/activity-level evaluations
GAA applies the OECD DAC criteria for their evaluations, as well as the 3C principles (co-ordination, complementarity and coherence), and added value and visibility. During the past five years, 9 ex-post evaluations, 6 process evaluations, 1 evaluability assessment and other type of evaluation were conducted. GAA plans to invest in impact evaluations in order to better assess the projects’ impact/outcome-oriented results rather than outputs realised. Fifteen development-related Snapshot of evaluation resources evaluations and 5 language-related evaluations Portugal will be conducted during the 2014-2016 period.
Resources The number of evaluation staff has increased, as the GAA’s workload has also expanded as a result of the merger. The evaluation staff in the former IPAD was composed of only two or three however GAA is currently staffed by six full-time employees, five of which engage in evaluation activities.
Head / Director / Assistant Director
Professional evaluation staff EUR 100 000 0.085% of the ODA budget 4
Average evaluations produced per year
184
II. PORTUGAL
Camões, I.P. has separate budgets for the different organisation units. The 2014-2016 evaluation plan does not define the budget for evaluations, but the budget attached to evaluations in 2015 is EUR 100 000 excluding salaries.
Principles of Evaluation Independence The level of structural independence is considered to be strong. The Head of GAA reports directly to the Board of Camões, I.P. and GAA is independent of the operations units.
Competence and capacity building Competence of the evaluation staff is addressed by training and also complemented by hiring external consultants and local experts in the partner countries. As a means of reinforcing the human resource and making best use of the available expertise in the country, Camões, I.P. has built a database of evaluators. Although the registration in this roster is non-binding, interested individuals and collective entities with competences to evaluate the interventions in the areas of Portuguese language, culture and co-operation are registered.
Transparency and participation Final evaluation reports are made publicly available only after a discussion with all relevant stakeholders. The relevant operations units fill in the fiche contradictoire identifying the actions to be taken in order to meet the recommendations. The follow-up actions are later defined and implementation of the accepted recommendations monitored. GAA also publishes an annual report, which highlights progress made on implementing the recommendations provided and identifies lessons learned. Final evaluation reports and other key documents such as evaluation policy and annual reports are made available on the website mostly in Portuguese, which is the official language of the country’s development co-operation.
Knowledge sharing Portugal is aware of the need for investing in a systematic knowledge sharing mechanism that promotes informed decision making. Portugal started to develop an integrated information system in 2013-2014 to follow the Busan commitments. The new system would enable online data input through an electronic platform that would automatically validate data, provide the facility for online consultation and downloading of statistical data.
Co-ordination with donors and country recipients Portugal has conducted six joint evaluations during the past five years. All of them have been with the Ministry of Foreign Affairs of the partner countries (Angola, Cape Verde, Mozambique, Timor Leste and S. Tome and Principe). Camões, I.P. makes efforts to involve country recipients in the evaluation cycle. The partner country embassies in Lisbon and the other stakeholders in the field can be involved, when drafting ToR. They are also invited to participate in the project implementation on the ground.
EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW © OECD 2016
185
II. PORTUGAL
Quality assurance Quality assurance of the evaluation reports is part of GAA’s responsibilities. GAA provides technical support in evaluation matters to the operations units and line ministries. To ensure the quality of evaluations, Camões, I.P. also appoints a management group for each evaluation and a matrix for assessing the quality is created (OECD 2016).
Note to reader: The section at the beginning of Part II entitled “Introduction and key for the member profiles” provides explanatory notes on the profiles.
186