PAHO Course Report Aug07 EPPES

‘Effective project planning and evaluation in health research’ TDR skill-building course PAHO, Washington DC, USA 13-16 Aug 07 Room 612

COURSE REPORT 17 Aug 2007

Trainers: Lyda Osorio & Beatrice Halpaap

Course report – Aug 07 Page 1 of 12

REPORT CONTENT Report summary List of participants Agenda Project summaries Identification of potential future trainers Lesson learnt Course assessment Acknowledgements Annex 1 – Project documentation developed during the course (PowerPoint file) Annex 2 – Project documentation revised after the course (to be added later)

Course report – Aug 07 Page 2 of 12

REPORT SUMMARY This course was run within the unit of Research Promotion and Development in the Health System Strengthening area at PAHO in Washington DC. It was organized and sponsored in collaboration among PAHO, CIDEIM and TDR. The course had 2 main objectives: (1) to enhance project management skills of PAHO staff within the unit and (2) to introduce the course to PAHO before they make any decision in and explores way of supporting the dissemination of the course in Latin America The course was facilitated by Lyda Osorio (new trainer from CIDEIM) and Beatrice Halpaap. Due to unexpected events the organization of the course was challenging.

The start of the course was delayed by half day and its length was reduced to 3 days.

However, in despite of this, the course went well. Participants went successfully through all steps and applied them to their own projects. The atmosphere was found conducive. Participants were very receptive and committed. They found the skills they learnt helpful and relevant to their daily work and team. Participants work mainly on research management and administrative projects. They, therefore, recommended that an adapted example be developed and used for such audience. Lessons learnt from this course have been compiled and presented in this report in order to be shared with other trainers.

Course report – Aug 07 Page 3 of 12

LIST OF PARTICIPANTS Trainers Lyda Osorio CIDEIM Av 1N 303 Cali, Colombia Tel +572 668 21 64, lydaosorio@cideim.org.co Beatrice Halpaap Global Health Division, Yale School of Public Health 60 College st, New Haven 06510, USA Tel +1 203 479 25 96, halpaapb@who.int; beatrice.halpaap@yale.edu

PAHO participants Name Luis Gabriel Cuervo

Group project

Email cuervolu@paho.org

Jaume Canela

Implementation of the research register Implementation of the research register PAHO ethics

Tania Pereyra

PAHO ethics

pereyrat@paho.org

Linda Pereyra

Transition process for collaborating centers Transition process for collaborating centers Patience safety information

pereiral@paho.org

Liliana Silva

Manuela Villar Babak Mohit

Course report – Aug 07 Page 4 of 12

silvalil@paho.org canelaza@paho.org

villarma@paho.org mohitbab@paho.org

AGENDA Starting time: Coffee breaks (AM): Lunch breaks: Coffee breaks (PM): End of the day:

09:00 daily except Monday (first day): 13:30 10.30 – 10.45 12.30 - 13.30 15.00 - 15.15 17.30/18:00

As the course is using a ‘learning by doing’ approach based on a step-by-step process, full attendance from all participants is a prerequisite for its success.

Monday 13 Aug 2007 Pm - 13.30 1. Welcome, Introduction and Overview of the course 2. Module 1: Good Practices in Biomedical Research 3. Module 2: The Concept of Planning in Health Research

Tuesday 14 Aug 2007 Am – 09.00 4. Module 3: Defining the Purpose and the Scope of the Project 5. Module 3 – Case studies Pm - 13.30 6. Module 3 – Case studies

Wednesday 15 Aug 07 Am – 09.00 7. Module 4: Establishing the Project Development Plan 8. Module 4 – Case studies Pm - 13.30 9. Module 4 – Case studies

Thursday 16 Aug 07 Am – 09.00 10. Module 5: Implementing, Monitoring, Evaluating & Reporting 11. Module 5 - Case studies Pm - 13.30 12. Course assessment − Group discussion − Questionnaires 13. Next steps

Course report – Aug 07 Page 5 of 12

PROJECT SUMMARIES PROJECT 1- Establishment of PAHO research registry PAHO needs to establish a research registry which would ensure that all research projects are identified and register in a systematic way. This would help to track all research activities. This would enhance PAHO research management as well as its promotion. This project focuses on the establishment and implementation of such a registry.

PROJECT 2- A proposal to facilitate compliance with PAHOERC processes There is an increasing demand on PAHO’s Ethical Review Committee which results in increasing workload, delays and poor compliance of the review process. This project will address this by identifying and proposing a strategy that will be useful to cope with the new situation and ensure smooth compliance with the ethical review process.

PROJECT 3- Facilitating system transition for managing WHO Collaborating center within PAHO region So far the management of WHO collaborating centers has been a paper based process.

WHO is now initiating a new electronic system to facilitate this process. This project focuses on supporting the transition from paper based system to electronic system for the region of PAHO.

PROJECT 4- Providing information on quality of care In order to improve patience surveillance it has been requested to provide information on medical errors that have been happening in Latin America. This project will focus on the systematic review of medical errors in the PAHO region.

Course report – Aug 07 Page 6 of 12

IDENTIFICATION OF POTENTIAL FUTURE TRAINERS −

−

In their course assessment questionnaires, both trainers have recommended Linda Pereira as a potential future trainer. Linda has shown a continuous interest and a clear grasp of the methodology. She was always very fast in seeing how to apply skills learnt to her own work but also to other participants’ work. She made many good interventions and showed a very nice way to communicate in a constructive way. She also showed good team work approach.

One trainer has recommended Liliana Silva as potential future trainer. Liliana was very interested and receptive to the methodology. Although the course coincided with Linda’s first day of work at PAHO, she could rapidly apply the skills learnt to her own work.

Course report – Aug 07 Page 7 of 12

LESSONS LEARNT This course was run within the unit of Research Promotion and Development in the Health System Strengthening area at PAHO in Washington DC. It was organized and sponsored in collaboration among PAHO, CIDEIM and TDR. The course had 2 main objectives: (1) to enhance project management skills within the unit and (2) to introduce the course before PAHO makes any decision in and explores way of supporting the dissemination of the course in Latin America.

Organization of the course An organizational checklist was developed and shared among trainers and with the focal point at the host institution. Due to unexpected events only 7 participants could attend and the start of the course was delayed of half a day. Four projects were selected as case studies and four small groups were established. The course started on Monday afternoon and was scheduled to end on Thursday. Time had therefore to be managed carefully to allow all groups to go through all steps within 3 days. The list of participants was finalized only few days before the course and the trainers did not have the opportunity to communicate with participants prior to the course. In this context the following lessons learnt have been collected. − Prior coordination among trainers allows a smooth co-training process − Unexpected events might come up and trainers should be ready to address raising issues with flexibility, keeping key course objectives in mind − Trainers must check on the training site that all is in place half a day before the course. This gives them time to fix potential issues. A course should not therefore be scheduled to start on Monday morning. − Remember to request a mouse for each laptop that will be used for the case studies

The course − Ensure group composition and projects are clear for all participants on day 1 − When introducing the concept of objectives it is important to emphasize that objectives are deliverables and are strategic; they are not key activities − We recommend trainers to carefully follow the example proposed in the Step-bystep guide for trainers for the Gantt chart demonstration. Indeed the example has been developed in order to ensure that various software features are slowly introduced and that participants can get familiar with simple links before we show more complicated links. − When participants are not researchers, we recommend to: o acknowledge and highlight that the example will not relate to their daily work and explain that they will be still able to get the key messages and principles and apply them to their own work o increase the use of daily life example to illustrate key points o go through the modules as they are in a way that participants are able to understand the main messages, which later they can adapt to their needs rather than trying to relate the messages directly to the participants.

Course report – Aug 07 Page 8 of 12

− Before the Power Point software demonstration, check with participants if they are familiar with the drawing tools in Power Point in order to avoid that they are bored with the demonstration. − During the Power Point software demonstration it would actually be useful to develop a simple PERT for the example used for Gantt and show how to develop a PERT chart that is logic and makes sense and how to present it.

The reporting of the course − A debriefing between the trainers every evening helped to discuss issues and anticipate problems. It also allowed trainers to collect lessons learnt from the day. − We recommend that trainers communicate and clearly divide responsibilities not only to prepare the course and to facilitate it but also to report.

Co-training − Prior course communication and agreement on clear respective responsibilities to organize, run and report on the course are key for success − It is also important for the 2 trainers to discuss, prior to the course, the co-training process i.e. sitting arrangements, sequence of presentations (will the trainers present one module after each other?), interventions from a co-trainer while the other one is presenting, etc.)

Facilitating − Standing while presenting ensures that all participants see the presenter and vice versa. This also enhances the audience attention. − It is important to pause after a question to the audience so as to provide time to all participants to think. − When questions are raised by participants it is important to clarify the question before answering or deflecting it to other participants − It is important to ensure that trainers get all participants involved in discussions − If an issue is raised repeatedly by a participant it might be a good idea to discuss the issue with the participant at coffee break − It is important to take into consideration cultural aspects while interacting with participants

Course report – Aug 07 Page 9 of 12

COURSE ASSESSMENT Assessment of the course by participants Open discussions at the end of the course In general participants found the course very helpful and relevant to their work. They felt the atmosphere was conducive for group work. They recommend that the course should be run over 4 days to reduce time pressure. They also highlighted that an example related to their type of work e.g. admi, management would have been better. − All participants found the course helpful for reasons such as enhancing their organizational skills, providing confidence to implement the project, enhancing communication among the team and also with management – clear definition of work, agreement on timelines, etc. - , development of ownership − ‘Biomedical research’ title inhibits non researcher to attend − The course was found to provide a good platform to take time to focus on planning, enhance team work within the unit − Rising conflicts among participants were well addressed − The value of covering both the planning and the evaluation aspects in the same course was highlighted − First day was found too heavy and more interaction would be welcome − The example was not relevant to the group. It was recommended to develop administrative project example e.g public health and surveillance − More daily life examples to be used while illustrating points − In general participants recommended to provide more time for case studies. − The projects used for case studies have to be well thought through before the course to allow participants to mainly focus on process (vs content) − In general time pressure was felt by all participants. However all understood that this was due to unexpected events which lead to shorten the length of the course Participant questionnaires CIDEIM has revised the participant assessment questionnaires initially developed by TDR to validate the skill-building course. They used these revised questionnaires during a skill-building course at CIDEIM in Jul 07. The questionnaires were reviewed, revised and used for this course.

Summary analysis of the questionnaires for participants Course assessment Regarding the content, 100% of participants agreed that the course content was useful, clear and practical. Similarly, they have confidence in applying the acquired abilities and feel that these abilities will have an impact in their daily work. The application and impact were mentioned to improve team communication, efficiency and provide confidence. All scores were 4 or above (5 being the highest score and 1 the lowest score). The potential limitations to implement the acquired abilities were time, need that all the team is familiar with the planning and evaluation concepts and methods, Course report – Aug 07 Page 10 of 12

“new projects need more time, patience and negotiation”, and need of extra help to clarify projects.

The logistics of the course was given a median of 4 with one participant recommending holding the course away from the worksite. However, one participant gave a 3 score for the overall logistics plus the invitation, previous communication and equipment. Comments about course environment were positive with all scores 4 or above. Participants suggested to include examples more related to administration, have 1 more day for the course and think that “busy managers” are unlikely to disconnect totally of their work to attend the course.

Assessment of trainers Overall trainers received scores 4 or 5 in presentation skills, trainings, course dynamic and environment, and case studies. There were lower scores for time management, although one participant recognized there were external factors affecting the agenda.

Feed back on the revised questionnaires − It took about 15min to 20min to fill in which is acceptable − ‘Comments’ box should be provided for each section − ‘NA’ option is required − Some participants found that there were too many questions, others thought that all questions were specific and relevant − Why not 3 answer options instead of 5? − Some questions might need to be more specific? E.g. which concepts? − Confidentiality needs to be assured through a system- e.g on line with a system or to provide a box in the room where to place the questionnaires − Confidence to answer sincerely is set during the course depending on the atmosphere and the level of openness − Some direct questions could be re-phrased more indirectly to motivate honest answers − What about giving the questionnaires earlier in the week to allow participants to rate some course aspects as the course develop and this way avoid forgetting what happened at the beginning of the course?

Quiz- including baseline could be done???

Course report – Aug 07 Page 11 of 12

ACKNOWLEDGEMENTS This course has been developed by TDR with the aim to integrate it and disseminate it into academic and research institutions in developing countries and by developing countries. In order to transfer ownership to institutions in developing countries and ensure a sustainable capacity development locally, TDR welcomes the commitment of these institutions and the support of additional partners.

TDR would like to thank the Research Promotion and Development unit at PAHO/HSS for their interest in the course and in supporting its dissemination in Latin America and for organizing/sponsoring a course within the institution. TDR would also like to thank CIDEIM for supporting this course by providing a trainer.

Many thanks also go to the organizers and the participants who made this course an enjoyable and successful experience. Muchas Gracias!

Course report – Aug 07 Page 12 of 12