Governance Assessment Methods and, Applications of Governance Data in PolicyMaking Workshop Good Governance for Medicines programme
Mohamed Ramzy WHO Regional Office for the Eastern Mediterranean 1-4 June 2009 Cairo Egypt
Essential Medicines and Pharmaceutical Policies
Objectives
1. Briefly introduce WHO's Good Governance for Medicines (GGM) programme 2. Share experience on WHO's transparency assessment methodology 3. Share insights and reflections on translating assessment results into policies
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 2
Pharmaceutical sectors is a great target to corruption and unethical practices R&D and clinical trials
R&D priorities
Patent
Unlawful appropriation royalties
Counterfeit/ substandard Tax evasion Conflict of interest
Manufacturing Registration Pricing
Cartels
Selection Unethical donations
Pressure
Procurement & import
Collusion
Overinvoicing
Thefts
Distribution Inspection
Falsification safety/ efficacy data
Bribery
Prescription Dispensing State Capture
Pharmacovigilance Unethical promotion
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 3
Promotion
ﺳﻠﺴﻠﺔ اﻟﺪواء ﻗﺪ ﺗﻮﺟﺪ ﻣﻤﺎرﺳﺎت ﻏﻴﺮ أﺧﻼﻗﻴﺔ ﻓﻲ أي ﻣﻦ هﺬﻩ اﻟﻤﺮاﺣﻞ اﻟﺒﺤﺚ اﻟﻌﻠﻤﻲ واﻟﺘﺠﺎرب اﻟﺴﺮﻳﺮﻳﺔ
أوﻟﻮﻳﺎت اﻟﺒﺤﺚ اﻟﻌﻠﻤﻲ
ﺑﺮاءة اﻹﺧﺘﺮاع اﻟﺘﺼﻨﻴﻊ
اﻹﻋﺘﻤﺎد اﻟﻐﻴﺮ ﻗﺎﻧﻮﻧﻲ ﻟﻺﺗﺎوات
اﻟﺘﺴﺠﻴﻞ اﻹﺣﺘﻜﺎر
اﻷدوﻳﺔ اﻟﻤﺰﻳﻔﺔ اﻟﺘﻬﺮب اﻟﻀﺮﻳﺒﻲ واﻟﻤﺘﺪﻧﻴﺔ
ﺗﻀﺎرب اﻟﻤﺼﺎﻟﺢ
اﻟﺘﺴﻌﻴﺮ
ﻣﻤﺎرﺳﺔ اﻟﻀﻐﻮط
اﻹﺧﺘﻴﺎر -اﻹﻧﺘﻘﺎء اﻟﺘﻮاﻃﺆ
اﻟﺘﺒﺮﻋﺎت اﻟﻐﻴﺮ أﺧﻼﻗﻴﺔ
اﻟﺸﺮاء واﻹﺳﺘﻴﺮاد
زﻳﺎدة ﻗﻴﻤﺔ اﻟﻔﻮاﺗﻴﺮ
اﻟﺘﻮزﻳﻊ
اﻟﺴﺮﻗﺔ
اﻟﺘﻔﺘﻴﺶ ﺗﺰوﻳﺮ ﺑﻴﺎﻧﺎت اﻟﺴﻼﻣﺔ واﻟﻔﻌﺎﻟﻴﺔ
ﺳﻴﻄﺮة اﻟﻘﻄﺎع اﻟﺨﺎص ﻋﻠﻲ ﺻﻨﻊ اﻟﻘﺮار اﻟﺘﺮوﻳﺞ اﻟﻐﻴﺮ أﺧﻼﻗﻲ
اﻟﺮﺷﻮة
اﻟﻮﺻﻒ اﻟﺼﺮف رﺻﺪ اﻵﺛﺎر اﻟﺠﺎﻧﺒﻴﺔ اﻟﺘﺮوﻳﺞ
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 4
Numerous technical guidelines already exist… the challenge is to balance them with ethical practices
Ethical practices
Technical guidelines
GMP
GCP
Counterfeits
Manual on Marketing Authorization
Accountability
Transparency
Efficiency and effectiveness
Responsiveness
Forward vision
Institutional pluralism
Participation
WHO model list of EM
Good procurement practices
Ethical criteria
Rule of law
Etc…
Etc…
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 5
Good Governance for Medicines programme: a model process
Clearance MOH
PHASE I
PHASE II
PHASE III
National assessment
Development national GGM framework
Implementation national GGM programme
Assessment report
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 6
GGM framework officially adopted
GGM Strategic Plan of Action
PHASE I
PHASE II
Assessment of transparency and accountability
PHASE III
Assesses vulnerability to corruption of systems in place Looks at key functions of the pharmaceutical sector systems National assessment
Ò Regulation: registration, licensing, inspection, promotion, clinical trials Ò Supply: selection, procurement, distribution
Elements evaluated: Assessment report
Ò Country's regulations and official documents Ò Written procedures and decision-making processes Ò Committees, criteria for membership and conflict of interest policy Ò Appeals mechanisms and other monitoring systems
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 7
Key players in medicines: Who and What do we assess? Medicine Regulators: DRAs, Regulatory police, court, customs
Policy makers: Parliament, Executive branch
Others: general public, civil society, media
Patients/ consumers
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 8
Authorities
Key players in medicines Decision makers and Implementers
Medicines suppliers: Public manufacturers, importers, procurement procurement agencies, distributors, agencies dispensers
Prescribers: doctors, nurses, paramedics
PHASE I
PHASE II
PHASE III
National GGM framework Discipline based approach
Values based approach
GGM framework officially adopted
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 9
Development of National GGM Framework
‰ Developed through a national consultation process with all stakeholders
PHASE I
PHASE II
PHASE III
Components of a national GGM Framework
1. Ethical framework of moral values & ethical principles Ò Justice/fairness
5. Established anti-corruption legislation 6. Whistle-blowing mechanism
Ò Truth
7. Sanctions on reprehensible acts
Ò Service to common good
8. Transparent and accountable regulations and administrative procedures
Ò trusteeship
2. Code of conduct 3. Socialization programme 4. Promotion of Moral Leadership
Values based approach
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 10
9. Collaboration with other GG & AC initiatives 10. Management, coordination and evaluation of GGM programme (Steering Committee & task force)
Discipline based approach
PHASE I
PHASE II
Implementation of National GGM Programme
PHASE III
Strengthening systems by increasing transparency and accountability Implementation national GGM programme
Promoting awareness (general public & health professionals) Ò Dissemination of information (newsletter, website, etc.) Ò Social marketing (radio, TV jingles, posters, etc.)
Communication plan
Ò Critical thinking and discussions (seminars, conferences)
Building capacity (managers and public policy makers) Ò Technical training programme Ò Leadership training programme
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 11
Countries efforts focus on moving from phase I to phase III
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 12
Phase I
(10 countries)
Phase II
(11 countries)
Phase III
(5 countries)
Objectives of transparency assessment study
Provide countries picture of level of transparency and vulnerability to corruption of 8 functions: Ò Regulation: registration, licensing, inspection, promotion, clinical trials Ò Supply: selection, procurement, distribution
Recommendations to increase transparency and accountability in pharmaceutical public sector Initiate establishment of GGM programme in countries Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 13
Assumption
The more transparent any system is, the less vulnerable to corruption it will be Ò Presence of publicly available and easily accessible documents = sign of transparency Æ reduces vulnerability to corruption (and vice versa) Ò Awareness about their existence suggest they have been disseminated and used on practice
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 14
Questions use 4 different methods
Method 1
(information) 9
Evaluates knowledge of KI on structures/procedures
9
1 question with binary answer (yes/no)
Method 2
Method 4
(information)
(perception) 9 Open questions 9
9 9
Helps for recommendations
Evaluates knowledge KI
1 question including sub-questions with binary answer (yes/no)
Method 3
(perception) 9
Pose statement & ask KI to what extent agree
9
Allows comparison between evidence & real practice
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 15
Rating indicators method 1
1 question with binary answer: Yes or No Ò Yes (existence of document) = 1 Ù low vulnerability to corruption Ò No (document does not exists) = 0 Ù high vulnerability to corruption Ò Transcribe 1 or 0 on consolidation sheet
Use interpretation guidelines in the manual for each indicator Transcribe score calculated (0 or 1) on scoring sheets Needs to be supported by evidence
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 16
Rating indicators method 2
Each question has several criteria Binary answer for each criteria On questionnaire calculate scoring for each answer Ò Disregard all invalid answers (D.K., N.A.) Ò Score = total "yes"/total valid answers Ò If more than half are D.K. or N.A. Æ disregard whole question for this KI
Transcribe score calculated (between 0 and 1) on score sheet
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 17
Example method 2
Indicator: Are there clear written criteria for selecting the members of the essential medicines committee? No
Yes
1. Written criteria
0
1
2. Criteria publicly available
0
1
3. Specify professional qualification required
0
1
4. Specify the technical skills and work experience related to the area
0
1
5. Require declaration of conflict of interest (e.g. investment in pharmaceutical business)
0
1
6. Give a timeframe to serve as a committee member
0
1
9
1
4
1
Total
Total yes Total valid answers Scoring (total yes/total valid answers)
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 18
4 5 0.8
D.K.
Rating indicators method 3
Asks KI level of agreement with a statement Remember may be sensitive Uses Likert scale Strongly disagree
Disagree
Undecided
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 19
Agree
Strongly agree
N.A.
D.K.
Example method 3
‰ To what extent to you agree with the following statement: "the members of the registration committee are systematically and objectively selected based on the written criteria in force in your country"? 9 8 7
No. of answers
6 5 4 3
1
2 1
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 20
N.A.= not applicable; D.K.= do not know
D .K .
. .A N
ag re e ly
Ag re e St ro ng
ec id ed
e is ag re D
U nd
St ro ng ly
di sa gr
ee
0
Rating indicators method 4
Open questions Æ mainly for narrative report Brings additional information useful for recommendations
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 21
Score each function to define level "vulnerability to corruption" (Annex 5)
For each indicator (method 1 & 2) Ò Transcribe rate for each KI on score sheet Ò Calculate average rating (possible range between 0 and 1)
For each function Ò Calculate score Æ "vulnerability to corruption" Ò 10-point scoring system 0.0 ‐2.0 Extremely vulnerable
2.1 ‐4.0 Very vulnerable
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 22
4.1‐6.0 Moderately vulnerable
6.1‐8.0 Marginally vulnerable
8.1 – 10.0 Minimally vulnerable
Example of scoring registration function KI 1
Profession* Indicator I.1 Indicator I.2 Indicator I.3 Indicator I.4 Indicator I.5 Indicator I.6 Indicator I.7 Indicator I.8 Indicator I.9 Indicator I.10 Indicator I.11 Indicator I.12 Indicator I.13 Indicator I.14 Indicator I.15 Indicator I.16
KI 2
KI 3
KI 4
KI 5
KI 6
G
G
G
G
N
P
1 1 1 0.83 1 1 1 0.63 0.75 0
1 1 1 0.50 1 1 1 0.63 0.75 0
1 0.88 1 0.50 1 1 1 0.63 0.75 0
1 1 1 0.50 1 1 1 0.38 0.63 0
1 1 1 1 1 1 1 0.63 0.63 0
1 1 1 0.67 1 D.K 1 0.60 0.60 0
KI 7
KI 8
KI 9 KI 10
The Total for P each P line P is P 1 1 1 1 given by the 0.88 0.88 0.88 0.88 sum of KI's 1 0.86 0.86 0.86 0.33 scores. 0.50 0.40 0.33 1 0.875 D.K 1 1 1 0.50 0.63 0.71 0.57 0 0
1 D.K 1 0.85 0.33 0
1 1 1 0.50 0.50 0
** The average for each 0.86 0.57 0.29 question 0.57 0.29 is calculated 0 0.25 0.60 only 0 0.29 1 1 1 on1valid 1 responses 1 1 and 1 all1 1 D.K. and N.A. answers are see text in narrative report discarded
Total
Average per question**
10.00 9.38 9.57 5.57 9.88 7.00 10.00 5.95 6.22 0.00
1.00 0.94 0.96 0.56 0.99 1.00 1.00 0.60 0.62 0.00
3.71 10
0.37 1
see text in narrative report
Total ***Final score registration Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 23
9.03
7.52
Example of scoring registration function
Profession* Indicator I.1 Indicator I.2 Indicator I.3 Indicator I.4 Indicator I.5 Indicator I.6 Indicator I.7 Indicator I.8 Indicator I.9 Indicator I.10 Indicator I.11 Indicator I.12 Indicator I.13 Indicator I.14 Indicator I.15 Indicator I.16
KI 1
KI 2
KI 3
KI 4
KI 5
KI 6
KI 7
KI 8
G
G
G
G
N
P
P
P
1 1 1 0.83 1 1 1 0.63 0.75 0
1 1 1 0.50 1 1 1 0.63 0.75 0
1 1 1 1 1 1 1 1 0.88 1 1 1 0.88 0.88 0.88 0.88 1 1 1 1 1 0.86 0.86 0.86 0.50 0.50 1 0.67 0.33 0.50 0.40 0.33 Method The KI is 1asked 1 1 3 question. 1 1 1 0.875 1 1 1 level 1 of D.K (dis)agreement D.K 1 D.K in1 his/her 1 respect 1 1to a given 1 1 statement. 1 1 1 0.63 0.38 0.63 0.60 0.50 0.63 0.85 0.50 0.75 0.63 0.63 0.60 0.71 0.57 0.33 0.50 0 0 0 0 0 0 0 0
Method 4 questions: 0.86 0.57 0.57 0.29 question 0 0.25 is 0.60 An0.29 open-ended 1 1 1 1 1 1 1 1 asked to the KI.
KI 9 KI 10
P
Total
Average per question**
P
0 1
10.00 9.38 9.57 5.57 9.88 7.00 10.00 5.95 6.22 0.00
1.00 0.94 0.96 0.56 0.99 1.00 1.00 0.60 0.62 0.00
3.71 10
0.37 1
0.29 1
see text in narrative report see text in narrative report Total ***Final score registration
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 24
9.03
7.52
Example of scoring registration function
Profession* Indicator I.1 Indicator I.2 Indicator I.3 Indicator I.4 Indicator I.5 Indicator I.6 Indicator I.7 Indicator I.8 Indicator I.9 Indicator I.10 Indicator I.11 Indicator I.12 Indicator I.13 Indicator I.14 Indicator I.15 Indicator I.16
KI 1
KI 2
KI 3
KI 4
KI 5
KI 6
KI 7
KI 8
G
G
G
G
N
P
P
P
KI 9 KI 10
P
1 1 1 1 1 1 1 1 1 1 1 0.88 1 1 1 0.88 0.88 0.88 1 1 1 1 1 1 1 0.86 0.86 0.83 0.50 0.50 0.50 1 0.67 0.33 0.50 0.40 1 1 1 1 1 1 1 0.875 1 1 1 1 1 1 D.K D.K 1 D.K 1 1 1 1 1 1 1 1 1 0.63 0.63 0.63 0.38 0.63 0.60 0.50 0.63 0.85 *** The final score is given by summing 0.75 0.75 0.75 0.63 0.63 0.60 0.71 0.57 0.33 0 up all 0 the 0 averages 0 0 for0each0 question 0 0
Total
Average per question**
P 1 0.88 0.86 0.33 1 1 1 0.50 0.50 0
(orange boxes) and dividing the result by the method 0.86 0.57 number 0.29 0.57of 0.29 0 1&2 0.25 indicators 0.60 0 0.29 1 (12 1in this 1 example). 1 1 1 1 1 1 This number is1 then multiplied by 10 to be represented see in narrative report intext a 0-10 scale.
10.00 9.38 9.57 5.57 9.88 7.00 10.00 5.95 6.22 0.00
1.00 0.94 0.96 0.56 0.99 1.00 1.00 0.60 0.62 0.00
3.71 10
0.37 1
see text in narrative report
Total ***Final score registration Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 25
9.03
7.52
Analysing results and report writing
Annex 4: Annotated content list for final report Analytical framework Ò Quantitative (scoring of vulnerability to corruption) Ò Qualitative Summarise relevant information from notes taken during interviews Compare answers with evidence found Compare method 1&2 with method 3 (perception of KIs Æ see table 1)
Recommendations for actions Ò Flow naturally from questions scoring low Ò Analysis of answers to open ended questions
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 26
Selecting key informants: influences directly the quality of the findings
Knowledgeable and interest in pharmaceutical sector Multi-perspective Ò Mix of senior, middle managerial and junior level personnel Ò Different institutions and organizations: MOH, MRA, procurement office, committee members, industry, associations, academia, CSOs, etc. (see box 2, page 12)
Minimum of 10-15 interviews per function Æ saturation (no new information is coming from interviews) KIs can be interviewed for several functions (but counted only once in total number of KIs)
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 27
For more information ‌
Visit www.who.int/medicines/ggm
Or contact emp@emro.who.int ramzym@emro.who.int
Essential Medicines and Pharmaceutical Policies May 2009 – Good Governance for Medicines 28