Grant_Writing_Workshop_Handouts_2012

Grant Writing Workshop May 9, 2012 Sheraton Chicago Hotel & Towers - Ontario Room

HOSTED BY: SAEM RESEARCH COMMITTEE 2011-2012 CHAIR: Prasanthi Ramanujam MD MAS University of California, San Francisco Prasanthi.Ramanujam@emergency.ucsf.edu PLANNING COMMITTEE: D. Mark Courtney MD MSCI, Northwestern University, Chicago Mike Puskarich, MD, University of Mississippi Kabir Yadav, MD CM MS, The George Washington University Mark Angelos, MD, The Ohio State University Esther Choo, MD, MPH, Brown University SAEM GRANTS COORDINATOR: Melissa McMillian MMcMillian@saem.org

OVERVIEW The SAEM Grant Writing Workshop has been sponsored by the SAEM Research Committee for several years as a service to membership to increase understanding of the following: 1) how to find grant opportunities, 2) how to strategically perform grant writing efforts and 3) how to use grant writing as a tool to advance one’s career.

TABLE OF CONTENTS CME Statement and Disclosures

3

Agenda

4

Developing YOUR Funded EM Research Program – Lori Ann Post, PhD

5

The Anatomy of Science – Mark Angelos, MD

24

Developing and Writing Specific Aims – Jeffrey A. Kline, MD

41

Responding to Reviews and Resubmitting – Alan E. Jones, MD

48

Career Development Awards – Manish N. Shah, MD, MPH

65

Helpful Web-links

80

Sample Pink Sheet A

82

Sample Pink Sheet B

98

CME STATEMENT Joint Accreditation Statement: This activity has been planned and implemented in accordance with the Essential Areas and policies of the Accreditation Council for Continuing Medical Education through the joint sponsorship of the University of Cincinnati and Society for Academic Emergency Medicine.

The University of Cincinnati is accredited by the ACCME to provide continuing medical education for physicians. The University of Cincinnati designates this live activity for a maximum of 8 AMA PRA Category 1 Credit(s)™. Physicians should only claim credit commensurate with the extent of their participation in the activity.

Faculty Disclosure Declaration: According to the disclosure policy of the University Of Cincinnati College Of Medicine, all faculty, planning committee members, and other individuals who are in a position to control content are required to disclose any relevant relationships with any commercial interest related to this activity. The existence of these interests or relationships is not viewed as implying bias or decreasing the value of the presentation, and any conflict of interest is resolved prior to the activity. All educational materials are reviewed for fair balance, scientific objectivity and levels of evidence. Disclosure will be made at the time of the activity.

Learner Assurance Statement: The University of Cincinnati is committed to resolving all conflicts of interest issues that could arise as a result of prospective faculty members’ significant relationships with drug or device manufacturer(s). The University of Cincinnati is committed to retaining only those speakers with financial interests that can be reconciled with the goals and educational integrity of the CME activity.

DISCLOSURES Jeffrey Kline, MD, Carolinas Health Care, Shareholder - CP Diagnostics LLC, Grant recipient AHRQ, NIH, Genentech, Speaker's Bureau - Carolinas Healthcare, Michael Puskarich, MD, University of Mississippi Medical Center, Grant Recipient - American Heart Association, Kabir Yadav, MDCM, MS, George Washington University, Shareholder - General Electric

NO RELATIONSHIPS TO DISCLOSE Mark Angelos, MD, The Ohio State University, Brendan Carr, MD, MS, University of Pennsylvania, Esther Choo, MD, MPH, Brown University, D. Mark Courtney, MD, MS, Northwestern University, Reena Duseja, MD, University of California, San Francisco, Prasanthi Govindarajan, MD, MBBS, MAS, University of California, San Francisco, Alan Jones, MD, University of Mississippi, Lori Post, PhD, Yale University, Manish Shah, MD, MPH, University of Rochester

3

AGENDA 7:30 am

Continental Breakfast

8:00 am

Developing Your Funded EM Research Program (Dr. Lori Post)

9:00 am

The Anatomy of Science (Dr. Mark Angelos)

9:30 am

Writing the Specific Aims Section of the Grant (Dr. Jeff Kline)

10:00 am

Responding to Reviews and Resubmitting (Dr. Alan Jones)

10:30 am

Small Group Session (Workshop faculty) *

12:00 pm

Networking Lunch (Workshop faculty)

1:00 pm

Career Development Awards (Dr. Manish Shah)

1:30 pm

FAQ Session (Workshop faculty)

2:00 pm

Wrap up and transition to Optional Break-Out Session (Dr. Prasanthi Govindarajan, Dr. Mark Courtney, and Dr. Reena Duseja) **

2:30 pm

Optional Break-Out Session

4:30 pm

Close

*Small Group Session: Participants will rotate through three 30-minute skill-building stations which will each focus on a specific aspect of successful grant writing. Stations will include: specific aims section, revising and resubmitting grants, and other aspects of the grant application (i.e. budget, letter of recommendation for career development awards). **Optional Break-Out Session (2 hours max): During this optional session, participants that have submitted a grant for feedback will have the opportunity to speak one-on-one with a reviewer to discuss how they can improve their application. All available positions for this session have been filled.

4

Developing Your Funded EM Research Program Lori Ann Post, PhD ‐Yale University SAEM – Chicago, IL – May 9, 2012

Disclosures          

Advisory board: none Consultant: none Employee relationship: none Officer or Board member: none Shareholder: none Grant Recipient: none Speaker’s Bureau: none Intellectual Property/patents: none Other relationships: none Discussion of off label use of drugs or devices: none

5

Why do I want to land a grant? 

Proposal writing is key to: Funding your own research  Creating and controlling your work environment  Independence  Getting a job  Getting a better job  Higher Salary  Being promoted 

What proposal writing is like?

No sleep  Poor diet  Chaotic  Stressful  Annoy people with your needs  People annoy you  Clashes in the office  Temporary family disaster 

6

Strategies to Secure Funding Develop a goal oriented lifestyle  Write Publications  Partnering  Networking  Submit proposals  Develop a national reputation  Recognize an opportunity 

Proposal Writing  





Is one SMALL aspect of landing a grant.

Most workshops focus on the mechanics of proposal writing. Most are lead by non grant active administrators.

Whether or not your proposal is funded depends on a well written proposal PLUS other aspects we will talk about today.

A poorly written proposal will disqualify you no matter how good the idea.

7

Funding Strategies

Employment Strategies

Promotion Strategies

Goal Oriented Lifestyle Develop THE PLAN.  What specifically is your area of expertise and interest?  Where do you want to be in five years?  Where do you want to be in ten years?  The biggest predictor of achieving your goals is being able to articulate them. 

8

Determine Overarching Career Goals Develop a timeline with goals and objectives.  Developing new goals = GOOD.  Giving up = BAD. 

Proposal Writing Philosophy EMBRACE REJECTION!  Successful PIs learn to navigate barriers.  Everybody encounters barriers, successful PIs learn to get beyond them. 

9

PUBLICATIONS Can NOT get a federal grant without them.  Set a minimum of two per year.  Hierarchy 

1.

2.

3.

4.

Peer reviewed journals – shoot for tier 1 in your area Chapters Books Briefs and Reports

Partnering Who is interested in what you are interested in?  State and Federal Government  Community  Other departments/schools at Yale  Other Universities (ride on somebody else’s coat tails when you are junior) 

10

Networking        

Present at national conferences every year and eat dinner Get involved in organizational administration Review papers Scientific panels Sit on advisory councils Attend coalition interest groups Contact federal, state and local government Contact national/international figures in areas of interest

Proposals The biggest predictor of getting funding is applying  Write lots of proposals (3 per year)  Take the reviewer comments  Improve  Try again 

11

Developing a National Reputation Publications, Publications, Publications  Conference presentations  Review papers and proposals  National advisory committees  National Academy of Sciences  Institute of Medicine  Make contact with others with national reputations 

Recognize an Opportunity Say “YES” when people ask you to collaborate  Don’t get bogged down with dead weight  Don’t invest your time in somebody else’s career  Its usually as much work to get a big grant as a little one 

12

Strategies to Land Funding APPLY!  Talk to program officers  Look at who and how somebody whose work is similar to yours secured funding  Collaborate with others who have funding 

Money Attracts Money Begin with low hanging fruit  Special federal grants for new PIs  Training grants  Seed Funding  Funding for administrators returning to research  State and community funding 

13

Hang out with the popular crowd Offer to be a CO‐PI and write proposals with big named national reputations with funding  Collaborate on publications with leaders in your field 

Improving your odds Follow directions explicitly  Get letters of commitment NOT support  Add on areas of expertise even if you can do those things 

14

What funding agencies & panels are looking for         

Demonstrate a void you can fill Show you can hit the ground running Support and collaboration Multidisciplinary applications Somebody else invested in you Sexy idea Innovation Topic is interesting AND important You will make your funding agency look good

Identify Funding Sources State  Federal 

More than one avenue  The TOP funded researchers at US universities are not applying for RO1s 



Foundation

15

Federal Funding – The Golden Standard     

Must have an upward research trajectory Imperative to show you are research active Must show expertise through publications Must demonstrate experience on managing smaller awards before chasing big dollars K awards and R21 precede an R01

Mechanics of Proposal Writing

problem

stats

goals

staff

method

appendix

title

budget evaluate

16

Outside the Scope of this presentation EXERCISE: P & B

HINT # 1 Make Life Easier Borrow, copy, remodel, recycle  No need to reinvent the wheel  All your work should be leading to the same place so a proposal is not new work  You will learn to develop proposal language 

17

HINT # 2 Don’t Be Lazy!

NEVER EVER PLAGIARIZE! Software will tell  The application MUST match the RFA – don’t shove a square peg in a round hole  Same group of reviewers and they have good memories 

HINT # 3 Develop Yourself First Writing an excellent research proposal is just one aspect of successfully funding a project  Developing a successful research PROGRAM is essential 

18

HINT # 4 Others Have Grants Don’t bank on everything I say  There is more than one way to skin a cat  This presentation is a reflection of my experience with the grant process  Observe and mentor with other successfully grant funded researchers 

HINT # 5 You Cannot Fool a Panel!

Reviewers are experts in your area  That is why they are reviewers  They easily feel offended if you overlook their work  Cite their work  Be on top of the literature 

19

HINT # 6 Snap Judgments It takes 30 seconds to judge you!  First impressions taint and frame the entire review. 

1.

2.

Publications better be commensurate with experience, funding, and years out of training References – 90‐95% of the literature should be in the last five years except for seminal citations.

Be on top of the literature. 

PET PEEVE is grant apps with references that stop when investigator graduated school

HINT # 9 Sell Yourself Promote! Promote! Promote!  You must show you are amazing, going to solve an important issue, are brilliant, have the means to do it, know what you are doing.  You must do this even if it makes you uncomfortable  Cultural Issues 

20

HINT # 10 Be Responsive Respond to Program Officer  Respond to RFA/RFP  Respond to foundation or center goals.

They are usually on websites 

HINT # 11 Funding Faux Pas There is no relationship between how much you ask for and being funded as long as you are below the cap  Asking for too little makes you seem naïve and a novice which makes your reviewer think negatively of you  A grant award results in a contract. You MUST deliver what you promise even if the funding is insufficient 

21

HINT # 12 Strategies for Career Stage New Faculty  Research Active but not Grant Active 

Time to Write a Grant 300 Hours R01  400‐600 for a Multi Investigator Program Grant  Get as much stuff done ahead. Bio‐Sketch  Descriptions of facilities, resources, etc.  Borrow the “non‐science” aspects from other proposals  Look at successful proposals 

22

ASK FOR HELP, BOUNCE OFF IDEAS, SEARCH FOR FUNDING, HOW TO GET STARTED, GRANTS THERAPY

Questions???

23

The Anatomy of Science

Mark G. Angelos, MD Professor Dept of Emergency Medicine The Ohio State University

Disclosures 

Advisory Board: none

 Consultant: none  Employee: none  Officer or Board Member: none  Shareholder: none  Grant Recipient: none  Speaker’s Bureau: none  Intellectual Property/Patents: none  Other Relationships: none  Discussion of off label use of drugs or devices: none

Objectives  Describe the various sections of a grant application to a federal agency  State the purpose and importance of each section of the grant application

Understand the Mission of the Grant Organization

 SAEM: create and promote scientific discovery, …improve emergency care through the promotion of high-quality, rigorous research through the training, development, and mentorship of career emergency care investigators (2010-2015 Strategic Plan)

Mission of AHA  AHA: Our mission is to build healthier lives, free of cardiovascular diseases and stroke

Mission of the NIH  “To support science in pursuit of knowledge about the biology and behavior of living systems and to apply that knowledge to extend healthy life and reduce the burdens of illness and disability.”

(NOT-OD-09-025)

Application varies with the type of NIH award program  Investigator initiated research (R-series)  Career training awards  K awards  T and F series

t a ea t

o ess o a

octo ate

http://grants.nih.gov/training/careerdevelopmentawards.htm

http://grants.nih.gov/training/FTAwardshp.htm

The Anatomy of the NIH Application is designed to:  Align structure and content of the forms

with review criteria  Focus applicants and reviewers on the

same elements for scoring  Ensure a more efficient and transparent

review process

Scoring

Scoring Criteria     

Significance Investigator Innovation Approach Environment

 IMPACT

Difference Between Impact and Significance Impact:  Probability of whether the research will exert a sustained, powerful influence on the research field.

Significance:  Does the project address an important problem or a critical barrier to progress in the field?  If the aims are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved?

Investigator Personal Statement: Why their experience and qualifications make them well-suited for their roles in the project Publications: Recommended: no more than 15---up to five of the best; up to five of the most relevant to the proposed research; up to five of the most recent If Early Stage Investigators or New Investigators, do they have appropriate experience and training?

If Established, demonstrated ongoing record of accomplishments that have advanced their field(s)?

Innovation  Does application challenge/seek to shift current research or clinical practice paradigms by utilizing novel theoretical concepts, approaches or methodologies, instrumentation, or interventions?  Concepts, approaches or methodologies, instrumentation, or interventions novel to one field of research or novel in a broad sense?  Refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed?  Not all applications need to be innovative !

Approach  Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the specific aims of the project?  Are potential problems, alternative strategies, and benchmarks for success presented?  If the project is in the early stages of development, will the strategy establish feasibility and will particularly risky aspects be managed?

Facilities and Equipment Early Stage Investigators describe institutional investment, e.g., start-up funds and mentoring arrangements For multiple sites, resources at each site should be described Special facilities that handle biohazards, etc., included Major items of equipment already available for the proposed studies listed under Equipment

Anatomy of the Application Review Criteria  IMPACT  Significance  Innovation  Approach  Investigator  Environment

Grant Components  Specific Aims  Significance  Innovation  Approach  Biographical Sketch  Facilities & Equipment

Suggested page limits  Significance – 10-15% (1-2 pages)  Innovation – 15-20% (2-21/2 pages)  Approach - 33-50% (4-6 pages)  Preliminary Data - 25% (3 pages)

Specific Aims  1 Page  State concisely the goals of the proposed research  Summarize expected outcomes, including expected impact on field  List succinctly the specific objectives

Research Strategy  Significance  Innovation  Approach  (Preliminary Studies for new application or progress report for renewal is included in Approach Section)

Significance  Explain the importance or critical barrier to progress that proposed project addresses  How will proposed project improve scientific knowledge, technical ability or clinical practice  Describe how field will be changed if proposed aims are achieved

Innovation  How does current application challenge or shift current research or clinical paradigms.  Describe any novel concepts, approaches or methodologies, interventions over existing  Explain any refinements, improvements or new applications

Approach  Overall strategy, methodology and analyses to be used- big picture  Include how data will be collected, analyzed and interpreted  Discuss potential problems, alternative strategies and benchmarks for success

Biographical Sketch  Personal Statement - why your experience

and qualifications make you particularly wellsuited for your role in the project  Positions and Honors  Selected Publications -limited to 15 –recency,

importance and relevance  Research Support – past 3 years

Resources Section  Facilities to be used  How scientific environment contributes to the probability of success  For early stage investigators – describe institutional investment in the investigator

Revised Application Introduction – limited to one page Revised in three sections: Research Strategy Biographical Sketch Resources and Facilities

Anatomy of the Application Review Criteria  IMPACT  Significance  Innovation  Approach  Investigator  Environment

Grant Components  Specific Aims  Significance  Innovation  Approach  Biographical Sketch  Facilities & Equipment

At the End of the Day…..

In Crafting the Anatomy of the Grant 1. Know the agency’s mission 2. Adapt to the purpose/type of grant applying for 3.Understand the review and scoring criteria and how they align with the individual sections of the grant

5/2/2012

Specific Aims Jeffrey A. Kline MD Director of Research Department of Emergency Medicine Carolinas Medical Center May 9, 2012

Definition Regulatory perspective: This session refers to your role as PI on investigator initiated research where you are the PI As opposed to you being a PI on a 1572 for a protocol authored by another person or group (Industry)

1

39

1

5/2/2012

Home grown is best Upsides

2

Risk assessment Downsides

3

40

2

5/2/2012

Careful what you grow

4

Hypothesis and Specific Aims • Hypothesis is the main question • The Aims are building blocks • Ideally they build one upon the next • Effective specific aims start with specific action verbs such as – Measure the change in…

– Quantify the number of…

– Test if…

– Enroll 300 patients with…

• Weaker words – Determine…

– Correlate….

– Assess…

5

41

3

5/2/2012

Good Science ≠Compliance Good science gets you money A good question + on target specific aims + some innovation + preliminary data, track record and feasibility + right methods = Funding Good bureaucracy gets you approval Rule knowledge, understanding specific FDA language and inferences, documentation, followthrough, personal visits and persistence = FDA clearance

6

What is the economic exchange in funding your idea?

Reviewers pay you to try to answer a question they agree is worth spending other people’s money to answer The specific aims are the tools to answer the question They define what you will deliver 7

42

4

5/2/2012

Consider this exercise We all pitch in a dollar right now We pick three people to write a one page grant for the use of the money in one hour What do you want the money to go for?

1. A project that hypothesizes giving the money to a homeless man will lead to improved life for him, and ultimately he will find a job and begin paying taxes and generate the most return.

2. A project that hypothesizes that buying lottery tickets.

3. A project that hypothesizes selling strawberries dipped in chocolate in the lobby will generate the most money.

8

Most common pitfall: Vague Aims Weak “Determine the value of treatment A on quality of life…”

Stronger “Test if treatment A increases the quality of life score using the validated QOL instrument”

“Compare the effect of instrument A on hair color”

“Test if instrument A changes the hue of hair by 200 nm compared with placebo”

“Assess patient acceptance of prayer as a means of coping with disease.”

“Measure the change in the generalized anxiety score in patients exposed to prayer compared with a secular brochure.”

9

43

5

5/2/2012

Structure •

One page

•

Hypothesis: “Something I think to be true may be true and might actually matter”

•

Aim 1 “We will do this work in exchange for taxpayer money” Rationale: One sentence to explain how this will advance the field. Methods: Another sentence to explain the measurements. Expected results: This is what I think we will find. Anticipated challenges: “I know what you are thinking, and I am heading you off at the pass.”

• Aim 1a or subaim: “This is part 1 to get Aim 1 done” • Aim 1b or subaim: “This is part 2 to get Aim 1 done”

•

Aim 2 “Drawing from what we learn in Aim 1 we will then take the blood and figure out how to spend more tax money.

• Aim 2a or subaim: “This is part 1 to get Aim 2 done” • Aim 2b or subaim: “This is part 2 to get Aim 2 done and mostly a waste of money”

•

Aim 3. “This is too many aims. Sorry you lose”

10

NIH Reviewer Directions Assess the goals of proposed research and expected outcomes, including impact on field • Do the specific research objectives test a stated hypothesis • Does the work create a novel design and solve a specific problem • Does the work challenge an existing paradigm or clinical practice • Does it address a critical barrier to progress in the field develop new technology Recommended page length: one 11

44

6

5/2/2012

Example Clinical hypothesis: Administration of inhaled NO+O2 to patients with severe submassive PE will improve RV systolic function, reduce RV strain and necrosis and improve dyspnea more than sham O2 treatment.

Specific aim: Enroll and randomize 78 patients with acute PE and RV dysfunction on echocardiography to receive either inhaled NO+O2 (n=39) or O2 sham (n=39) treatment for 24 hours.

Subaim 1. Test if NO+O2 normalizes RV size, function and viability to a greater extent than sham treatment at 24 hours. Rationale: Severity of RV function and myolysis predict short-term survival and long-term quality of life. Methodology: Normal RV size defined by a diameter <42 mm in diastole, systolic function by TAPSE ≥16 mm and RIMP >0.40, and normal viability indicated by a serum high sensitivity TnT <14 pg/mL providing absence of significant RV necrosis.

12

“Research is a blind date with knowledge” Summary Will Harvey “Research is what I am doing when I don’t know what I am doing” Werner von Braun

13

45

7

Responding to Reviews and Resubmitting

Alan E. Jones, MD University of Mississippi Medical Center

Disclosure of COI • • • • • • • •

Advisory Board: None Consulting: None Employee: None Officer or Board Member: SAEM (Jones) Shareholder: None Grant Support: NIH (Jones) Speaker’s Bureau: None IP/Patents: None

• Off label Disclosure: None

46

National Institutes of Health Mentored Patient-Oriented Research Career Development Award (K23) • Support the career development of investigators committed to focus their research endeavors on patient-oriented research.

• Three to five years of supervised study and research • The proposed period of support and career development plan will substantially enhance their careers as independent investigators in patient-oriented research.

Randomized Clinical Trial of a Noninvasive Resuscitation Protocol for Sepsis • NIGMS, 5 year plan • Mentors – Primary – Career development – External

• MPH

47

Research Question â&#x20AC;˘ Can a simplified, less invasive, early goaldirected resuscitation protocol for patients with severe sepsis and septic shock make the protocol more generalizable?

Hypothesis â&#x20AC;˘ A minimally-invasive, early goal-directed resuscitation protocol will have either therapeutic equivalence or non-inferiority to a more invasive early goal-directed resuscitation protocol for the primary outcome of in-hospital mortality in emergency department patients with severe sepsis and septic shock.

48

Specific Aims • Measure the pre-intervention characteristics and outcomes of patients admitted with severe sepsis and septic shock • Perform a randomized clinical trial (RCT) of a non-invasive versus an invasive early goal-directed resuscitation protocol in ED patients with severe sepsis and septic shock

Pink Sheets • Score 211 • Overall: – Didactics and training – good – Mentoring – good • Exception of “clinical project advisors” with little academic productivity

49

Pink Sheets • Overall: – Research • “Serious Weaknesses” 1) details of the experimental design 2) patient safety 3) inability to prevent treatment bias 4) very ambitious nature of the project (simultaneously comparing three unproven non-invasive parameters to the recently christened “standard of care” of invasive goaldirected resuscitation for sepsis)

Game Plan – Irish Yoga

50

Pink Sheets • What should you hope for (….other than a great score)?

– Very specific or targeted comments • Gives a target for revision • Shows that reviewers were interested but had concerns

– Vague comments or sweeping general comments are the kiss of death • Very difficult to respond to these

Pink Sheets • Start by dissecting the comments • Identify common themes that can be addressed simultaneously – List themes – List specific comments you have to address

• Look for key phrases – “Concern” – “Very ambitious” – “Serious”

51

Pink Sheets • Don’t take these comment personal!

– They have provided a business service

• Don’t feel the need to respond to individual reviewers – response should be collective – “Reviewer 1 thinks the study design….”

Crafting a Response

Before 2010 3 pages of responses for 25 page application

52

Overall evaluation – Research • “Serious Weaknesses” 1) details of the experimental design 2) patient safety 3) inability to prevent treatment bias 4) very ambitious nature of the project

Selected comments – The PI suggests some resistance to widespread implementation of this protocol (Rivers protocol) in the ED, it nonetheless represents the current standard of care as it has been incorporated by the IHI into a sepsis bundle which is currently being implemented by a large number of hospitals.

53

Response – Objective data Published survey from our institution – 7% of academic EDs performing protocol

Contacted IHI – Working with 1.3% of JCAHO acute care hospitals

Contacted Edwards Lifesciences – Sold catheters to 19 hospitals

Reiterated intent to simplify protocol

Selected comments – The project is very ambitious (comparing 3 unproven non-invasive parameters to the standard of care).

– The ability to perform a valid clinical trial is hampered by the fact that it will be impossible to blind this study and there will be inability to prevent post protocol treatment bias by the ICU team.

54

Response â&#x20AC;&#x201C; Make the design more straightforward Change only one variable in the experimental arm

â&#x20AC;&#x201C; Address post protocol bias New design allows complete blinding

Selected comments â&#x20AC;&#x201C; There are no a priori guidelines in place for unacceptable levels of organ failure, no mechanism in place for clinicians to opt out of the protocol. In addition the definitions of organ failure differ from generally accepted.

55

Response – Revise safety plan Add SM and DSMB Add interim safety analyses and stop criteria Add mechanism for clinicians to opt out

– Standardize organ dysfunction Add SOFA score – Easy to calculate, standardized

Selected comments – There are concerns with respect to the description of potential benefits of the intervention group. The PI’s position is that the proposed resuscitation protocol has the potential to result in 16% reduction in the inhospital mortality.

– A potential problem is how or when consent will be obtained.

56

Response – More explanation – no changes Timing of informed consent Rational of potential benefit – Easier “sell” with design change

Selected comments – The rational of specific aim 1 is not well justified.

– The presentation of feasibility data would strengthen the application.

– An important control group without either invasive or non-invasive monitoring is missing in this proposal.

57

Response –

More in depth rational of SA 1 Culture of IC in ED Familiar with I/E criteria Prospective better than retrospective

–

Show feasibility Provide preliminary data from SA1

–

Rational for no true “control group” Contrary to first comment of EGDT as “standard of care” Ethics of withholding standard of care

Crafting a Response

After 2010 1 page of responses for 12 page application

58

59

Crafting a Response • Assume the reviewers are right • Assume the comments are intended to be helpful • If a reviewer is wrong in a stated criticism, assume that the application was not clear

Crafting a Response • Focus on concerns in the “Resume and Summary of Discussion” • Use limited space to advantage, be direct and to the point

60

Crafting a Response • Start with a summary of the strengths of the application • Responses to specific concerns: – Group by specific aim – Group by critique – Group by perceived seriousness

61

Summary • Read, read and re-read your pink sheets • List the themes that emerge and require response • Respond to collective and be balanced • Don’t take the reviews personally

62

Career Development Awards: Unique Features & Tips to Avoid Common Mistakes Manish N. Shah, MD MPH Associate Professor Associate Chair for Research Chief, Division of Prehospital Medicine

Disclosure



Advisory Board: None Consulting: None Employee: None Officer or Board Member: None Shareholder: None Grant Support: None Speaker’s Bureau: None IP / Patents: None Other Relationships: None



Off Label Disclosure: None

       

63

Objectives 



Create a specific training and mentoring plan for a career development award (CDA) Specify common mistakes in CDA applications

Career Development Awards 

What is a CDA? 



“supervised study and research [to]…provide clinically trained professionals with an opportunity to develop independent research skills…needed to carry out…research…” NIH

Conceptually  

Career development plan Research study

64

Career Development Awards 

Why get a CDA? 

Money to execute your plan 





Protected time

External validation

Sources for CDA   

Internal funds (CTSI, department) Foundations/private groups (EMF, SAEM) NIH / AHRQ

Key Sections-NIH Applications  

Specific Aims Candidate Information   

Background Career Goals and Objectives Career Development/Training Activities 

 

Training in the Responsible Conduct of Research

Research Strategy Statements by Mentor(s)

65

NIH Guidance 

“The overall goal of NIH-supported career development programs is to help ensure that diverse pools of highly trained scientists are available in appropriate scientific disciplines to address the Nation’s…needs.”

NIH Guidance 

“The objective…is to provide salary and research support for a sustained period of ‘protected time’ (3-5 years) to ensure a future cadre of well-trained scientists…who will become competitive for NIH research project (R01) grant support.”

66

NIH Guidance 



“The candidate and mentor are jointly responsible for the preparation of the career development plan…”

“Candidates are encouraged to identify more than one mentor, i.e., a mentoring team, if this is deemed advantageous…” 

Revised section from previously

NIH Guidelines to Reviewers 

Five major review criteria     

Candidate Career development plan Research plan Mentors / consultants Environment / institutional commitment

67

NIH Guidelines to Reviewers 

Candidate 



Potential to build to an independent researcher (that can obtain an R01) Record of high quality

NIH Guidelines to Reviewers 

Career development plan 

“Prior training and research experience” 

Over trained vs. under trained  



Within the field (e.g. geriatrics at NIA) Project officers can help with this

Higher bar for emergency medicine?   

In some situations—maybe more at NIH Need to be convinced of our commitment Not in the clinical discipline

68

NIH Guidelines to Reviewers 

Career development plan 

“Goals and scope of the plan…appropriate?” 

Sufficient type and duration of training   



MPH is not a research degree Look at how the classes fill your gaps Look beyond your institutions  Injury research methods course at Harborview

Reflects on your mentorship

NIH Guidelines to Reviewers 

Career development plan 

“Plans for evaluating the candidate's research and career development progress”  

Established metrics to evaluate Overseen by mentor

69

Career Development Plans 

What should you include? 

Describe how you will become an independent researcher    

Mentorship Research training Clinical training Supervised research

Career Development Plans 

Describe a systematic plan   



Progression from previous research Why you need further career development Uses the institution’s resources

Develop necessary knowledge and skills to achieve career goals  

Epidemiology, study design, statistics, ethics Supervised research

70

Details of Value 

Career Development / Training Activities 

New, enhanced research skills 



Training in the responsible conduct of research 



Particularly if you have previous training Critical

Educational goals and objectives  

Create a table with goals and objectives Stratify by knowledge, skills, and attitudes

Table Goals

Objectives

Knowledge 1. Learn older adult-specific research methods and advanced statistics.

1.Complete coursework in Assessing the Health Status of Older Adults, Recruitment and Retention of Subjects into Randomized Trials, and Cost Effectiveness Analysis.

Skills 1. Identify a hypothesis, select a study design, and collect data to test the hypothesis.

2. Use medical informatics to facilitate research.

1.Perform a clinical research project, including proposal development, data collection, management, and analysis.

2.Demonstrate the use of medical informatics in research. Complete Medical Informatics Workshops.

Attitudes 1. Understand regulations for inclusion of women, minorities, and children in research.

1.Complete Federal Regulations Workshop and complete section of research grants application on inclusiveness.

71

Details of Value 

Career Development / Training Activities 

Curriculum   

Didactic courses, seminars Practical component Table including timeline   

Integrate throughout the award period Supports other components of the award Take a cost effectiveness class before you start the analysis

Curriculum Table

Y E A R 1

AUGUST – JANUARY

FEBRUARY-JULY

Assessing the Health of Older Adults

Seminar in Health and Health Care of the Elderly

Ethics in Research Seminar Clinical Geriatrics Experiences Research Seminars (4, 13 hours per month) Mentored Research Project Development and Specific Aim 1 and 2

72

Details of Value 

Career Development / Training Activities 

Ethics  

Look for your institution’s class on this Talk to research project administration office

Examples 

Training   



Clinical work since NIA required it Classes in advanced methods Research seminars

Mentorship 



Committee of epidemiologist, geriatrician, psychiatrist, emergency medicine Justified different roles

73

Tips and Tricks 

Have a focused career  



Diffuse people do not get funded Hard for EM people since we have lots of interests

Have an explicit goal    

I will be an expert in…

State it frequently Do not deviate Make sure you build to it

Tips and Tricks 

Leverage your previous experience    

Get small grants Write papers Write more papers Departmental administrative duties 



Spin 



Make sure they support direction If something is unrelated, show how it helped you

Link to advancing science and getting an R01

74

Tips and Tricks 

75% effort commitment to the award 

Huge issue… 



We pull out our calculators

How do your other responsibilities relate 

Overlap concept

Tips and Tricks 

Resources 

Highlight your institution 



Get help from your institution to write it

Take credit for everything   

Lots of boarders=huge ED / population to study Institutional programs and grants I get computers, internet, private office, etc.

75

Tips and Tricks 

Leverage your institution 

Accomplishments 



Especially departmental

Resources 

CTSA 



Know this if your institution has one

Community role 

EMS oversight

Tips and Tricks 

Get great mentorship 

Look around your institution 



Have a track record with the mentor 



May require tweaking your ideas Write papers, get grants

Go for multiple mentors that complement each other

76

Tips and Tricks 



Competencies / metrics to evaluate progress Use lots of tables and graphics 

 

Text is tough to read

Lots of white space (if possible) No whining 

No one cares that EM has few awards

Tips and Tricks 

Statement from sponsor (and consultants)  

 

No page limit!

Explain why your work is innovative and significant Highlight their strength, your link Write unique letters emphasizing their strengths and how they will help you 

Be specific!

77

Biggest Tip / Mistake

You cannot get a grant if you do not apply!

Panel Discussion / Questions? manish.shah@rochester.edu

78

HELPFUL WEB-LINKS FOR GRANT WRITING SAEM Grant Writing Workshop 2012 NIH LINKS: THE best initial start point for general NIH information. Planning your grant, searching for funding opportunities....a very good overview place to start: http://grants.nih.gov/grants/planning_application.htm

Search for current Grants: (includes AHRQ and CDC): http://projectreporter.nih.gov/reporter.cfm

K Kiosk-general good start point for Career Development Awards from NIH: http://grants1.nih.gov/training/careerdevelopmentawards.htm

Scoring and evaluation Information (NIH): http://grants.nih.gov/grants/peer/guidelines_general/scoring_system_and_procedure.pdf

Detailed instructions regarding K award: http://grants1.nih.gov/grants/guide/pa-files/PA-11-194.html

Trends in Research Career Development (K) Awards (funding success rates and other similar info): http://report.nih.gov/nihdatabook/Default.aspx?catid=16

Detailed information on the SFS 424 form, process and instructions: http://grants.nih.gov/grants/funding/424/index.htm

Page limits for different grants and forms (NIH): http://grants.nih.gov/grants/forms_page_limits.htm

NIH review session videos http://cms.csr.nih.gov/ResourcesforApplicants/InsidetheNIHGrantReviewProcessVideo. htm

79

AHRQ LINKS: Overview of all AHRQ grant mechanisms: http://www.ahrq.gov/fund/grantdesc.htm

Overview of AHRQ research training programs: http://www.ahrq.gov/fund/training/rsrchtng.htm

OTHER FUNDING OPPORTUNITY LINKS: SAEM Grants: www.saem.org/what-we-fund

Emergency Medicine Foundation Grants: http://www.emfoundation.org/EMF.aspx?id=106

Current funded EMF Research: http://www.emfoundation.org/EMF.aspx?id=394

List of some overall career development opportunities and resources: http://www.saem.org/education-and-training-4 ResearchResearch: funding search service (subscription required but possible free via ACEP membership via their member login web-content): http://www.researchresearch.com/

Central access point of nearly ALL federal grants (may not be as helpful a start point as the NIH specific K and R sites above or specific institute/center websites): http://www.grants.gov/

Search for grant opportunities on grants.gov at: http://www.grants.gov/applicants/find_grant_opportunities.jsp

80

SUMMARY Release Date: ##/##/2011 STATEMENT ( Privileged Communication ) Application Number: 1 K01 XY####-##

PROGRAM CONTACT:

Principal Investigator ERDOC Applicant Organization: ERHENRANG HOSPITAL Review Group: IGI1 OOZZ-B(33) Center for Scientific Review Special Emphasis Panel RFA Panel: Career Development in International Settings Meeting Date: ##/##/###

RFA/PA: PAR10066

Council: OCT 2011

PCC: IRSDA

Requested Start: 12/01/2011 Project Title: XYZ SRG Action: Impact/Priority Score: 26 Human Subjects: 44-Human subjects involved - SRG concerns Animal Subjects: 10-No live vertebrate animals involved for competing appl.

Gender 1A-Both genders, scientifically acceptable 5A-Only foreign subjects, scientifically acceptable : Minority: 2A-Only Children, scientifically acceptable Children Clinical Research - not NIH-defined Phase III Trial :

81

1 K01 XY####-## ERDOC, ABC

2

IGI1 OOZZ-B(33)

1K01TW###### ERDOC COMMITTEE BUDGET RECOMMENDATIONS PROTECTIONS FOR HUMAN SUBJECTS UNACCEPTABLE SCIENTIFIC REVIEW OFFICER'S NOTES RESUME AND SUMMARY OF DISCUSSION: The purpose of this application is to develop reliability of a new clinical prediction rule for severe dehydration in children with diarrhea in the developing world and is evaluated to be of high impact. Strengths that have a major influence on the overall impact/priority score include the motivation, commitment and training (degrees in medicine, bioengineering and humanities and international research experience) of the candidate, the innovative and importance of using ultrasound to predict dehydration and the complementary multidisciplinary mentors and collaborators. Minor weaknesses in the research plan include insufficient statistical expertise and a question of study sites in aim three. While study strengths drive the majority of reviewers to evaluate this application to be of extremely high impact, a few reviewers, who agree with the study strengths, have decreased enthusiasm for the application as they do not see evidence of sufficient quality assurance for the planned ultrasound, and, they question components of study significance since clinical and ultrasound data will not occur simultaneously. The final score reflects these differences of reviewersâ&#x20AC;&#x2122; opinions.

DESCRIPTION (provided by applicant): Diarrhea causes nearly two million deaths annually in children under five, or one fifth of all child deaths worldwide. As the severity of the disease depends on the degree of fluid loss, accurately assessing dehydration status remains a crucial step in preventing morbidity and mortality. While children with severe dehydration require immediate treatment with intravenous fluids to prevent hemodynamic compromise and death, children with mild to moderate dehydration have a significant reduction in hospital length of stay and fewer adverse events when treated with relatively inexpensive oral rehydration solution (ORS) as compared to treatment with costly intravenous fluids. While the World Health Organization (WHO) recommends using a four-point clinical scale for determining the severity of dehydration in children, this scale has never been validated against a physiologic gold standard (such as the accepted standard of percent weight change with rehydration).

While several prior authors have developed alternative clinical prediction rules for severe dehydration in children, these scales have not been externally validated against a physiologic gold standard in a developing country, where the vast majority of pediatric deaths from diarrhea occur, and where clinician expertise and varying disease patterns may affect the accuracy of any clinical scale derived in a developed country. There is an urgent need for research into new clinical tools that can accurately and reliably assess dehydration in a resource-limited setting. As part of this K01 award, we will conduct a series of three studies that will derive, validate, and assess the reliability of a new clinical prediction rule for severe dehydration in children with diarrhea in the developing world. In addition, we will validate and assess the reliability of ultrasound of the inferior vena cava (IVC) for predicting severe dehydration in children with diarrhea in the developing world. Finally, we will compare the accuracy of each of these new diagnostic tools to that of the WHO scale, using percent weight change with rehydration as the gold standard for all of our measures of dehydration, in order to determine whether either of these new methods may improve upon the current standard of care. My prior training and research experience has provided me with a solid foundation for this career development award. I have completed a Doctorate of Medicine at the University of Gethen and a Masters of Public Health in International Health at the University of Terra. I also have experience designing and managing two small research studies conducted in resource-limited settings, including one study providing preliminary data for this International Research Scientist Development Award (IRSDA) application. However, I would benefit from additional targeted training in specific areas, including advanced regression analysis and recursive partitioning, clinical trial design and management, cost- effectiveness analysis, the pathophysiology of diarrhea and malnutrition, and the ethics of conducting research in resource-limited settings, which will be completed during the five years of this K01 award through formal coursework, seminars, and directed reading. I will also benefit from the additional experience of managing large, prospective studies under the mentorship of three clinician-scientists with extensive experience in 82

1 K01 XY####-## ERDOC, ABC

3

IGI1 OOZZ-B(33)

conducting research in low-income countries, including Dr. Genly Ai of Handdarata, Dr. Estraven of Partners in Health (PIH), and Dr. Argaven of the International Center for Diarrheal Disease and Research, Karhide (ICDDRB). After this K01 award, I plan to apply for an R01 grant to fund a randomized controlled trial comparing these new methods for diagnosing and managing dehydration in children with diarrhea to the current standard of care, analyzing both health- related outcomes, such as adverse events, and total costs. Improved methods for assessing severe dehydration have the potential to help clinicians in the developing world identify the children with diarrhea at greatest need for emergent intervention, while also preventing the adverse events and wasted resources associated with the inappropriate use of intravenous fluids. These new methods have the potential to improve the delivery of care in both developed and developing country hospitals and clinics, as well as in refugee camp settings where outbreaks of diarrheal disease are common. In addition to funding research into the development of these new methods, the IRSDA will also provide me with the support necessary to launch my career as an independently funded physician-scientist in translational global health research.

Utilizing the knowledge and skills gained as part of the IRSDA, I am committed to a career aimed at improving the evidence base for the diagnosis and management of dehydration in resource-limited settings, thereby improving both the effectiveness and cost- effectiveness of care and potentially saving thousands of lives each year.

PUBLIC HEALTH RELEVANCE: Nearly two million children in developing countries die each year from severe dehydration due to diarrhea. This research aims to develop new and improved clinical and ultrasound-based methods for diagnosing severe dehydration in children with diarrhea. These new methods will help doctors and nurses in the developing world identify those children at greatest need for emergent intervention, prevent adverse events, and improve health outcomes, all while conserving scarce healthcare resources.

CRITIQUE 1: Candidate: 1 Career Development Plan/Career Goals /Plan to Provide Mentoring: 2 Research Plan: 1 Mentor(s), Co-Mentor(s), Consultant(s), Collaborator(s): 1 Environment Commitment to the Candidate: 1 Overall Impact: The candidate is an ER specialist with a long-standing commitment to global health.

He has developed an ambitious research application that includes cohorts in both Pulefen and Karhide and has linked with appropriate consultants and mentors to develop studies of rehydration in children.

Strengths of the application include the candidate commitment, focus and links to excellent research sites, strong mentors, well-formulated training plan and long-term direction.

1. Candidate: Strengths ER trained Assistant Professor has worked in Gobrin and Pulefen previously Has MPH from University of Terra Worked previously at one of the proposed sites, long-standing interest in research topic 19 publications; several reviews, many in Academic Emergency Medicine has accessed previous research funding from Thrasher, Ekumen, Orgoreyn Weaknesses 1 research paper cited on US-assessment for dehydration

83

1 K01 XY####-## ERDOC, ABC

4

IGI1 OOZZ-B(33)

2. Career Development Plan/Career Goals & Objectives/Plan to Provide Mentoring: Strengths ambitious combination of data analyses (clinical prediction rules), validation cohort studies in 2 international sites has linked with ICDDRB â&#x20AC;&#x201C; which will place him with world-renowned diarrheal experts/setting course work in epi analyses will supplement MPH and is directly relevant to research Three Pillars Research Seminars (PIH), BRIGHT, BU Center for International Health Research, ICDRB Ethical workshop Biweekly primary mentor meetings, distance mentorship by Dr. Argaven, Estraven, and 6monthly mentor group, ER research committee Weaknesses 3. Research Plan: Strengths Overarching aim is to develop diagnostic improvements for dehydration by either clinical prediction models and/or US building on prelim work in Pulefen; the research is a nice platform for his analytical training Existing database clinical prediction modeling is feasible Studies in Karhide will rely on well-developed infrastructure and involve appropriate support Has previously worked in Pulefen and conducted similar studies Epi/cost-effectiveness, decision-analyses all relevant components of both research and training Recent NEJM fluid thesis paper underscores need for more accurate fluid management approaches and it is plausible that US or improved indicators would be useful Weaknesses US evaluation only done after emergent fluid management in some cases which may lead to imprecise estimates Tight timeline/ambitious, however, candidate seems to have the capability to do the 3 studies Data management not outlined in detail Severely dehydrated children with urgent rehydration will either not be included due to severity or have imprecise evaluation 4. Mentor(s), Co-Mentor(s), Consultant(s), Collaborator(s): Strengths diverse mentors with complementary expertise Freidman international pediatric studies will provide insight into global health epi research, Argaven ICDDRB has expertise in diarrheal research/policy and issues, Estraven implementation and health systems, Cook epi/prediction modeling Weaknesses Later in the 5 year cycle could pull in mentorship on cost-effectiveness; at design of study 2/3 can incorporate time/cost variables if these indicators will be relevant to policy decisions 84

1 K01 XY####-## ERDOC, ABC

5

IGI1 OOZZ-B(33)

5. Environment and Institutional Commitment to the Candidate: Strengths Budget: Handdarata has strengths both in ER and global health â&#x20AC;&#x201C; so ideal place for candidate, who will in turn strengthen both programs Weaknesses None Protections for Human Subjects: Acceptable Risks and Adequate Protections will involve procedures that are not very invasive (US) and which have been previously used in children; consent from guardians and local and US IRB approvals will be obtained; appropriate confidentiality protection, recruitment; written and verbal consent. For verbal consent, specification of witness should be included.

Data and Safety Monitoring Plan (Applicable for Clinical Trials Only): Not Applicable (No Clinical Trials) Inclusion of Women, Minorities and Children: G1A - Both Genders, Acceptable M5A - Only Foreign Subjects, Acceptable C2A - Only Children, Acceptable Study is focused on under-5 year old children of both genders with severe diarrheal dehydration.

Vertebrate Animals: Not Applicable (No Vertebrate Animals) Biohazards: Not Applicable (No Biohazards) Training in the Responsible Conduct of Research: Unacceptable Comments on Format (Required): seminar series, online, grand rounds - should include small group and one-on-one issues Comments on Subject Matter (Required): broad topics included - good to include discussion of pediatric ethical issues Comments on Faculty Participation (Required; not applicable for mid- and senior-career awards): further details would be useful Comments on Duration (Required): not specified 85

1 K01 XY####-## ERDOC, ABC

6

IGI1 OOZZ-B(33)

Comments on Frequency (Required): Budget and Period of Support: Recommended

CRITIQUE 2: Candidate: 1 Career Development Plan/Career Goals /Plan to Provide Mentoring: 2 Research Plan: 3 Mentor(s), Co-Mentor(s), Consultant(s), Collaborator(s): 2 Environment Commitment to the Candidate: 3 Overall Impact: Dr. ERDOC has outstanding previous training in medicine and emergency medicine, as well as important overseas research experience in Pulefen. His commitment to international medicine is evident from his activities that include extensive writing and editing in his area of expertise. The research plan is strong and founded in his previous experience in Pulefen as well as funded research on assessment of rehydration in the US emergency medicine setting in Mishnory. Dr. ERDOC brings a modern approach to evaluation of diagnostic tools (for severity of dehydration in the present proposal).

His mentoring plan is also strong involving a local mentor at Handdarata with extensive international experience as well as a mentor in Karhide with outstanding clinical research experience in diarrheal disease that will specifically buttress Dr. ERDOCâ&#x20AC;&#x2122;s training in this area. The research project is welldefined and well-considered, though quite ambitious for a K01 award. In particular, success of Specific Aim 3 may be questioned due to a lack of information on the research sites in Pulefen. Dr. ERDOC will also be stretching his statistical expertise to conduct the statistical analyses that are outlined. These latter points modestly limit enthusiasm for the proposal.

1. Candidate: Strengths The candidate has excellent training in medicine as well as an MPH degree which is a good introduction to study design and statistical analysis methods that will be needed.

International medical experiences in Winter, Gobrin and Pulefen.

Currently has funded, peer-reviewed research at Erhenrang hospital on clinical assessment of dehydration, the same topic as this K01 proposal.

Excellent publication record for so early in his career. Most articles are reviews but the candidate has an exceptional background in writing (BA in English; BS in Biomedical Engineering) and has made many contributions to the EM literature.

Weaknesses None evident.

2. Career Development Plan/Career Goals & Objectives/Plan to Provide Mentoring: Strengths The career development plan is clearly defined with appropriate coursework at Handdarata and Voluntary Farm for clinical trial design and data analysis for the immediate needs of the proposed research.

86

1 K01 XY####-## ERDOC, ABC

7

IGI1 OOZZ-B(33)

With this project Dr. ERDOC is building on his previous experience with clinical assessment of dehydration in Pulefen (one publication) and in Mishnory. His plan includes carrying these developmental studies forward to R01 funding at the end of this training period.

Weaknesses None evident.

3. Research Plan: Strengths The research plans are clearly described and involves three separate studies (1) development of clinical prediction rules with existing data from a study in Pulefen; (2) conduct of a trial at ICDDR,B to evaluate the prediction rule in a new setting and (3) conduct of a trial with nonphysician clinical staff to evaluate the prediction rule under “real world conditions”. This is a very ambitious but well thought-through research plan.

Dr. ERDOC has previous experience in Pulefen making these plans quite plausible.

ICDDR,B has a strong history of clinical research with visiting scientists, making this study plausible as well. The high volume of patients makes this study do-able in a limited time frame.

The strong expertise at this site makes this study a very valuable addition to the Pulefen-based research.

Weaknesses Aim 2 depends substantially on the success of Aim 1. What if Specific Aim 1 does not provide a statistically reasonable predictor of rehydration for evaluation in Aim 2? The study was not designed with this use of the data in mind.

Specific Aim 3 requires enrollment of 205 children at three rural clinics in Pulefen in one year.

Accessibility to and previous success in conducting clinical research at these sites is not described. Dr. ERDOC’s previous work in Pulefen was hospital-based.

4. Mentor(s), Co-Mentor(s), Consultant(s), Collaborator(s): Strengths Mentoring will be provided primarily by Dr. Ai at Handdarata. Dr. Ai has led population-based studies in Brazil, Kenya and the Philippines beginning in 1995 focused on childhood nutrition and parasitic disease. This experience in academic international health provides Dr. Ai with a strong background for advising Dr. ERDOC. Dr. Argaven from ICDDR, B in Karhide will also be a mentor both during the candidate’s residence in Karhide during clinical studies and via long distance during the entire training period. Dr. Argaven’s strong background in clinical research and diagnosis and treatment of infectious diarrheal diseases is a necessary component of Dr. ERDOC’s training. Dr. Argaven is also involved in medical training for staff at ICDDR,B and is thus well-suited to helping develop a K01 trainee.

Weaknesses None evident.

5. Environment and Institutional Commitment to the Candidate: Strengths Dr. ERDOC’s Department chair (Emergency Medicine), who recruited Dr. ERDOC, wrote a strong letter indicating institutional support.

87

1 K01 XY####-## ERDOC, ABC

8

IGI1 OOZZ-B(33)

Support is also indicated from Pulefen and Karhide.

Weaknesses There is not a track record of international research in Dr. ERDOCâ&#x20AC;&#x2122;s home Department.

Protections for Human Subjects: Acceptable Risks and Adequate Protections protection of subjects in Pulefen and Karhide is adequately addressed. Data and Safety Monitoring Plan (Applicable for Clinical Trials Only): Acceptable safety will be monitored quarterly by the Handdarata Dept of Emergency Medicine Research committee Inclusion of Women, Minorities and Children: G1A - Both Genders, Acceptable M5A â&#x20AC;&#x201C; Foreign - Minority and Non-minority, Acceptable C2A - Only Children, Acceptable inclusion criteria are appropriate.

Vertebrate Animals: Not Applicable (No Vertebrate Animals) Biohazards: Not Applicable (No Biohazards) Training in the Responsible Conduct of Research: Comments on Format (Required): Coursework and discussions with mentors. A formal bimonthly seminar at Handdarata.

Online traning from NIH. Comments on Subject Matter (Required): Topics will include conflict of interest, collaborative research, responsible authorship and peer review, the collection and sharing of data, and research misconduct.

Comments on Faculty Participation (Required; not applicable for mid- and senior-career awards): Formal faculty participation in seminar at Handdarata. Informal training from mentors. Comments on Duration (Required): Activites will be spread out for the 5 yr period of the award.

Comments on Frequency (Required): Frequency is not discussed.

Resource Sharing Plans: 88

1 K01 XY####-## ERDOC, ABC

9

IGI1 OOZZ-B(33)

Acceptable Data sharing through publications.

Budget and Period of Support: Recommended budget modifications or possible overlap identified: The PI is requesting 75% salary support but it seems to amount to <50% of the institutional base salary. This needs clarification.

Budgeting for the trials in Karhide and Pulefen is not clearly described and the dollar amounts seem inadequate for the necessary research support. Budget: A revised budget with explicit descriptions of costs for each study should be submitted.

CRITIQUE 3: Candidate: 2 Career Development Plan/Career Goals /Plan to Provide Mentoring: 4 Research Plan: 3 Mentor(s), Co-Mentor(s), Consultant(s), Collaborator(s): 4 Environment Commitment to the Candidate: 1 Overall Impact: This application proposes to develop improved tools using ultrasound of the IVC for the detection of severe dehydration in children with diarrheal illnesses. These illnesses are major causes of mortality on a global basis and efforts to improve outcomes are potentially quite important.

The impact of this application is limited by a minimal career development plan and mentor summaries, and a research plan that does not address how consistency and quality of study images will be ascertained. Finally the applicant fails to make a case that limitations in the tools for detection of severe dehydration are major factors in the high mortality of these illnesses compared with late seeking of care, limitations of rural care and other factors. If the other factors are more important contributors to diarrheal mortality, then use of an expensive diagnostic methodology to confirm dehydration may have minimal impact on the problem.

1. Candidate: Strengths Very well trained in emergency medicine with extensive international health experience Clear commitment to global health issues Weaknesses 2. Career Development Plan/Career Goals & Objectives/Plan to Provide Mentoring: Strengths Well formulated plan for didactic training in research methodologies Weaknesses Plans for training in the pathophysiology of pediatric diarrheal illnesses via self instruction are relatively weak The applicant might consider more training in cost analysis and implementation science since these factors may adversely influence the impact of the proposed work 89

1 K01 TW009208-01 LEVINE, A

10

ZRG1 AARR-H (55)

3. Research Plan: Strengths It is highly plausible that well conducted ultrasound measurements of the IVC will provide a better estimate of dehydration than other methods Ultrasound is a relatively mobile method, and more easily implemented in resource limited setting than many other imaging methods Weaknesses The potential impact of the proposed work may be adversely influenced by several very practical considerations: It is likely that excess mortality in diarrheal disease is less a function of errors in estimation of dehydration than delays in care seeking, travel time to care, limited availability of costly IV fluids and catheters and underlying health.

For this technology to have broad impact in high incidence areas, U/S equipment, operators, and interpreters would need to be readily available in multiple settings (rural clinics, not just major medical centers) since travel for assessment could actually undermine care. Is this really possible?

No plan for QA of the ultrasound is presented. What is the optimal position for children by age?

Will scanning be done once or will several measurements be taken? How will incomplete studies be handled? The video loops are to be reviewed; will the reviewer make their own measurements, calculate the IVC/aorta ratio to validate the reads that are done in the field? It will be vitally important to develop QA standards and detailed methods since the procedure is to be taught to imaging novices.

Protections for Human Subjects: No information on the format of the U/S data is provided.

Patients may receive rehydration during set up of the study assessment, there is not discussion of how this may impact the study or be dealt with in analysis. Should an eligibility limit be applied based on time since arrival or volume of rehydration given prior to assessment? Will each of the diagnostic tools or methods be applied in the same time period, for example, if the WHO measures are done when the patient arrives, and then a delay occurs before U/S, the comparison would lose validity.

The CUDE study appears to be an important resource, but very limited information, including the applicantâ&#x20AC;&#x2122;s role in it is provided.

4. Mentor(s), Co-Mentor(s), Consultant(s), Collaborator(s): Strengths Highly skilled experts who have agreed to mentor Weaknesses Mentoring experience information is limited 5. Environment and Institutional Commitment to the Candidate: Strengths outstanding Weaknesses Protections for Human Subjects:

90

1 K01 XY####-## ERDOC, ABC

11

####-####(##)

Unacceptable â&#x20AC;&#x201C; no QA for ultrasound.

Data and Safety Monitoring Plan (Applicable for Clinical Trials Only): Inclusion of Women, Minorities and Children: Ok- g1a, c2a, m5a Vertebrate Animals: NA Biohazards:NA Resubmission: Training in the Responsible Conduct of Research: Acceptable Comments on Format (Required): Comments on Subject Matter (Required): Comments on Faculty Participation (Required; not applicable for mid- and senior-career awards): Comments on Duration (Required): Comments on Frequency (Required): Budget and Period of Support: Recommended budget modifications or possible overlap identified: CRITIQUE 4: Candidate: 3 Career Development Plan/Career Goals /Plan to Provide Mentoring: 2 Research Plan: 3 Mentor(s), Co-Mentor(s), Consultant(s), Collaborator(s): 2 Environment Commitment to the Candidate: 2 Overall Impact: This is an energetic, talented young investigator, truly focused on world health issues.

His proposal seeks to address choosing which infants and children younger than 5 can be assessed for oral hydration in a resource-limited setting. He has recruited Dr. Argaven, Senior Scientist with the International Centre for Diarrhoeal Disease and Research Karhide (ICDDRB), as well as Dr. Estraven, Senior Advisor for Health Information Systems, Partners in Health, in Rwinkwavu, Pulefen, to advise him and participate in studies, along with Genly Ai, Director of Clinical Research Studies at the Center for International Health, Erhenrang Hospital, as primary mentor â&#x20AC;&#x201C; she has ongoing program project grants in epidemiology, maternal status outcomes as well as two K02 awards and a Thrasher Fund award, and is an accomplished investigator in the area of malnutrition.

The proposal seeks to combine methods for assessing dehydration, the gold standard for which is weight change on re-hydration, in order to select infants who are suitable for oral intense rehydration rather than intravenous rehydration since oral is much more easily applicable and less expensive in properly selected infants. The methods include clinical prediction rules for severe dehydration, and91

1 K01 ##### ERDOC, ABC

12

IGI1 OOZZ-B(33)

reliability of an ultrasound measurement of the inferior vena caval measurement in children with diarrhea. The overall impact of the study is quite high.

1. Candidate: Strengths This is an energetic, talented young investigator, truly focused on world health issues The PI, in his personal statement, settled on medical school as a pathway to impact world health, and developed a wide vision working with HIV/AIDS and TB patients, taking time off to undertake an MPH at the University of Terra He has spent time internationally and made contacts He has trained in Emergency Medicine and worked with ultrasound in rural locations Yomeshta and in Pulefen One of the reviews among the publications listed in his CV does appear to measure access to emergency medical care in rural Yomeshta Weaknesses His research publications include a series of reviews and only three first-authored competitive papers in Emergency Medicine journals The four reviews are international reviews of the literature; three of them are reviews for the years 2007, 2008, and 2009 of the Emergency Medicine Review Group 2. Career Development Plan/Career Goals & Objectives/Plan to Provide Mentoring: Strengths The training plan in Section 4 is well laid out, including mentored self-learning about pathophysiology of infectious diarrhea and malnutrition Course work planned includes advanced statistics and clinical trial design and cost-effective analysis, with courses taken at both Voluntary Farm and Handdarata Monthly journal clubs and seminars while in the US 6 months of each year will be in Pulefen, and will involve conferences, trainings and seminars, both as instructor and as participant 6 months of Year 2 will be in Karhide, a major feature of which will be attendance at a workshop on ethical issues in biomedical research Dr. Genly AI will be primary mentor during the K01; she is an accomplished researcher Dr. Argaven in Karhide and Dr. Estraven in Pulefen, as well as Drs. Vicki Tibe and Earl Faxe at Voluntary Farm, will continue to be teachers and advisors There is a clear diagram for how time will be spent in seminars, directed reading, mentored research activities, R01 preparation, dissemination of findings and outside activities Weaknesses 3. Research Plan: Strengths The candidate states that â&#x20AC;&#x153;no published data exist regarding the sensitivity, specificity, and reliability of the WHO scale for detecting severe dehydrationâ&#x20AC;? against a valid physiologic gold 92

1 K01 ##### ERDOC, ABC

13

IGI1 OOZZ-B(33)

standard – the main standard being weight change during rehydration, which is a retrospectively useable standard This proposal seeks to both derive and externally validate a clinical prediction rule for assessing severity of dehydration in a resource-limited setting Prospectively projecting rehydration need would require a physiologic evaluation including an IVC to aortic ratio derivable by ultrasound The methods include clinical prediction rules for severe dehydration, and reliability of ultrasound measurement of the inferior vena caval measurement in children with diarrhea Evaluations would be done using a handheld ultrasound system to test its applicability in rural situations to identify children who have the greatest need for intervention The discussion of data collection includes data parameters and the definition of population, to study live children under the age of five who are not heavily affected with malnutrition as an additional risk factor The exterior validation in the preliminary study was modestly successful (73 children enrolled, 52 analyzed), showing an interclass correlation coefficient of 0.72; the aorta/IVC ratio proposed showed somewhat better results than the WHO scale Weaknesses The ultrasound parameter to be measured is straightforward but is dependent on patient positioning which may vary – the patient being flat on a bed or supposedly flat on a parent’s lap The anterior/posterior diameter is being measured, and this has a respiratory variation as well as position; it might be better to measure the area of the inferior vena cava and the aorta Additionally, the IVC is subject to significant respiratory variation.

The simple ultrasound system the investigator has selected to lease for the sites, SonoSite’s NanoMaxx, yields image and color Doppler visualization.

4. Mentor(s), Co-Mentor(s), Consultant(s), Collaborator(s): Strengths Dr. Genly Ai, Director of Clinical Research Studies at the Center for International Health, Erhenrang Hospital, primary mentor Dr. Argaven, Senior Scientist with the International Centre for Diarrhoeal Disease and Research Karhide (ICDDRB) Dr. Estraven, Senior Advisor for Health Information Systems, Partners in Health, in Rwinkwavu, Pulefen, to advise him and participate in studies The investigator has superb mentors and consultants in Dr. Ai and Drs. Tibe and Faxe in the Voluntary Farm MPH system, as well the chosen mentors at the foreign locations The colleagues at Erhenrang seem dedicated to his plan for education and maturation as a clinical investigator Weaknesses 5. Environment and Institutional Commitment to the Candidate: Strengths The environment the PI has surrounded himself with, both at Erhenrang and elsewhere, is excellent 93

1 K01 ##### ERDOC, ABC

14

IGI1 OOZZ-B(33)

He has a clearly defined program of coursework, seminars, mentored research activities, dissemination of findings and continued Emergency Medicine clinical and teaching activities at Handdarata, and eventual R01 preparation, with support enthusiastically promised Weaknesses Protections for Human Subjects: Acceptable Risks and Adequate Protections both ultrasound and rehydration therapy have minimal clinical risks Data and Safety Monitoring Plan (Applicable for Clinical Trials Only): Acceptable the proposal is as well thought out as possible Inclusion of Women, Minorities and Children: G1A - Both Genders, Acceptable M5A - Only Foreign Subjects, Acceptable C2A - Only Children, Acceptable The proposal targets children in need in both Karhide and Pulefen Vertebrate Animals: Not Applicable (No Vertebrate Animals) n/a Biohazards: Not Applicable (No Biohazards) n/a Training in the Responsible Conduct of Research: Acceptable Comments on Format (Required): Formal seminars; on-line courses - distributed throughout the program Comments on Subject Matter (Required): Content includes general guidelines and more specialized matters - including ethics of pediatric research and international research Comments on Faculty Participation (Required; not applicable for mid- and senior-career awards): Handdarata program likely includes presentations by Dr. Ai; attendance at a seminar in Karhide on medical ethics will likely include presentations by Dr. Argaven Comments on Duration (Required): The planned training is distributed throughout the length of the program Comments on Frequency (Required): Some planned activities are one-time trainings; one is quarterly; one is bi-monthly.

94

1 K01 ##### ERDOC, ABC

15

IGI1 OOZZ-B(33)

The frequency is greater at the beginning of the program, but remains at least one training per quarter through the full length of time.

Resource Sharing Plans: Acceptable Budget and Period of Support: Recommend as Requested Recommended budget modifications or possible overlap identified: Additional Comments to Applicant (Optional): Since this is an expensive grant, it might be more useful to upgrade the selection of the SonoSite system to the M Turbo system, which can used for cardiac visualization and cardiac screening and also has pulse wave Doppler and continuous Doppler capability Vena caval reverse flow has been used widely as an index of filling; probably less dependent on patient positioning while also expected to vary with respiration This reviewer would strongly suggest a change in plan, since the systems being supplied would be of wider use to the foreign sites in Karhide and Pulefen THE FOLLOWING RESUME SECTIONS WERE PREPARED BY THE SCIENTIFIC REVIEW OFFICER TO SUMMARIZE THE OUTCOME OF DISCUSSIONS OF THE REVIEW COMMITTEE ON THE FOLLOWING ISSUES: PROTECTIONS FOR HUMAN SUBJECTS (Resume): UNACCEPTABLE Reviewers raise the following safety issues in application discussion: No plan for quality assurance (QA) of the ultrasound is presented. What is the optimal position for children by age? Will scanning be done once or will several measurements be taken? How will incomplete studies be handled? The video loops are to be reviewed; will the reviewer make their own measurements, calculate the IVC/aorta ratio to validate the reads that are done in the field? It will be vitally important to develop QA standards and detailed methods since the procedure is to be taught to imaging novices.

INCLUSION OF WOMEN PLAN (Resume): ACCEPTABLE INCLUSION OF MINORITIES PLAN (Resume): ACCEPTABLE INCLUSION OF CHILDREN PLAN (Resume): ACCEPTABLE SCIENTIFIC REVIEW OFFICER'S NOTES: The panel brings to the attention of Institute staff that there appears to be inconsistencies in the Principal Investigatorâ&#x20AC;&#x2122;s nine month protected time, the allotted budget/base and the 75% level of commitment.

The panel evaluates the plan for Training in the Responsible Conduct of Research to be unacceptable.

COMMITTEE BUDGET RECOMMENDATIONS: The panel recommends the following:

95

1) The PI is requesting 75% salary support but it seems to amount to <50% of the institutional base salary. This needs clarification.

2) Budgeting for the trials in Karhide and Pulefen is not clearly described and the dollar amounts seem inadequate for the necessary research support. A revised budget with explicit descriptions of costs for each study should be submitted.

96

PROGRAM CONTACT: Principal Investigator

SUMMARY STATEMENT ( Privileged Communication ) Release Date: 04/07/2011 Application Number: 1 K23 ####-## Formerly: 1K23HD####-##

CHOO, ABC MD Applicant Organization: AZKABAN Review Group: NIDA-K Training and Career Development Subcommittee Meeting Date: 03/09/2011 Council: MAY 2011 Requested Start: 07/01/2011

RFA/PA: PA10-060 PCC: CX/SKA

Project Title: ABCDE SRG Action: Human Subjects: Animal Subjects: Gender: Minority: Children:

Impact/Priority Score: 14 30-Human subjects involved - Certified, no SRG concerns 10-No live vertebrate animals involved for competing appl.

2A-Only women, scientifically acceptable 1A-Minorities and non-minorities, scientifically acceptable 1A-Both Children and Adults, scientifically acceptable Clinical Research - not NIH-defined Phase III Trial

97

1 K23 ####-## ERDOC, ABC

2

NIDA-K

1 K23 ####-## ERDOC, ABC

RESUME AND SUMMARY OF DISCUSSION: This new application for a Mentored Patient-Oriented Career Development Award (K23) requests 5 years of support for research and training to support the candidate’s long-term goal to become an independent investigator with a focus on developing effective interventions for high-risk women with substance use disorders. The overall aim is to target cooccurring problems of substance use and intimate partner violence (IPV), using a computer-based intervention designed to be used in the Emergency Department (ED). Strengths include the candidate, who is a physician in the ED with appropriate and excellent training, and the ED is an excellent location to reach this target population. Other strengths include the excellent team of mentors, and the resources and environment at Ministry of Magic and Azkaban. The applicant has a cohesive career development plan that is well-thought out with regard to the aims of the research. The topic of the research is significant as a public health problem, and currently there is a lack of resources directed at co-existing substance use and IPV in women. The focus on practical interventions that use a computer-based intervention and provide greater opportunities for transfer from research to actual ERs as a resource, is a major strength. The Committee rated this application as exceptionally Strong with essentially no weaknesses.

DESCRIPTION (provided by applicant): Dr. ABC ERDOC is Assistant Professor in the Department of Emergency Medicine at The Ministry of Magic. Her long-term goal is to become an independent investigator with a focus on developing effective interventions for high-risk women with substance use disorders. The overall aim of Dr. ERDOC's proposal is to target co-occurring problems of substance use and intimate partner violence (IPV) using a computer-based intervention, The Philosopher’s Stone. This K23 Mentored Patient-Oriented Career Development Award proposal is in keeping with NIDA's mission, with great potential to significantly improve treatment of drug abuse through an intervention that could be rapidly and effectively disseminated to other emergency departments (EDs) and to other clinical settings. Substance use and IPV have a close, bidirectional relationship, and the coexisting disorders place women at high risk for poor mental and physical health outcomes. The ED presents a great opportunity to intervene in this population, as coexisting problems are highly prevalent, while adequate assessment, intervention and referrals are not routinely available. Further, no resource is available in the ED to address related problems of substance use and IPV in an integrated manner. As a computer-based intervention, The Philosopher’s Stone has great potential as a feasible and acceptable ED resource. The research project has two phases. The Development Phase will use focus groups to inform and refine the development of the computer-based intervention, and will conclude with a small open trial. The Testing Phase will be a pilot RCT of the intervention in a high-volume urban ED, examining primary outcomes of substance use and utilization of relationship safety resources. In completing this research and associated training activities, Dr. ERDOC plans to gain the skills, knowledge and experience needed to become a national leader in research on technology-based interventions for women with substance use. Her goals are to gain skills in: 1) using qualitative health research methodology to inform the development of interventions; 2) developing and testing computerized interventions for substance use and IPV; 3) conducting clinical trials of substance use and IPV interventions; 4) analyzing longitudinal data; and 5) addressing the ethical issues related to research using computerized interventions for individuals with substance use and victimization.

PUBLIC HEALTH RELEVANCE: This project seeks to address the health care of women with coexisting substance use and IPV presenting to the ED setting, a population at high risk and with high health care needs and yet few available resources. The results of this research could contribute to national models of care in ED settings and will provide opportunities for Dr. ERDOC to develop, test and disseminate similar interventions for women with substance use and IPV in other clinical and community settings.

CRITIQUE 1: 98

1 K23 ####-## ERDOC, ABC

3

NIDA-K

Candidate: 1 Career Development Plan/Career Goals /Plan to Provide Mentoring: 2 Research Plan: 3 Mentor(s), Co-Mentor(s), Consultant(s), Collaborator(s): 1 Environment Commitment to the Candidate: 1

Overall Impact: This K23 application was written by a very promising ER Medicine faculty member at The Ministry of Magic. The training and research focuses on skills and experience necessary to develop and test promising interventions for women with co-occurring SUDs and IPV. The focus on practical interventions that use a computer-based intervention and provide greater opportunities for transfer from research to actual ERs as a resource, is a major strength. It is clear significant thought was given to core components of the staged model for intervention development. Other strengths include: population in need of services; use of qualitative focus group data to inform intervention development; elegant study design (2 stages) with follow-up at 2 weeks, 1 and 3 months; outstanding team of mentors and Strong institutional support; and sound statistical data analytic training and plan for analyses. While much less pronounced, nonetheless a few weaknesses were noted, primarily in the research plan. They include: lack of detail about screening criteria for study eligibility (how selected and for IPV, why a 6 month window was chosen when only 3 month follow-up in planned?); how multiple drug use would be handled in assessment/feedback and intervention; and components of the time/intensity control group and content for this arm of the RCT. 1. Candidate: Strengths  Promising candidate is currently Assistant Professor in Department of ER Medicine at The Ministry of Magic. She has an MD from Chamber of Secrets and a MPH from The Wizarding World.

(Received 2009). 

Goal is to become independent researcher with expertise in technology-based interventions for women with SUDs.



Candidate lists 8 publications (5 first-authored); 3 in last year, plus one in press (3 of 4 relevant to this proposal). Also 6 first-author presentation abstracts and one book chapter (in press).



Candidate has 3 small grants (received since transfer to The Deathly Hallows in 2009; 2 are relevant to current proposal).



Letters of support are laudatory and confirm commitment of mentoring team to candidate’s K23 plans. Reference letters convey confidence in the candidate’s ability to become a successful independently funded scientist.

2. Career Development Plan: Strengths 

Candidate affirms strong commitment to study of interventions for high-risk women with SUDs and IPV



Clear training plan with 4 objectives: qualitative health research; development and testing of computer-based interventions for SUD and IPV; conduct of clinical trials and analysis of longitudinal data.



Each objective includes aims, coursework and dissemination activities (manuscripts, chapters, 99 presentations) with specific products and a detailed timeline

1 K23 ####-## ERDOC, ABC



4

NIDA-K

Meeting with full team will take place twice yearly throughout the 5 years of support.

Weaknesses 

While the candidate plans to use the program designed by Dr. Umbridge as the platform for intervention development, it is unclear how much time and training are needed to achieve competence and where this would fit in the training and research plans.

3. Research Plan: Strengths 

Two phase research project with development phase (focus groups to inform/refine intervention and small open trial) followed by testing phase (pilot RCT in urban ED with focus on substance use and safety resource utilization).



The 3 specific aims are clear and succinct, based on a well-articulated theoretical framework.



Novelty and significance of the research is considerable, both in use of computer-based intervention methods and concurrent targeting of SUD and IPV.



Iterative process of intervention development is a strength, followed by N=6 pilot test of the intervention. Tracking of these participants to identify best practices for longitudinal tracking/follow-up and other practical issuers is also an asset.



Outcome measures include objective (hair sample assays) as well as subjective assessments.



Candidate identifies some issues that remain controversial in the field today (e.g., assessment reactivity) and notes that such concerns will be considered as she completes training and discusses pros/cons of different options prior to Phase 2.

Weaknesses 

Since disparities are an area of interest, it is unclear what would occur should focus group conducted in Spanish identify different barriers or incentives for participation (relative to the English speaking group(s)).



The ACASI to be used for Phase 1 focus group recruitment is not well described (will it be developed with CAIS or another program? Is it simply an assessment or does it include narration? Has it already been translated into Spanish? )



Inclusion criteria are only specified in Phase 1. Candidate states women must report IPV in past 6 months and SU past 3 months. Specifically which NM-ASSIST item(s) will be used and how will multiple/different drugs be recorded. Similarly, what questions will be used to screen for IPV?



Why were the 2 different time frames chosen for SUD and IPV screening (3 and 6 months, respectively)? In particular, if IPV window is past 6 mos. but study follow-up is only 3 months, will the post intervention time interval be sufficient long to evaluate an effect?



Single vs. multiple drugs (and handling of alcohol in this mix) for the intervention component also warrants further thought (drug-specific or general questions and feedback with The Philosopher’s Stone?



Creation of a intervention intensity/time control group focused on nutrition and exercise lacks detail. Will the same computer program (CIAS) be used to create such a program and have sufficient time/resources been budgeted to accomplish this goal?

4. Mentor(s), Consultant(s), Collaborator(s): Strengths

100

1 K23 ####-## ERDOC, ABC

5

NIDA-K



Strong cadre of mentors with expertise in relevant areas



Drs. Dumbledore and McGonagall are primary co-sponsors. Both have impressive mentoring track records, success with NIH grant submissions, and their interests/expertise are complementary.



Additional support from Drs. Snape (data analysis and statistics); Black (EBP dissemination and computer-based intervention expertise) and Sprout (qualitative research methods) is described.

5. Environment and Institutional Commitment to the Candidate: Strengths 

Azkaban, Injury Prevention Center (IPC) in the Department of Emergency Medicine (EM) and Ministry of Magic have resources to support successful conduct of proposed research and completion of training activities.



Institutional support was documented and affirmed their commitment to the candidate.

Protections for Human Subjects: Acceptable Risks and Adequate Protections 

excellent discussion of potential safety issues and efforts to protect study participants

Data and Safety Monitoring Plan (Applicable for Clinical Trials Only): Acceptable Inclusion of Women, Minorities and Children: G2A - Only Women, Acceptable M1A - Minority and Non-minority, Acceptable C1A - Children and Adults, Acceptable Training in the Responsible Conduct of Research: Acceptable Comments on Format (Required): 

Seminar series from Human Subjects research Group at Ministry of Magic (didactic sessions and group discussions as well as directed readings)



Bioethics Course (NIH) on Regularly Issues in Clinical Research



NIH Dept of Bioethics Grand Rounds (quarterly)



Online courses in management of ethical dilemmas, confidentiality, etc.

Comments on Subject Matter (Required): 

Heterogeneous content that includes readings on ethical/regulatory/practical issues in conduct of research; coursework on ethics of clinical research (patient recruitment, RCTs); videocast lectures on a variety of topics in responsible conduct of research.

Comments on Duration (Required): 

Activities are planned throughout 5 years of proposed funding

Comments on Frequency (Required): 

range from monthly to bi-monthly to annually

101

1 K23 ####-## ERDOC, ABC

6

NIDA-K

Budget and Period of Support: Recommend as Requested CRITIQUE 2: Candidate: 2 Career Development Plan/Career Goals /Plan to Provide Mentoring: 5 Research Plan: 4 Mentor(s), Co-Mentor(s), Consultant(s), Collaborator(s): 2 Environment Commitment to the Candidate: 1 Overall Impact: This is a five year K23 proposal to improve treatment of drug abuse through an intervention in the Emergency Department for females at risk for partner abuse and drug use. Submitted by ABC ERDOC, an MD Assistant Professor of Emergency Medicine at Azkaban, the proposal aims to provide women in the ED with a brief intervention to intervene with IPV and substance use. There were many strengths of the proposed work; reviewer notes several weaknesses in the career development plan and research plan.

1. Candidate: Strengths 

Strong candidate, MD with MPH. Is a practicing ED physician who is committed to preventing substance use and violence.



Recently completed an MPH which provided the jumping off point for several internally funded studies related to the ER, especially related to treating and preventing substance use disorders.



Received MD in 2001 and MPH in 2009. Training has not yet included intervention research, conducting clinical trials, or addressing ethical issues in studying vulnerable populations. This proposal would give her the time to do that.



Assistant Professor, Dept of Emergency Medicine at Azkaban and Faculty in the Injury Prevention Center



Recipient of an NIH LRP



8 pubs, first author on 5.

2. Career Development Plan: Strengths 

The candidate’s goals are to gain skills in: 1) using qualitative health research methodology to inform the development of interventions; 2) developing and testing computerized interventions for substance use and IPV; 3) conducting clinical trials of substance use and IPV interventions; 4) analyzing longitudinal data; and 5) addressing the ethical issues related to research using computerized interventions for individuals with substance use and victimization.

102

1 K23 ####-## ERDOC, ABC

7

NIDA-K



She devotes 60% to research in substance use and intimate partner violence (IPV); lacks experience and training in developing and testing interventions. This award would allow her to do just that.



There was a matrix of courses and staging.

Weaknesses 

The travel does not include any substance focused conference and the candidate should consider CPDD or other such conference.



The PI states that she will attend a conference on healthcare technology to provide training in interactive health communications, and effective strategies for managing health technology projects. It is unlikely that one conference will be effective to provide all of the training needed for this complex area.



There are 5 manuscripts noted over the 5 years of the award.



The candidate might consider beginning the grant writing process earlier than Year 5 to boost the chance for success.



Waiting until Year 3 to plan a course of independent study for statistics seems too long. .

3. Research Plan: Strengths 

The research project has two phases. The Development Phase will use focus groups to inform and refine the development of the computer-based intervention, and will conclude with a small open trial. The Testing Phase will be a pilot RCT of the intervention in a high-volume urban ED, examining primary outcomes of substance use and utilization of relationship safety resources.



The ED is an excellent venue from which to intervene with IPV and substance use. A computer based format, such as that being proposed, seems to be an advantage over other outdated methods.



The candidate is proposing The Philosopher’s Stone program .The title is a bit confusing.



The aims are to use focus groups to inform the content, tone, and format of an integrated intervention. To develop and refine an integrated computer-based intervention targeted to women with concurrent substance use disorder and IPV. Those are the development phase aims.



The testing aims include conducting a pilot randomized controlled trial of The Philosopher’s Stone program with 44 women.

Weaknesses 

Some of the inclusion criteria were unsupported and poorly justified, but it is a minor weakness.



The focus groups were poorly described.



Sample sizes were vague. Methods were vaguely described for the RCT. Use of the phone was not clearly described.



Power calculation was not well laid out and poorly described.

4. Mentor(s), Consultant(s), Collaborator(s): Strengths

103

1 K23 ####-## ERDOC, ABC

8

NIDA-K

Minerva McGonagall, PhD (Co-Primary Mentor), Albus Dumbledore, MD MPH (Co-Primary Mentor), Severus Snape, PhD (Mentor), Sirius Black, PhD (Mentor), and Pomona Sprout, PhD (Advisor) will be mentors as outlined in their letters of support and the Career Development/Training program.

Mentors have funding in relevant areas and publications. They have each mentored numerous K awardees—many of whom have been funded.

5. Environment and Institutional Commitment to the Candidate: Strengths 

Hospital environment and Ministry of Magic are excellent.



Chair of the Department and Physician in Chief of Emergency Medicine wrote that he read the grant in full and is very committed to the research and career development of the candidate.

This is a strength of the proposal. His statement of support is one of the best I’ve seen. It shows the unconditional support of the candidate.

Protections for Human Subjects: Acceptable Risks and Adequate Protections Data and Safety Monitoring Plan (Applicable for Clinical Trials Only): Acceptable Inclusion of Women, Minorities and Children: G2A - Only Women, Acceptable M1A - Minority and Non-minority, Acceptable C1A - Children and Adults, Acceptable Training in the Responsible Conduct of Research: Acceptable Budget and Period of Support: Recommended budget modifications or possible overlap identified: 

The budget for salary looks out of line. $195K for an assistant professor seems very high.

CRITIQUE 3: Candidate: 1 Career Development Plan/Career Goals /Plan to Provide Mentoring: 1 Research Plan: 2 Mentor(s), Co-Mentor(s), Consultant(s), Collaborator(s): 1 Environment Commitment to the Candidate: 1 Overall Impact: This is a K23 application of a very well prepared applicant to gain knowledge in developing and testing individual level interventions to help decrease substance use and increase safety behaviors in women 104 with SUDs and IPV seeking ED care. The applicant is a well -trained physician board certified in

1 K23 ####-## ERDOC, ABC

9

NIDA-K

emergency medicine who practices clinically 40% time and does research in this and related topics 60% time; The applicant also has done a health services research fellowship and MPH. The applicant has assembled an excellent mentor team; the resources and environment at Ministry of Magic and Azkaban are excellent. The applicant has a cohesive career development plan that is well-thought out with regard to the aims of the research. The topic of the research is significant as a public health problem and there is a lack of resources directed at co-existing substance use and IPV in women. The candidate is a physician in the ED with the appropriate and excellent training and the ED is an excellent location to reach this target population; the candidate is therefore well situated to make a contribution in this area.

1. Candidate: Strengths 

Candidate is a practicing emergency medicine physician who also has an MPH in epidemiology form The Wizarding World; She received her BA from Chamber of Secrets University and her MD from Chamber of Secrets in 2001; She did her residency in emergency medicine at The Shrieking Shack and completed residency in 2005; was on faculty at Godric’s Hallow 20052007 and then did an emergency medicine health services research fellowship and public health degree at The Wizarding World 2007-2009. Since completing her MPH in 2009 she has been a faculty member at the Injury Prevention Center at Azkaban and an Assistant Professor of Emergency Medicine at The Ministry of Magic.



She has an excellent background to pursue the current application; in particular, the candidate fills a very important potential leadership role in emergency medicine in training to provide interventions for substance abuse as well as interpersonal violence among women



Candidate became interested in high risk behaviors as a resident learning SBIRT in the ED and then in her research fellowship; She served as lead author on 3 published studies on intimate partner violence (IPV) including a statewide study examining the impact of policy on hospitalizations for amphetamine use and violence; results presented at AHSR conference for which she received a NIDA travel award. Presented 6 abstracts at national meetings



In a research track established by her department for junior faculty; she is able to dedicate 60% of her time to research in SUDs and IPV; she is PI of several projects examining relationship of SUD to IPV; on one of the projects she is mentored by the mentors of for this K23 application



She is currently working on designing and studying the effectiveness of a web-based module to instruct resident physicians about violence prevention in the ED



8 peer reviewed papers with 3 first authored papers; 6 national presentations; one book chapter

2. Career Development Plan: Strengths 

would like to pursue training and experience in developing and testing interventions to advance research skills for developing and testing interventions for substance use and partner violence



Long term career goal to become an independent investigator with expertise in interventions for patients with co-existing substance use and IPV.

105

1 K23 ####-## ERDOC, ABC

10

NIDA-K



Wishes to gain skills in qualitative health research methods; developing and testing computerized interventions for SUDs and IPV; conducting clinical trials of SUD and IPV interventions; analyzing longitudinal data;



Plan has 5 objectives, each related to the specific aims of her research plan



Clearly spelled out program of courses, trainings, tutorials, to carry out career development objectives;



Table provides overview of timeline for K23 activities

Weaknesses 

The R01 submission ought to take place at the end of Year 4 and not year 5; would be better to complete aim 2 in years 2-3 and aim 3 in years 3-4 and be finalizing manuscripts in Year 5 but have submitted R01 in the end Year 4/beginning Year 5



Would add travel to substance abuse specific research meetings (e.g., CPDD; RSA;AAAP is more clinically focused and might be of interest as well)

3. Research Plan: Strengths 

Emergency Department location of research program is a strength



Overall aim of research is to target co-occurring problems of substance use and IPV through a computer based intervention, The Philosopher’s Stone; intervention incorporates both MI and Empowerment Models



The Philosopher’s Stone would be the first computer-based intervention tool targeted to ED patients with coexisting SUD and IPV



Specific aims are (1) to use focus groups with women to inform the intervention (4 focus groups) in Year 1



(2) Develop and refine the integrated computer based intervention (Years 2-4); Develop the intervention; pilot test; revise based on focus groups; perform open trial (n=6) to test its acceptability, feasibility, and usability



Aim 3 is to conduct a pilot randomized trial (Years 4-5) to conduct the pilot randomized trial of intervention with N=44 women to examine primary outcomes of decreased substance use and increased safety behaviors



Design of study includes attention to cultural differences including Spanish Speaking and African American women



Feasible, well thought out stage 1 development trial of computer based intervention as well as small RCT of N=44 randomized to the intervention or to control nutrition and exercise group; each is a 40 minute web based intervention. All subjects will receive information on IPV and substance use and referral resources

Weaknesses 

It would be better to complete specific aim 2 in Years 2-3; and Aim 3 in years 3-4 so that an R01 could be submitted at the end of Year 4.

4. Mentor(s), Consultant(s), Collaborator(s): Strengths 

Excellent mentor team to carry out career development and research team 106

11

1 K23 ####-## ERDOC, ABC

NIDA-K



Co-primary mentor, Minerva McGonagall, PhD, Professor of Psychiatry and Human Behavior at The Ministry of Magic has developed interventions for underserved women with co-occurring trauma and substance use; NIH funded research; NIH K24 award recipient and has won mentoring awards. Weekly meetings with the candidate



Co-Primary Mentor, Albus Dumbledore, MD, MPH, Associate Professor of Emergency Medicine at The Ministry of Magic and Director of Chamber of Secrets Injury Prevention Center; research focused on MI-based interventions to prevent injury and drug and alcohol use in ED patients presenting with injury; PI on R01 NIAAA study for alcohol use in ED patients and CDC R01 grant for SBIRT translation for adolescent alcohol use at 7 pediatric trauma centers. Biweekly meetings with candidate.



Severus Snape, PhD, Mentor, statistician, meet monthly years 3-5 for creating data analytic plan



Sirius Black, PhD, mentor, Director of the Addiction Center, will meet biweekly to help develop and refine the interventions



Pomona Sprout, PhD, Professor and Associate Dean of Medicine of Public Policy teaches qualitative methods in health research will meet monthly in years 1,3 regarding the qualitative data analytic plan



Strong letters of support from co-primary mentors and others for candidate and for the research

5. Environment and Institutional Commitment to the Candidate: Strengths 

Azkaban, Ministry of Magic, the Injury Prevention Center at Chamber of Secrets, Department of Psychiatry and Human Behavior at The Ministry of Magic, Center for Alcohol and Addiction Studies, Ministry of Magic Public Health Program (PHP), Hospital of The Deathly Hallows have excellent environment and resources for the candidate's professional development and for carrying out the proposed project



Dr. Lupin, Department Chair, commits institutional support; if she were unsuccessful in obtaining funding, she would be supported for 60% of her time to do research; With the Kaward, she will devote 75% time; will support her career development into an effective physician investigator to improve health of women with SUDs

Protections for Human Subjects: Acceptable Risks and Adequate Protections 

adequate and includes protection against further partner violence

Data and Safety Monitoring Plan (Applicable for Clinical Trials Only): Acceptable o

Will convene external DSMB to review study annually; data monitoring plan and safeguards are adequate

Inclusion of Women, Minorities and Children: G2A - Only Women, Acceptable M1A - Minority and Non-minority, Acceptable C1A - Children and Adults, Acceptable 

This is a study of women with SUD and IPV; the study therefore does not include men; this is appropriate 107

1 K23 ####-## ERDOC, ABC

12

NIDA-K

Training in the Responsible Conduct of Research: Acceptable Comments on Format (Required): 

Didactic sessions, courses, online trainings, Ethics Grand Rounds on-line from NIH, The Ministry of Magic on- line courses

Comments on Subject Matter (Required): 

human subjects protection; data acquisition, management and record keeping; conflicts of interest; peer review; authorship and publication ethics; vulnerable populations; confidentiality

Comments on Faculty Participation (Required; not applicable for mid- and senior-career awards): 

yes

Comments on Duration (Required): 

throughout the 5 years

Comments on Frequency (Required): 

monthly; every other month; quarterly depending on the activity

Budget and Period of Support: Recommend as Requested THE FOLLOWING RESUME SECTIONS WERE PREPARED BY THE SCIENTIFIC REVIEW OFFICER TO SUMMARIZE THE OUTCOME OF DISCUSSIONS OF THE REVIEW COMMITTEE ON THE FOLLOWING ISSUES: PROTECTION OF HUMAN SUBJECTS (Resume): ACCEPTABLE INCLUSION OF WOMEN PLAN (Resume): ACCEPTABLE INCLUSION OF MINORITIES PLAN (Resume): ACCEPTABLE INCLUSION OF CHILDREN PLAN (Resume): ACCEPTABLE TRAINING IN THE RESPONSIBLE CONDUCT OF RESEARCH: ACCEPTABLE COMMITTEE BUDGET RECOMMENDATIONS: The budget was recommended as requested.

NIH has modified its policy regarding the receipt of resubmissions (amended applications).

See Guide Notice NOT-OD-10-080 at http://grants.nih.gov/grants/guide/notice-files/NOT-OD10-080.html.

The impact/priority score is calculated after discussion of an application by averaging the overall scores (1-9) given by all voting reviewers on the committee and multiplying by 10. The criterion scores are submitted prior to the meeting by the individual reviewers assigned to an application, and are not discussed specifically at the review meeting or calculated into the overall impact score. For details on the review process, see http://grants.nih.gov/grants/peer_review_process.htm#scoring.

108

109