Making Telehealth work: it’s the people & procedures! Elizabeth A. Krupinski, PhD University of Arizona
Š 2014 UA Board of Regents
What is Human Factors? • Examine social, psychological, biological & physical components that inform design, development & operation products or systems • Goal = optimize performance, user safety, satisfaction with ultimate goal maximizing benefits user experience
© 2014 UA Board of Regents
HF Is a Process
Š 2014 UA Board of Regents
Tech Acceptance Model 3 • • •
Efficiency, safety, user satisfaction impact service delivery & TA TA = behavioral intention mediated by perceived usefulness & perceived ease use TA = determining factor use behavior & ultimate adoption
Approaches o Case scenarios o Surveys & Questionnaires o Interviews o Focus groups o Observations (ethnographic analyses) o Task, methods & user analyses o Time & time-motion studies o Anthropotechnology (cultural x-fer) o Many others!
Common Methods • Ethnographic analysis: observe uses technology/system practical environment; qualitative/observational in “real world”; • Focus Groups: qualitative; elicit opinions process/technology; n = small subject to bias; useful any time • Prototyping: iterative @ several stages with various methods; $$$ = do early
© 2014 UA Board of Regents
Common Tools • Surveys & Questionnaires • Interviews • Focus groups • Observations (ethnographic analyses) • Task, methods & user analyses • Time & time-motion studies • Anthropotechnology (cultural x-fer tech) • Many others!
© 2014 UA Board of Regents
Designing Surveys & Questionnaires • Review what already exists • Check IRB & Human Subjects • Start with easy & comfortable to answer screening to find out if asking right people & people less likely terminate once put in work • Clear, precise, short, easy to read questions ] • Predefined answers provided & appropriate to question • Room for additional information • Pilot & validation © 2014 UA Board of Regents
Screening Questions • Which of the following devices do you own? Check all that apply • Smart Phone • iPad • Heart rate monitor watch • Blood glucose monitor • Step monitor • Bluetooth phone speaker in your car
© 2014 UA Board of Regents
Instruments & Responses • Major types instruments
• Attitudes & opinions • Tests of knowledge • Programs, services, tools, devices • Self-reported behaviors • Demographics
• Major types responses
• Likert-scale & semantic differential • Multiple choice • True or False • Rank order • Open-ended © 2014 UA Board of Regents
Useful Tips • Anonymous vs not • Introduction & instructions • Design for intended sample/population • Avoid unnecessary & repetitive questions • Short better than long surveys (~ 10 min) • Limit types of scales, always same direction • Avoid inconsistency (minutes & hours) • One question at a time & logical flow • Try to avoid relying on memory © 2014 UA Board of Regents
Bad Questions • 1) How many hours a day do you monitor your HR? • □ 0 – 59 minutes • □ 1 – 5 hours • □ 6 – 1- hours • 2) Step monitors are expensive & a waste of time. st agree agree neutral disagree st disagree • 3) How many servings fruit did you eat last Mon? 0 1 2 3 4 5 6 7 8 9 10 • 4) How many teenage children do you have? 12345678 © 2014 UA Board of Regents
5
4
3
2
1
Satisfaction
How satisfied are you with….?
Very Satisfied
Satisfied
Neither Satisfied nor Dissatisfied
Dissatisfied
Very Dissatisfied
Agreement
How much do you agree that…?
Strongly Agree
Agree
Neither Agree nor Disagree
Disagree
Strongly Disagree
Extent
To what extent do you…?
To a Large Extent
To a Moderate Extent
To Some Extent
To a Little Extent
Not at All
Helpfulness
How helpful is…?
Very Helpful
Somewhat Helpful
Neither
Not so Helpful
Not at All Helpful
Interest
How interested in…?
Considerable Interest
Moderate Interest
Some Interest
Little Interest
No Interest
Relative Quantity
Should..do more or less of…?
Much More
Somewhat More
Fine as Is
Somewhat Less
Much Less
Importance
How important is…?
Very Important
Somewhat Important
Neither Important not Unimportant
Somewhat Unimportant
Very Unimportant
Quality Rating
Please rate quality of…
Excellent
Above Average
Average
Below Average
Poor
Avoiding Bias Etc. • Leading questions • Built-in assumptions • Double-barreled questions • Leaving out response choices • Social desirability • Unrepresentative samples • Under-coverage • Non-response bias • Voluntary response bias • Use skip logic or conditional branching © 2014 UA Board of Regents
Bad Questions 1)People who use the Wonder Corp. HR monitor watch report they love it. How likely are you to use the Wonder Corp. HR monitor? v. likely likely neutral unlikely v. unlikely 2) How likely is it that you would use a heart rate monitor you could wear as a watch? v. likely likely unlikely v. unlikely
Š 2014 UA Board of Regents
Assessing User Satisfaction
Cognitive Walkthrough • 1+ evaluators inspect user interface • Do set tasks & evaluate ease learning & understandability • Interface = mock-up, prototype, developed • Input = user profile, users' knowledge task domain & task cases • Evaluators = HF engineers, software developers, marketing, etc. • Best in design stage also in code, test, & deployment
© 2014 UA Board of Regents
Cognitive Walkthrough • Define input • Who are users? • What tasks analyzed? • What is correct action sequence/task? • How interface defined?
© 2014 UA Board of Regents
Cognitive Walkthrough • Analysis Phase • Will users try to achieve right effect? • Will users notice correct action available? • Will user associate correct action with effect to be achieved? • If correct action performed will user see progress being made towards task solution?
© 2014 UA Board of Regents
Heuristic Evaluations • Used evaluate human-computer interaction • Alternative to using actual users • Economical • 3-5 experienced evaluators • Apply established guidelines/heuristics to review of system
© 2014 UA Board of Regents
10 Usability Heuristics • Visibility system status • Match between system & real world • User control & freedom • Consistency & standards • Error prevention • Recognition rather than recall • Flexibility & efficiency use • Aesthetic & minimalist design • Help users recognize, diagnose & recover from errors • Help & documentation
© 2014 UA Board of Regents
Contextual Inquiries • Fly on wall: Observe & record behavior in context, without interfering activities • Day in life: Catalog activities & contexts users experience entire day • Personal inventory: Document things people ID important • Shadowing: Tag along & observe routines, interactions, contexts & ask questions to ensure understand what users do & why
© 2014 UA Board of Regents
Systems to Evaluate • Patient-Clinician Dynamics • Communication, relationships etc. • Cultural Issues • Training • Selection of appropriate individuals • Organizational factors • Administrative, technical & provider staff • Patient & provider location issues & logistics • Legal & billing factors • Environmental factors • It’s not just bandwidth - display, interface, seating, lighting, colors, temperature, noise, etc. © 2014 UA Board of Regents
Why study HF? o Understand end results for particular healthcare practices & interventions o Develop better ways monitor & improve quality care o Provide evidence benefits, risks, & results treatments & make more informed decisions o Identify effective strategies improve quality & value care
What are Benefits? o Efficient care processes o Effective communication between HC providers o Understanding patient’s current medical condition o Implement effective & sustainable RCA solutions o Reduced risk medical device use error o Easier to use (or more intuitive) devices o Reduced risk health IT-related use error o Easier to use (or more intuitive) health IT o Reduced need for training o Easier repair & maintenance o Cost savings via prevention/mitigation adverse events o Safer working conditions o Improved patient outcomes Natl Ctr Human Factors Engineering in HC http://medicalhumanfactors.net/
Some Useful Surveys • TM Satisfaction & Usefulness Survey Bakken et al. JAMIA 2006;13:660-667 • Technology Readiness Index (Tri) Parasuraman J Service Res 2000;2:307-320 • Structure-Process-Outcome Framework Angst et al. J Mngt Info Sys 2012;29:257-292 • Health IT Usability Evaluation Model Brown et al. J Biomed Informat 2013;In Press Online First • Health Education Impact Questionnaire Osborne et al. Pt Educ Couns 2007;66:192-201
© 2014 UA Board of Regents
Useful Tips • Always consider system as whole
• Narrow hypotheses discrete & concrete questions; focus outcomes • Use qualitative & quantitative tools • Utilize control groups when feasible • Larger sample sizes = better • Consider longitudinal studies
© 2014 UA Board of Regents
Case scenario: Telehealth & aging patients
Š 2014 UA Board of Regents
Two Trends • Biomedicalization • Tendency to define any emotional, mental & physical process as medical problems • In society that positions youth & youthful bodies as norm – changes associated with aging often labeled pathological & need to “restore” to youth
© 2014 UA Board of Regents
Two Trends • Aging natural & variable • Transform & creating technologies, architecture, etc. to accommodate changes in emotional, mental & physical changes • Aging body as enabled rather than seeking to transform from inside out
© 2014 UA Board of Regents
Provider Attitudes • • • • •
Aging Semantic Differential Maxwell-Sullivan Attitude Survey UCLA Geriatrics Attitude Survey Implicit Association Test Health Professionals Beliefs & Opinions About the Elderly • Kogan Scale • AGED Inventory © 2014 UA Board of Regents
Provider Attitudes • Vary considerably: type provider, age experience, gender etc. • Overall more negative than positive • Gonzales (2010): med students often have stereotypic attitudes towards older adults • Gunderson (2005): rural FL physicians caring elderly ageist perceptions, esp against > 85 & nursing home populations • Impacts quality, options considered etc.
© 2014 UA Board of Regents
Stereotypes & Tech • Marginalized from design & development • Assigned object rather than subject role • Do come in daily contact with technology but commonly represented as lacking skills & comprehension to accept, negotiate & assess risks/benefits
© 2014 UA Board of Regents
Acceptance e-Health • • • •
Jung & Loria 2010 J Multidisc HC Technology Acceptance Model (TAM) 12 Swedish subjects > 45 in-depth interviews 3 services: online health guide, e-prescriptions, ask-thedoctor online service • All positive attitudes towards using, usefulness, ease of use, intention future use • Main reason not used previously – simple lack of awareness that they exist!
© 2014 UA Board of Regents
Telemonitoring • Steele et al. 2009 Intl J Med Informatics 21 Australians > 65 living independently Independence
Perception nursing homes
-
Significance independence
+ 8.5/10 score
Impact QOL
Benefits & change lifestyle
+ safety net, not interfere
Concerns
Cost, social implication, adherence, healthy, privacy, anxiety, system reliability
-
Personal preferences
Previous experience Training willingness
+
Design preferences
Wearable Ambient Embedded (under skin)
- (cost) +
External factors
Support Type care Housing Who subsidize
+&-
Š 2014 UA Board of Regents
Doing it My Way • • • •
Loe 2010 Sociol Health & Illness 10 women 90 – 96 aging in place interviewed Technology for mobility – walking, driving etc. Telephone top technology – emergency, stay connected, feel needed, relieve isolation • Grass-roots health monitoring networks • Computers, tv, radio – in control & connected • Technology – use relieve fear injury, helpless • Often choose NOT to use technology, medications etc. – maintain self-determination © 2014 UA Board of Regents
Rejecting Technology • Copelton 2010 Sociol Health & Illness • Hospital-sponsored walking group 50-79 yo • Free pedometer, manual, activity log, resistance band, t-shirt • Adopt goals & chart progress pedometer • Revised! No goals, records, pedometers! • Pedometers a turn-off • Social aspect main reason attend & return • Pedometers = competition, hierarchy, threat to sociability
© 2014 UA Board of Regents
Obviously Not For Me • • • • • •
Neven 2010 Sociol Health & Illness Test robot prototype enhance health aging 62 – 79 yo in lab tests & interviews Liked it, saw use for it, interacted with it But – not for me! Made prospective users into someone else - housebound, old, lonely, feeble, in need care & attention • Saw selves as just the opposite
© 2014 UA Board of Regents
Implications • Attitudes matter – patients & providers • Need to educate & transform stereotypes is key to transitioning delivery of healthcare to patients via telemedicine • Reasons for acceptance or rejection of telemedicine systems/technology may not always be what you would expect • People really only as young/old as they feel! • Adapt & transform technology - don’t target
© 2014 UA Board of Regents
Do Not Underestimate!
Š 2014 UA Board of Regents
Thank you! Questions? krupinski@radiology.arizona.edu