AJHPE Vol 8, No 1 (2016) by HMPG

AJHPE

ISSN 2078-5127

African Journal of Health Professions Education

Sponsored by

May 2016, Vol. 8, No. 1

www.foundation.co.za

AJHPE

African Journal of Health Professions Education | May 2016, Vol. 8, No. 1

EDITORIAL BOARD

EDITOR-IN-CHIEF Vanessa Burch University of Cape Town INTERNATIONAL ADVISORS Deborah Murdoch-Eaton Sheffield University, UK Michelle McLean Bond University, QL, Australia

EDITORIAL 3

Cultural competence or speaking the patientâ&#x20AC;&#x2122;s language? V Burch

SHORT REPORTS

The development of a reflective vascular training portfolio: Using a country-specific infrastructure J Pillai, T B Rangaka, C Yazicioglu, T Monareng, M G Veller

6 Randomised controlled trials in educational research: Ontological and epistemological limitations M Rowe, C Oltmann 9

Standard setting and quality of assessment: A conceptual approach S S Banda

RESEARCH

11 Medical studentsâ&#x20AC;&#x2122; views on the use of video technology in the teaching of isiZulu communication, language skills and cultural competence P Diab, M Matthews, R Gokool 15 Impact of curricular changes to enhance generic skills proficiency of 1st-year medical students D Murdoch-Eaton, A J N Louw, J Bezuidenhout 20 Perceived stressors of oral hygiene students in the dental environment N A Gordon, C A Rayner, V J Wilson, K Crombie, A B Shaikh, S Yasin-Harnekar 25 Balancing the educational choices in the decision-making of a dean of medicine: Fission or fusion? J E Wolvaardt, B G Lindeque, P H du Toit 30 Relationship between student preparedness, learning experiences and agency: Perspectives from a South African university N V Roman, S Titus, A Dison 33 Perceptions of undergraduate dental students at Makerere College of Health Sciences, Kampala, Uganda towards patient record keeping A M Kutesa, J Frantz 37 Fieldwork practice for learning: Lessons from occupational therapy students and their supervisors D Naidoo, J van Wyk 41 On being agents of change: A qualitative study of elective experiences of medical students at the Faculty of Health Sciences, University of Cape Town, South Africa J Irlam, L Pienaar, S Reid 45 Mapping undergraduate exit-level assessment in a medical programme: A blueprint for clinical competence? C P L Tan, S C van Schalkwyk, J Bezuidenhout, F Cilliers 50 The Umthombo Youth Development Foundation, South Africa: Lessons towards community involvement in health professional education L M Campbell, A J Ross, R G MacGregor 56 Exploring knowledge, perceptions and attitudes about generic medicines among final-year health science students V Bangalee, N Bassa, J Padavattan, A R Soodyal, F Nhlambo, K Parhalad, D Cooppan 59 Home-based rehabilitation: Physiotherapy student and client perspectives D Parris, S C van Schalkwyk, D V Ernstzen

SENIOR DEPUTY EDITORS Juanita Bezuidenhout Stellenbosch University Jose Frantz University of the Western Cape DEPUTY EDITORS Jacqueline van Wyk University of KwaZulu-Natal Julia Blitz Stellenbosch University Michael Rowe University of the Western Cape Elizabeth Wolvaardt University of Pretoria ASSOCIATE EDITORS Francois Cilliers University of Cape Town Lionel Green-Thompson University of the Witwatersrand Dianne Manning University of Pretoria Sindiswe Mthembu University of the Western Cape Ntombifikile Mtshali University of KwaZulu-Natal Anthea Rhoda University of the Western Cape Ben van Heerden Stellenbosch University Marietjie van Rooyen University of Pretoria Gert van Zyl University of the Free State

HMPG

CEO and PUBLISHER Hannah Kikaya Email: hannahk@hmpg.co.za EXECUTIVE EDITOR Bridget Farham MANAGING EDITORS Claudia Naidu Ingrid Nye TECHNICAL EDITORS Emma Buchanan Paula van der Bijl PRODUCTION MANAGER Emma Jane Couzens DTP & DESIGN Carl Sampson HEAD OF SALES & MARKETING Diane Smith I Tel. 012 481 2069 Email: dianes@hmpg.co.za ONLINE SUPPORT Gertrude Fani I Tel. 072 463 2159 Email: publishing@hmpg.co.za FINANCE Tshepiso Mokoena HMPG BOARD OF DIRECTORS Prof. M Lukhele (Chair), Dr M R Abbas, Dr M J Grootboom, Mrs H Kikaya, Prof. E L Mazwai, Dr M Mbokota, Dr G Wolvaardt ISSN 2078-5127

65 An exploration into the awareness and perceptions of medical students of the psychosociocultural factors which influence the consultation: Implications for teaching and learning of health professionals M G Matthews, P N Diab 69 The way forward with dental student communication at the University of the Western Cape, Cape Town, South Africa R Maart, K Mostert-Wentzel 72 An online formative assessment tool to prepare students for summative assessment in physiology S Kerr, D Muller, W McKinon, P Mc Inerney 77 The forensic autopsy as a teaching tool: Attitudes and perceptions of undergraduate medical students at the University of Pretoria, South Africa L du Toit-Prinsloo, G Pickworth, G Saayman 81 Preliminary study: Predictors for success in an important premedical subject at a South African medical school N J Allers, L Hay, R C Janse van Rensburg 84 CPD QUESTIONNAIRE

SUPPLEMENT

85 Health professions education and research capacity building in Africa: Opportunities and challenges, from the Medical Education Partnership Initiative at the University of KwaZuluNatal, Durban, South Africa

AJHPE is published by the Health and Medical Publishing Group (Pty) Ltd Co. registration 2004/0220 32/07, a subsidiary of SAMA | publishing@hmpg.co.za Suites 9 & 10, Lonsdale Building, Gardener Way, Pinelands, 7405 Tel. 021 532 1281 | Cell 072 635 9825 l Letters and articles for publication must be submitted online at www.ajhpe.org.za

Research Editorial Cultural competence or speaking the patient’s language? Cultural competence has become yet another buzz word in the education of healthcare professionals. According to Prasad et al.,[1] ‘culturally competent care assumes that healthcare providers can learn a quantifiable set of attitudes and communication skills that will allow them to work effectively within the cultural context of the patients they come across’. The questions that arise, therefore, are whether clinical trainees are being fully supported to acquire this fundamental skill and whether overloaded curricula can accommodate the ongoing demand for more ‘teaching time’. A recent survey[2] found that two-thirds of US medical schools offer a medical Spanish curriculum. These data are very encouraging until the article is read in more detail. Most of these curricula are elective, not eligible for course credit, and few schools reported the use of validated instruments to measure language proficiency after completing the curriculum. Major barriers to implementing these curricula include lack of time in students’ schedules, overly heterogeneous student language skill levels, and a lack of financial resources. What about other places in the world? For example, are non-English language proficiency skills being taught to medical students in the UK, where English is not the first language of 4 million residents (8% of the population) and the first language of up to 20% of Londoners is other than English?[3] Given this information, it is easy to argue that the number of languages spoken in the UK precludes the possibility of teaching one other useful language to students. However, according to the census, four of the five major non-English languages spoken in the UK are Indian dialects. Therefore, this argument does not really hold water and it obfuscates the need for healthcare professionals to learn a quantifiable set of communication skills that will allow them to work effectively within the communities they are mandated to serve. It is clear from the literature that while medical schools have turned their attention to the issue of cultural competence, they have largely failed to address the challenge of non-English language competence of doctors. When this matter is reviewed through the wider lens of health professions education, the literature on non-English language proficiency in other healthcare training programmes is scant, at best. Instead, the literature continues to expand with terms such as cultural awareness, cultural sensitivity, cultural diversity and, more recently, cultural humility. Surely, the most basic need of a person seeking healthcare is the need to be heard and understood? So, how are we doing in Africa? The issue closer to home is vastly different from the language homogeneity of the developed world. Africa has the greatest language diversity in the world[4] and most Africans speak more than one language – it has been said that ‘multilingualism is the African lingua franca’.[5] South Africa, a case in point, has 11 official languages, and several universities offering degree programmes in the health professions include obligatory, credit-bearing courses in a non-English language, e.g. all

health sciences students learn Afrikaans and isiXhosa at the University of Cape Town,[6] medical students learn isiXhosa and Afrikaans at Stellenbosch University[7] and isiZulu at the University of KwaZulu-Natal,[8] and pharmacy students learn isiXhosa at Rhodes University.[9] These and other training programmes have given effect to the mandate to equip graduates with a quantifiable set of communication skills that will empower them to enhance the delivery of healthcare to all South Africans. The article by Diab et al.[10] in this edition of AJHPE describes a language competence course addressing the needs of isiZulu-speaking patients. Two other articles[11,12] allude to the challenges of delivering healthcare when trainees do not speak the language of the community, and highlight the strategic importance of basic nonEnglish competence in multilingual societies. The issue of cultural competence is a long way from being comprehensively addressed. Non-English language proficiency of healthcare trainees in multicultural communities needs to be systematically addressed. Is this a challenge which the developed world will take on in a significant and meaningful way, or will cultural competence continue to avoid the need for patients to be heard and understood in a language other than English? Vanessa Burch Editor-in-Chief Department of Medicine, Faculty of Health Sciences, University of Cape Town, South Africa vanessa.burch@uct.ac.za 1. Prasad SJ, Nair P, Gadhvi K, Barai I, Danish HS, Philip AB. Cultural humility: Treating the patient, not the illness. Med Educ Online 2016;21:10. [http://dx.doi.org/10.3402/meo.v21.30908] 2. Morales R, Rodriguez L, Singh A, et al. National survey of medical Spanish curriculum in US medical schools. J Gen Intern Med 2016;30:1434-1439. [http://dx.doi.org/10.1007/s11606-015-3309-3] 3. Evans N. Do you speak English? 2011 census shows that 140,000 UK residents cannot speak the national language. The Mirror, 30 January 2013. http://www.mirror.co.uk/news/uk-news/2011-census-shows-140000cannot-1563462 (accessed 28 April 2016). 4. Atkinson Q. Phonemic diversity supports a serial founder effect model of language expansion from Africa. Science 2011;332:346-349. [http://dx.doi.org/10.1126/science.1199295] 5. Desai Z. The evolution of post-apartheid policy language policy in South Africa: An ongoing site of struggle. Eur J Intercultural Studies 1995;5:18-25. 6. Hartman N, Kathard H, Perez G, et al. Health sciences undergraduate education at the University of Cape Town: A story of transformation. S Afr Med J 2012;102:477-480. 7. Khoury LR. An evaluation of the use of an e-learning platform in complementing Xhosa language teaching and learning as an additional language. MEd thesis. Pretoria: University of South Africa, 2015. 8. Matthews M, van Wyk J. Speaking the language of the patient: Indigenous language policy and practice. S Afr Fam Practice 2016;58:30-31. [http://dx.doi.org/10.1080/20786190.2015.1083718] 9. Maseko P, Kaschula R. Vocational language learning and teaching at a South African university: Preparing professionals for multilingual contexts. Stellenbosch Papers in Linguistics PLUS 2009;38:130-142. 10. Diab P, Matthews M, Gokool R. Medical students’ views on the use of video technology in the teaching of isiZulu communication, language skills and cultural competence. Afr J Health Professions Educ 2016;8(1):11-14. [http:// dx.doi.org/10.7196/AJHPE.2016.v8i1.402] 11. Parris DL, van Schalkwyk SC, Ernstzen DV. Home-based rehabilitation: Physiotherapy student and client perspectives. Afr J Health Professions Educ 2016;8(1):59-61. [http://dx.doi.org/10.7196/AJHPE.2016.v8i1.561] 12. Matthews MG, Diab PN. An exploration into the awareness and perceptions of medical students of the psychosociocultural factors which influence the consultation: Implications for teaching and learning of health professionals. Afr J Health Professions Educ 2016;8(1):65-68. [http://dx.doi.org/10.7196/AJHPE.2016.v8i1.562]

Afr J Health Professions Educ 2016;8(1):3. DOI:10.7196/AJHPE.2016.v8i1.802

May 2016, Vol. 8, No. 1 AJHPE

Short Report The development of a reflective vascular training portfolio: Using a country-specific infrastructure J Pillai, BSc, MB BCh, FCS (SA), Cert Vascular Surgery (SA); T B Rangaka, MB BCh, MMed; C Yazicioglu, BSc, BHSc Hons; T Monareng, BDS, MB BCh, FCS (SA); M G Veller, MB BCh, FCS (SA), MMed Division of Vascular Surgery, Department of Surgery, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa Corresponding author: C Yazicioglu (c_yazicioglu@yahoo.com)

Reflective learning is considered an advanced form of learning; however, it has not been routinely incorporated into postgraduate and subspecialty educational surgical portfolios. The concept of training portfolios is not clearly understood by both trainees and teachers. Subspecialty surgical programmes rely heavily on logbooks and other forms of formative assessment to certify candidates. Case-based self-reflection in postgraduate training may be used as an additional educational tool and incorporated into the curricula vitae of trainees. We describe the method used to assess a vascular case, based on a self-reflective training method (vascular case portfolio). Afr J Health Professions Educ 2016;8(1):4-5. DOI:10.7196/AJHPE.2016.v8i1.523

The gathering and interpretation of information related to teaching and mentoring is evolving. Integral are the concepts of formative and summative assessment, during and at the end of teaching programmes, respectively. Formative assessment refers to the information that is required to adjust teaching and learning during educational activities. Students and teachers may benefit from the assessment process. The concept is difficult to quantify but may involve dynamic discussions, observation and practice analysis.[1,2] Summative assessment meets the need for accountability standards and is performed at a particular time, usually at the end of training. It evaluates student knowledge and can be scored by comparison with a benchmark or standard. It may include a final project, a question paper or a senior recital. Summative assessment only has formative value when the summative information is evaluated by students and teachers to guide practice and training methods.[1,2] Training portfolios, which have been shown to improve learning, are defined as collections of trainees’ experiences that demonstrate active learning (dynamic process), achievement and assessment.[3,4] There is, however, no consensus as to the precise components of the portfolio, which usually includes aspects of formative and summative assessments. Portfolios may include a logbook, attendance at symposia, institutional activities, peer-reviewed research and written examinations. A self-reflective written component, based on current patient care and management, has been suggested as a means to improve learning by selfreflection, self-monitoring and self-assessment (formative assessment).[1] Self-reflective surgical activity may lead to immediate positive adjustment of an action and promote quality life-long learning skills. Simultaneous additional objectives to this process include literature review, mentor feedback and formal documentation.[3,5] Ideal portfolio implementation and its incorporation into generalised programmes remain unknown. It is also unclear how specific reflective portfolios benefit teaching programmes.[5] Little is known about the poten tial benefit of sharing individual trainee experiences with other trainees.[3,4] For the concept to be successful, the trainee is encouraged to be autonomous and flexible in constructing the portfolio.

May 2016, Vol. 8, No. 1 AJHPE

The objective of this article is to describe the format of a vascular case portfolio (VCP) programme intended for South African (SA) vascular trainees. It integrates current training, educational and certification activities with formal portfolio organisation and assessment. The following aspects are discussed: current status of fellowship education and training; vascular portfolio; VCP programme; VCP template; facilitator, faculty and mentor responsibilities; and programme assessment.

Vascular fellowship training and education

Vascular trainees may enter a 2-year fellowship programme once they have registered as a specialist general surgeon. There are eight accredited (and university affiliated) vascular surgery training units in SA. Between 8 and 10 trainees are active at any one time. National Board Certification (Colleges of Medicine of SA (CMSA)) is achieved after a minimum of 2 years. The current certification requirements are the compilation of a logbook (surgical cases), and success in the college examination: multiple-choice questions (MCQs) and viva voce. The Vascular Society of Southern Africa (VASSA) is mandated by the CMSA to administer and prepare the examinations. VASSA organises two to three dedicated fellowship seminars a year (attended by all fellows and a teaching faculty) which consist of didactic lectures and case presentations. VASSA has recently included peer-reviewed research as an additional requirement. Fellows are encouraged to visit centres of excellence and attend accredited symposia locally and abroad.

Vascular training portfolio

The vascular training portfolio is the written summary to be submitted prior to participating in the CMSA examination: • logbook (surgical activity over the 2 years) • description and brief critical analysis of symposia attended and/or visits to train ing centres (should include benefit or criticism of activity/learning experience) • research • documentation related to VCPs • other activity (trainee teaching, institutional activities, etc.) may be listed.

Short Report Vascular case portfolio programme

The objective of the VCP is to encourage trainee reflection, self-assessment and subse quent self-monitoring of specific activities. This practice-based assessment involves selfevaluation of ‘real cases’ by filling in a standard ised tem plate. The case assessment should be a dynamic assessment of a case of the trainee’s choice. The case portfolio should allow for flexibility and include a trainee’s experiences (e.g. competency, novel concept, complication). Once the VCP document is completed, it will be forwarded to a facilitator who will ‘blind’ the document and attach a reference number. The facilitator will then forward the document to an external mentor who will complete the VCP documentat ion by appropriate feedback and commentary. The facilitator will ensure ‘mentor blinding’ and return the VCP documentation to the trainee within 7 - 10 days. Timely feedback facilitates appropriate trainee adjustment in subsequent practice. Copies of VCP documents will be retained by the facilitator. A completed ‘blinded’ document will be forwarded to a fellow s hip seminar faculty member (other than the mentor), who will present the document as a case report at a fellowship seminar (all fellows present). The faculty member will receive the VCP document at least 2 weeks prior to the fellowship seminar. Further VCP evaluation and sharing of similar trainee experiences will be encouraged at the seminar. The VCP logistic will be explained to fellows, mentors, faculty members and facilitators prior to programme initiation. This will comprise a pre-programme fellowship meeting and subse quent written communication.

Vascular case portfolio template

The discussion and learning experience should be autonomous, self-reflective and thought provoking. It may include positive, negative (complications) and novel experiences. The level of evidence and relevant papers should be discussed. Mentor feedback should highlight specific learning issues and improvement (Table 1).

Facilitator, faculty and mentor responsibilities

The facilitator is essentially ‘task master’ who ensures trainee compliance (three VCPs per year). The facilitator may stipulate prespecific target dates for each trainee and correlate this with fellowship seminars (anticipate four to six VCP discussions per seminar). Appropriate telephonic and mail reminders are essential. The facilitator will compile a list of volunteer mentors and may allocate VCPs to mentors with specific interests. Ideally, the facilitator should be a member of the executive committee of VASSA. The facilitator will keep records of all VCPs submitted and will be responsible for mentor/trainee blinding and substitution with reference numbers. Records may be submitted to VASSA Exco for future analysis of the programme. Prior to fellowship symposia, the facilitator will submit appropriate VCPs to faculty members for formal case presentation at the seminar. The VCP topic will be part of the seminar programme. The seminar faculty will present the VCP case and stimulate further discussion among fellows and all attendees. Two faculty members will be tasked with assessing the quality of the VCP in a standardised format: • concise presentation of case • specific learning issue identified • evidence used to substantiate argument • demonstrate reflection with understanding of topic. Standardised mentor feedback will include gaps in knowledge, existing knowledge, level of understanding, potential existing errors, specific suggestion on improvements and correlation with evidence. Table 1. Vascular case portfolio template Fellow details Year of training Date Title of case Diagnostic studies Medical management Intervention Evidence/literature (list papers) Mentor feedback

Each point will carry a 1 - 5 grading. The VCP quali ty grading analysis will be submitted to the facilitator for record keeping (Excel format). Two sets of VCP MCQs (for each VCP) will be constructed as a separate faculty task and submitted to the exam ination convenor. A separate VCP MCQ bank will be developed with a date allocation to each MCQ.

Vascular case portfolio programme assessment

Various aspects of the programme will be assessed at different stages. Compliance (number of VCPs submitted per year) and subjective trainee satisfaction will be documented by the facilitator. VCP quality will be assessed during fellowship seminars. An independent assessment of VCP MCQs will be undertaken after each examination (compared with non-VCP MCQs). Other assess ments, such as overall candidate examination success v. rate/quality of VCP submissions, may be undertaken. Incremental data collection will ensure continuous programme evaluation.

Conclusion

While there is no consensus regarding an optimal fellowship curriculum vitae, the total vascular training portfolio will provide direction and standardisation of educational activity. Self-reflec tive activities (VCP programme) may encourage ‘special interest’ development and motivate fellows towards specific research initiatives. Merging an existing established educational and examination programme with a VCP programme allows for easy implementation. Similar portfolio development programmes may also benefit other subspecialist trainees (e.g. gastrointestinal tract, trauma). If undertaken by various societies, future analysis of more robust multidisciplinary portfolios will provide invaluable educational information. References 1. Boston C. The concept of formative assessment. Practical Assessment, Research and Evaluation 2002;8(9):1-8. http://PAREonline.net/getvn. asp?v=8&n=9 (accessed 31 December 2012). 2. Carnegie Mellon University. Formative and Summative Assessment. http://www.cmu.edu/teaching/assessment/basics/formativesummative.htm (accessed 31 December 2012). 3. Hassan S. Use of structured portfolio in surgical training of postgraduate medical education. Educ Med J 2011;3(2):32-43. [http:// dx.doi.org/10.5959/eimj.v3i2.63] 4. Webb TP, Merkley TR. An evaluation of the success of a surgical resident learning portfolio. J Surg Educ 2012;69(1):1-7. [http://dx.doi. org/10.1016/j.jsurg.2011.06.008] 5. Buckley S, Coleman J, Davison I, et al. The educational effects of portfolios on undergraduate student learning: A Best Evidence Medical Education (BEME) systematic review. BEME Guide No. 11. Med Teach 2009;31(4):282-298. [http://dx.doi.org/10.1080/01421590902889897]

May 2016, Vol. 8, No. 1 AJHPE

Short Report Randomised controlled trials in educational research: Ontological and epistemological limitations M Rowe,1 PhD; C Oltmann,2 PhD Department of Physiotherapy, Faculty of Community and Health Sciences, University of the Western Cape, Cape Town, South Africa

1 2

Division of Pharmacy Practice, Faculty of Pharmacy, Rhodes University, Grahamstown, South Africa

Corresponding author: M Rowe (michael@realmdigital.co.za)

Randomised controlled trials (RCTs) are a valued research method in evidence-based practice in medical and clinical settings because they are associated with a particular ontological and epistemological perspective that is situated within a positivist world view. It assumes that environments and variables can be controlled to establish cause-effect relationships. However, current theories of learning suggest that knowledge is socially constructed, and that learning occurs in open systems that cannot be controlled and manipulated as would be required in a RCT. They recognise the importance and influence of context on learning, which positivist research paradigms specifically aim to counter. We argue that RCTs are inappropriate in education research because they force one to take up ontological and epistemological positions in a technical rationalist framework, which is at odds with current learning theory. Afr J Health Professions Educ 2016;8(1):6-8. DOI:10.7196/AJHPE.2016.v8i1.683

Evidence-based practice (EBP) is positioned as an inherent good in the medical and clinical literature, and not without reason. It relies on the integration of research evidence, clinical expertise and patient preferences,[1] and has become a foundation on which health systems are built and improved. There is however, a growing body of literature that takes a more critical stance towards EBP, especially when practitioners make assumptions about what constitutes ‘the evidence’ and how the data informing that evidence are gathered.[2,3] The evidence upon which EBP is premised is usually derived from experimental research conducted in professional disciplines that are firmly rooted in the positivist paradigm; the research method most closely associated with this is the randomised controlled trial (RCT). RCTs are quantitative, controlled experiments in which the effect of an intervention can be determined more objectively than by observational studies.[4] There is no doubt that the method has utility in determining cause-effect relationships between medical treatments and patient outcomes, making it a powerful design for intervention studies with the objective of determining the influence of one variable on another.[5] In an educational context it may initially seem reasonable to expect that an experimental design could determine the effect of a teaching intervention that aims to improve student learning. The argument is that by using randomisation to average out the differences between students, one would be able to demonstrate which teaching and learning strategies lead to the largest effect sizes. These data, presumed to be free of subjective interpretation, could then inform policies that drive the implementation of effective teaching interventions.[6] However, if we assume that the evidence gathered via experimental research provides insight into an objective reality, we must take a position on teaching and learning that is at odds with our best explanations for how learning happens. Therefore, if we want to use RCTs in educational research, we must assume that there is a cause-effect relationship in the teaching and learning interaction that can be objectively measured. In this article we

May 2016, Vol. 8, No. 1 AJHPE

argue that RCTs are an inappropriate design choice for educational research because they force one to assume ontological and epistemological positions that are at odds with theoretically informed perspectives of learning.

RCTs in educational research

We begin by highlighting the biased way in which RCTs are positioned relative to other forms of research endeavours, explicit in the language employed by RCT proponents. Goldacre[7] suggests that ‘Evidence-based interventions in teaching could … replace the current system where untested methods are passed to teachers through a variety of often dubious outlets’, and ‘We need a slow revolution that puts evidence at the heart of teaching’. Torgerson[8] asserts that RCTs are the ultimate expression of evaluative research, referring to ‘the importance and supremacy of the RCT’, and expresses concern that educational research tends to rely on ‘manifestly inferior’ qualitative methods. It seems clear that those who most strongly advocate the use of RCTs in education have an inherent bias against other methods of data collection, strongly positioning themselves within a positivist interpretation of reality. This does not mean that RCTs and other forms of experimental research are not valuable tools in the repertoire of the researcher; randomisation is rightly considered an appropriate design choice in clinical trials. By controlling for the influence of all other variables between groups – through the assumed equal distribution of those differences in a large enough randomised sample – any differences in outcome can be more confidently attributed to the intervention.[5] As the RCT is a powerful tool successfully used in medical and clinical research, some have suggested that it should therefore underpin all ‘good’ research regardless of context’.[9] However, one cannot assume that RCTs can provide more – and better – evidence, which inevitably leads to improvements in education.[6] RCT proponents hope that these trials can do for educational researchers what they have done for medical researchers, i.e. provide clear-cut answers around the relative benefits of one intervention over another.[7] RCTs are presented as a gold standard, able to determine ‘the truth’ by simplifying and

Short Report generalising the complex social interactions of the educational context.[10] For example, Goldacre[7] suggests that by ‘… collecting better evidence about what works best, and establishing a culture where this evidence is used as a matter of routine, we can improve outcomes’. By positioning the RCT as the best way to collect ‘better evidence about what works best’, it has been suggested as a means by which educational practice can be improved, as it generates absolute facts about an existing reality that is objectively measured.[10] By choosing the RCT as a method of gathering data, the researcher is taking a stance within a framework that describes what they believe about what it means to know something in the world. The way researchers make decisions about which methodologies are useful is determined in part by their ontological and epistemological perspectives. A research methodology is not simply a neutral plan for designing a systematic inquiry, but is instead informed by a theoretical perspective. The selection of a research method is therefore a proxy for expressing a belief about what it means to know and our attempts to better understand what we know.[11] Therefore, our beliefs around our ways of knowing in the world influence how we choose to investigate them.[10] According to Grix,[12] research is best done by establishing a relationship between what a researcher thinks can be researched (the ontological position), what we believe can be known about it (the epistemological position), and how to go about acquiring it (the methodological approach). Thus, the influence of the ontological and epistemological position on what and how a topic is investigated is clear.[12] As researchers we are required to make explicit claims about how we view reality (ontology) and what constitutes knowledge (epistemology),[13] because these perspectives have a significant influence on the methodology chosen and ultimately on the outcomes of the research.[14]

Educational and clinical contexts differ

It may initially seem as if clinical and educational contexts are similar and that the processes in both are therefore susceptible to the same methods of investigation. However, there are fundamental differences that make it difficult to see how experimental methods are appropriate in the process of evaluating learning. While it could be argued that both clinical and educational contexts represent complex (open) systems,[8] this typically conflates the clinical trial, where all variables are carefully controlled, with a health system. While health systems are complex environments, it is clear that RCTs are not used to investigate complete health systems. Instead, their objective is to achieve generalisable simplicity by holding all else equal and determining the effect of a single variable, the outcome of which is applicable across a variety of contexts.[10] Therefore, the RCT represents an attempt to create an ‘artificially closed system’[15] whereby the relationships between variables in controlled, non-complex contexts are determined.[10] This is appropriate in the positivist ontology and epistemology and is therefore the reason that the RCT is an appropriate method of gathering data in clinical research. However, it is not possible to create controlled, non-complex educational environments that enable cross-context predictions.[10] Randomisation does not control for other sources of variation and confounding factors that are likely to be found in educational contexts. These include for example, factors that lead to differences in studying methods, changes in learner motivation, and effects of other, non-intervention experiences that occur during the implementation of the intervention.[5] It is impossible to create a closed education system, even an ‘artificially closed system’,[15] and therefore almost impossible to identify how much of the intervention the learner actually

‘receives’, or to determine what the learner does with what the teacher provides.[6] The factors that influence learning outcomes cannot be recreated in different contexts, or even in the same context at different times.[5] Education therefore exists in an open system, and even if we attempt to reduce and limit change and variation – internally and externally – we will not be able to determine causality.[15] RCTs assume that every implementation of the intervention is the same, that everyone receiving the intervention will be affected in the same way, and that giving and receiving the intervention is divorced from the reality of the individual personalities and institutional contexts of the participants. In educational research it is difficult to tightly control variables and blind subjects in ways that are ethical or feasible, and one cannot ‘apply curriculum daily’ in the same way that one can prescribe medication.[5] It is therefore impossible to control for confounding variables in learning environments, making the attempt to use RCTs in this context ‘hopelessly flawed’.[10] The beliefs of the researcher around the context in which the research will take place clearly inform the choice of method used to gather data, which in turn informs the outcome of the study. By conflating a clinical trial with research that seeks to determine the effectiveness of a teaching intervention on student learning, RCT proponents ignore the fact that clinical and educational contexts are fundamentally different and that these differences require different methods of gathering data. We now present an argument demonstrating how beliefs around knowledge and the nature of reality may contradict our understanding of how learning happens.

Beliefs around reality and knowledge

If one believes in an objective reality that is separate from the people conducting research into that reality, one is more likely to view knowledge as a quantity of something that is to be accumulated.[16] The belief that knowledge is separate from knowers and that it can be transmitted to others suggests that a positivist approach to educational research should be considered.[17] The focus is likely to be on using reliable and valid tools to collect quantitative data about the learning intervention that is regarded as value free.[18] Positivist research maintains that knowledge is objective, that it involves hypothesis testing and identifies causality.[19] In this understanding of the world, learning is focused on the teacher and concerned largely with the transfer of information to the learner.[11] Therefore, a positivist perspective on the nature of reality is most likely associated with a method of teaching and learning that considers knowledge as something that can be transferred between people. An alternative perspective on the nature of reality is that it is interpretive, subjective and different for each person.[20] In this context, knowledge is socially constructed by individuals interacting with each other and the world.[17] Thus, interpretive perspectives of reality involve an attempt to understand phenomena from an individual’s perspective,[10] and consequently recognise that in certain contexts it is not possible to determine causality. Instead, interpretive research focuses on trying to understand and explain reality from the unique vantage point of individuals.[21] If knowledge is socially constructed and experienced differ ently by individuals in different contexts, it is a conception of reality that is fundamentally at odds with positivism and therefore unlikely to be explained with positivist research methods. There are many theories that seek to explain how learning happens. Regardless of which theory one adopts, they all describe an interpretation and negotiation taking place between an individual and their unique

May 2016, Vol. 8, No. 1 AJHPE

Short Report context. This understanding of learning recognises that it influences and is in turn influenced by the context[22] in a complex relationship between knowledge, the knower and knowing.[23] There is therefore a distinction between conceptions of research that are atomistic and orientated towards external products with the intention of producing outcomes, and conceptions that are holistic and analytical with an orientation towards internal process es, where the intention is to understand.[11] If the purpose of educational research is to understand learning in a holistic way, rather than to control and predict for certain predetermined outcomes, then interpretive research methodologies may be more appro priate than experimental methods such as RCTs.[9] One suggestion to move the discussion forward is for researchers to avoid the ‘methodological tribalism’ that takes up so much of our attention and to embrace a pragmatic approach to research, where we use the tool that is the best fit for the problem we are addressing.[24]

Conclusion

RCTs are not neutral methods of gathering unbiased data that describe an objective reality. They are positioned within paradigms that come with certain ontological and epistemological perspectives about the world, which seem to be at odds with ontological and epistemological perspectives of learning. The RCT requires the researcher to adopt a particular stance in terms of their beliefs about the world, which is in contrast with our understanding about how learning happens. To use RCTs in educational research, we are forced to assume a positivist view of the world in which the learning context must be simplified and controlled, rather than accepting the complexity and inherent subjectivity of the nature of learning, and working in it. The use of the RCT in educational research is therefore a flawed design choice, as it betrays a flawed understanding of the nature of learning. It requires us to accept that there exists a set of ‘correct conditions’, and that by controlling for the ‘right’ variables we are able to control learning in

May 2016, Vol. 8, No. 1 AJHPE

the same way that we control clinical trials. The problem with RCTs in educational research is therefore not only that they may be ineffective and impractical when determining the value of a learning intervention, but that they also force us to take up ontological and epistemological positions in a technical rationalist framework that perceives the world as having a single truth, which is inconsistent with a real understanding of learning. References 1. Sackett DL, Rosenberg WMC, Gray JAM, Richardson WS. Evidence based medicine: What it is and what it isn’t. BMJ 1996;312:71-72. 2. Biesta GJJ. Why ‘what works’ still won’t work: From evidence-based education to value-based education. Studies in Philosophy and Education 2010;29(5):491-503. 3. Dimitriadis G. Revisiting the question of evidence. Cultural Studies – Critical Methodologies 2008;8(1):3-14. 4. Chalmers TC, Smith H Jr, Blackburn B, et al. A method for assessing the quality of a randomised control trial. Control Clin Trials 1981;2(1):31-49. 5. Sullivan GM. Getting off the ‘gold standard’: Randomized controlled trials and education research. J Grad Med Educ 2011;3(3):285-289. 6. Norman G. RCT = results confounded and trivial: The perils of grand educational experiments. Med Educ 2003;37:582-584. 7. Goldacre B. Building evidence into education. Department for Education, London. 2013. https://www.gov.uk/ government/news/building-evidence-into-education (accessed 6 January 2016). 8. Torgerson CJ. Educational research and randomised trials. Med Educ 2002;36:1002-1003. 9. Denzin NK. The elephant in the living room: Or extending the conversation about the politics of evidence. Qualitative Research 2009;9(2):139-160. 10. Regehr G. It’s NOT rocket science: Rethinking our metaphors for research in health professions education. Med Educ 2010;44:31-39. [http://dx.doi.org/10.1111/j.1365-2923.2009.03418.x] 11. Brew A. Teaching and research: New relationships and their implications for inquiry-based teaching and learning in higher education. Higher Education Research and Development 2003;22:3-18. 12. Grix J. The Foundations of Research. London: Palgrave Macmillan, 2004. 13. Maton K, Moore R. Social Realism, Knowledge and the Sociology of Education – Coalitions of the Mind. London: Continuum International Publishing Group, 2010:1-13. 14. Jackson E. Choosing a methodology: Philosophical underpinning. Practitioner Research in Higher Education Journal 2013;7(1):49-62. 15. Danermark B, Ekstrom M, Jakobsen L, Karlsson JC. Explaining Society: Critical Realism in the Social Sciences. London: Routledge, 2002. 16. Mourad RP Jr. Postmodern Philosophical Critique and the Pursuit of Knowledge in Higher Education. Westport, CT: Bergin and Garvey, 1997. 17. Crotty M. The Foundations of Social Research. London: Sage, 1998. 18. House E. Realism in research. Educ Res 1991;20(6):2-25. 19. Creswell JW. Research Design: Qualitative and Mixed Methods Approaches. London: Sage, 2009. 20. Guba EG, Lincoln YS. Competing paradigms in qualitative research. In: Denzin NK, Lincoln YS, eds. Handbook of Qualitative Research. London: Sage, 1994;105-117. 21. Cohen L, Manion L, Morrison K. Research Methods in Education. 6th ed. London: Routledge, 2007. 22. Marton F, Hounsell D, Entwistle N. The Experience of Learning, 2nd ed. Edinburgh: Scottish Academic Press, 1997. 23. Maton K. Knowledge-knower structures in intellectual and educational fields. In: Christie F, Martin J, eds. Language, Knowledge and Pedagogy: Functional Linguistics and Sociological Perspectives. London: Continuum, 2007:87-108. 24. Lamont M, Swidler A. Methodological pluralism and the possibilities and limits of interviewing. Qualitative Sociology 2014;37:153-171.

Short Report Standard setting and quality of assessment: A conceptual approach S S Banda, MB ChB (UNZA), MSc (Glasgow), MMEd (Dundee), PhD (UNZA), FAcadMEd (UK) Department of Medical Education Development, School of Medicine, University of Zambia, Lusaka; and School of Medicine, Cavendish University, Lusaka, Zambia

Corresponding author: S S Banda (ssbanda2007@gmail.com)

The debate with regard to standard setting continues among scholars, but the perfect method remains elusive. Nonetheless, resolute demands for accountability by means of defensible, valid and reliable practices, including pass or fail decisions, provide an opportune vehicle for scholars to consider the quality of assessments and ramifications on standard setting. This conceptual review considers how standard setting can be placed strategically in the quality dialogue space to address concerns about credibility and defensibility in the literature. Quality performance standards and the effect of assessment outcomes are important in the educational milieu, as assessment remains the representative measure of achievement of the prescribed performance standard but also includes the quality of health professions education (HPE). The author suggests that standard setting can be a pivotal focus for technical and psychometric sufficiency of assessments and accountability of HPE institutions towards stakeholders. Standard setting should not be seen as a methodological process of setting pass/fail cut-off points only, but as a powerful catalyst for quality improvements in HPE by promoting excellence in assessments. Afr J Health Professions Educ 2016;8(1):9-10. DOI:10.7196/AJHPE.2016.v8i1.712

The practice of deciding the pass/fail cut-off point is commonly referred to as standard setting.[1] However, many standard-setting practices, e.g. the historical 50% pass/fail cut-off point, can be difficult to defend psychometrically, yet they continue to be used despite rational disapproval in the literature.[2-7] Demands for accountability by means of defensible, valid, reliable and robust assessment policies and practices, including pass/fail decisions, are increasingly becoming important.[5,8] Discussions are being held on credible and defensible standard-setting practices[6,7] for high-stakes examinations. This debate presents a vehicle to discuss quality of assessment, which has a crucial role in promoting the quality of health professions education (HPE) institutions.[9] This conceptual review considers how standard setting can be placed strategically within the quality dialogue arena to ensure that appropriate action is taken to address concerns about credibility and defensibility in the literature. The medical education communities in African countries are small and need to join forces to contribute to the dialogue on standard setting and its potential to improve the quality of HPE. This article also provides the background to the purpose of assessments and sources of common standard-setting procedures. The last section gives a conceptual framework to embed standard setting in the HPE quality discourse.

Assessment purpose

Assessment categorises examinees into two groups – those who pass and those who fail – with regard to curriculum objectives, content and skills.[5] The final mark represents an examinee’s attainment on the performance continuum implied by proficiency levels[5] and is represented on a test score range from 0% to 100%. This process means that standard setting is the embodiment of the attain ment of learning objectives[10] and the pass mark is the operationalisation of the performance standard.[11] A decision has to be made whether the cut-off marks and the resulting pass/fail results are sufficiently valid and accurately representative of the intended interpretations assigned to them. For specific information about the technical process and in-depth review of each method,

several publications may be consulted.[1-3] There are ~50 standard-setting methods reported in the literature.[7] Some of the well-known methods include the Angoff method;[2,4,8,10] Ebel method;[1,2,10] Bookmark method;[12,13] Borderline-group method and contrasts by group approach.[4,6,7,10]

A conceptual approach

Plain, fair and valid standard setting cannot result from unfair and invalid assessments.[7] Validity, reliability, educational effect, feasibility and acceptability frequently constitute the criteria used to select assessment methods.[14] Primarily, validity and reliability have been central to decisions about the assessment methods used in different HPE settings. Validity focuses on whether a test succeeds in testing the competencies for which it was designed.[1,4] Reliability or generalisability is a measure of the relative size of variability in scores due to error, with the objective of reaching a desired level of measurement accuracy across different tasks.[14] Acceptability is the extent to which many stakeholders endorse the measure and interpretation of scores.[14] Blueprinting, which requires test content to be carefully planned against learning objectives,[1,5] has also been advocated. The focus should be on psychometric rigour.[14] Because test theory, e.g. classic test theory, item response theory, generalisability theory,[1] informs psychometrics, it must be a critical consideration in the assessment of planning, analyses and interpretation, especially with regard to reliability. The abovementioned considerations can be grouped into ‘assessment technical sufficiency’ matters. However, an important result of the standard-setting process is the pass/ failure rate, which directly or indirectly implicates the quality of instruction, instructors and graduates. The reason is that assessment is the symbolic measure of confirming achievement of the prescribed performance standard. Together, these can be considered ‘impact of assessment outcomes’. Educational tension, therefore, arises between the expectations of quality of assessment, performance standards and impact of assessment outcomes.[7] Accordingly, technical sufficiency of assessment practices is not enough – impact considerations are as important. The close relationship between quality of assessment and standard-setting practices and their effect on HPE account abilities is neglected, poorly understood and underdeveloped at many HPE

May 2016, Vol. 8, No. 1 AJHPE

Research Short Report

Item response theory

Classic test theory

Generalisability theory

Test theory

Validity

Reliability

Educational effect

Acceptability

Feasibility

Framework for selection of assessment methods

Crocker and Zieky[7] noted that because of the judgemental nature of standards, validity was dependent on how sensible the standard-setting process and its outcome were, together with the consequences of pass rates and possible classification errors. The effect of assessment outcomes therefore extends to political, economic, social and policy domains.[1,7,11] Notably, decisions about the effect of assessments on accountability have to depend on the decision-making theory[7] or alternative decision theories.

Conclusion Assessments Purpose of assessment Formative Summative

Standard setting Proficiency standards Pass/fail cut-off marks Social constructivism

Assessment outcomes Failure rates Grade point average

Measurement error

Decision theory

References

Effect accountability to stakeholders

Political

Economical

Quality assurance

Social

Equity

Fig. 1. Conceptual framework of standard setting as quality assurance catalyst.

institutions. The ultimate key determinants of pass rates are the quality of the assessments and the pass/fail standard-setting practices and decisions. In the context of their validity, educators must be aware of this relationship as they plan, implement and interpret assessment outcomes. Consequently, this article proposes that standard setting should be seen as a critical aspect of the assessment structures and processes. It is dependent on assessment quality, as a test that does not cover appropriate content is not at the appropriate level of difficulty, is not reliable, and will not lead to appropriate decisions, regardless of the standard-setting methods employed.[11] However, educational assessments

not only comprise technical sufficiency; attention must also be paid to the accountability of the HPE institution to various stakeholders. The holistic representation is captured in the conceptual approach (Fig. 1), summarising key principles and concepts in a relational method. Standard setting can be a pivotal focus to attend to technical and psychometric sufficiency of assessments as well as accountability responsibilities of HPE institutions to stakeholders. Although standard setting is an important psychometric problem, it is not solely a technical issue.[1,11] The consequences of appropriate or inappropriate standards for society, institutions and individuals must be considered.[1]

May 2016, Vol. 8, No. 1 AJHPE

Standard setting should not be seen as a methodological process of setting the pass/fail cut-off point only, but as a powerful catalyst for quality improvement in HPE by promoting assessment excellence. While medical education departments can be important catalysts with regard to capacity and quality, in Africa, for example, their development remains basic. Therefore, numerous academic staff and policy makers should join the standard-setting dialogue and in particular its potential to improve the quality of HPE.

1. Crocker L, Algina J. Introduction to Classical and Modern Test Theory. Boston, MS: Wadsworth, 2006. 2. Cizek GJ. Setting Performance Standards: Foundations, Methods, and Innovations. London, UK: Routledge, 2012. 3. Cizek GJ, Bunch MB. Standard Setting: A Guide to Establishing and Evaluating Performance Standards on Tests. Thousand Oaks, CA: Sage, 2007. 4. Ben-David MF. Standard setting in student assessment. Med Teach 2000;22(2):120-130. 5. Hambleton RK. Setting performance standards on educational assessments and criteria for evaluating process. In: Cizek GJ, ed. Setting Performance Standards: Concepts, Methods, and Perspectives. Mahwah, NJ: Lawrence Erlbaum, 2001:89-116. 6. Zieky MJ, Piere M, Livingston S. Cutscores: A manual for setting standards of performance on educational and occupational tests. 2008. http://www.amazon.com/cutscores-standards-performance-educationaloccupational/dp/1438250304 (accessed 4 July 2015). 7. Crocker L, Zieky M. Proceedings of the Joint Conference on Standard Setting for Large-Scale Assessment, Washington, DC, 5 - 7 October 1994. Washington, DC: National Center for Education Statistics, 1995. 8. Kilminster S, Roberts T. Standard setting for OSCEs: Trial of borderline approach. Adv Health Sci Educ Theory Pract 2004;9:201-209. 9. Barman A. Standard setting in student assessment: Is a defensible method yet to come? Ann Acad Med Singapore 2008;37:957-963. 10. Ricker K. Setting cut scores: Critical review of Angoff and modifiedAngoff methods. Alberta Journal of Educational Research 2006;52(1):53-64. 11. Norcini JJ, Banda SS. Increasing the quality and capacity of education: The challenge for 21st century. Med Educ 2011;45:81-86. [http://dx.doi. org/10.1111/j.1365-2923.2010.03738.x] 12. Karantonis A, Sireci SG. The Bookmark standard setting method: A literature review. Educational Measurement: Issues and Practice 2006:4‐12. 13. Buckendahl C, Smith R, Impara J, Plake B. A comparison of Angoff and Bookmark standard setting methods. Journal of Educational Measurement 2002;39(3):253-263. 14. Norcini JJ, McKinley DW. Assessment methods in medical education. Teaching and Teacher Education 2007;23:239-250.

Research Medical students’ views on the use of video technology in the teaching of isiZulu communication, language skills and cultural competence P Diab,1 MB ChB, MFamMed; M Matthews,2 MB ChB, DOH, MPH; R Gokool,3 BA, BA Hons, MA 1

Discipline of Rural Health, School of Nursing and Public Health, University of KwaZulu-Natal, Durban, South Africa

School of Clinical Medicine, Nelson R Mandela School of Medicine, University of KwaZulu-Natal, Durban, South Africa

Discipline of African Languages, School of Arts, College of Humanities, Howard College Campus, University of KwaZulu-Natal, Durban, South Africa

Corresponding author: P Diab (diabp@ukzn.ac.za)

Background. The role of communicator has been included as a key competency for health science students in South Africa. Owing to the population’s diverse language and cultural backgrounds, communication between patients and healthcare professionals is challenging. In this study, the Attention, Generation, Emotion and Spacing (AGES) neurocognitive model of learning was used as a framework to create videos for language teaching for the vocational needs of students. Objectives. To explore students’ views on the use of videos of simulated clinical scenarios for isiZulu communication and language teaching and the development of cultural awareness. Methods. Videos were developed using first- and second-language isiZulu speakers with scripts (verified by the university’s Language Board) based on authentic clinical settings. Videos were shown to a target group of students, who were then interviewed in focus group discussions. Audio recordings from the discussions were transcribed and analysed thematically in three categories, i.e. communication, language skills, and cultural awareness, using deductive coding based on the objectives of the research. Results. Students affirmed numerous benefits of the videos and commented on their use and further development. Benefits described related well to the AGES model of learning and fulfilled the learning requirements of communication teaching, language acquisition and cultural awareness. Conclusion. The videos represent an innovative teaching method for the resource-constrained environment in which we work and are relevant to the 21st century learner. Further evaluation and development of the tool using different scenarios and African languages is recommended. Afr J Health Professions Educ 2016;8(1):11-14. DOI:10.7196/AJHPE.2016.v8i1.402

The University of KwaZulu-Natal (UKZN), Durban, South Africa (SA) has adopted a list of core competencies for medical students, derived from the Royal College of Physicians and Surgeons of Canada Physician Competencies (CanMEDS),[1] which includes as one of its key facets the role of communicator. Some of the objectives are to develop rapport and trust between patients and healthcare providers from different cultural backgrounds, with a particular mention of respect for ‘diversity and difference, including … the impact of ethnicity, gender, religion, education and culture’.[2] Training in clinical communication at UKZN follows the CalgaryCambridge method,[3] which stresses the importance of content (what information is obtained) and process skills (how information is obtained) in a medical interview, and places emphasis on a patient-centred method in which biomedical, contextual and patient perspectives are explored.[4] However, as a Eurocentric model, it has been suggested that the method is not useful in a multicultural and multilingual society such as SA. The authors believe that such a model can be used in the SA context by stressing the importance of the patient’s perspective and the meaning that his culture brings to the consultation. The 2011 census showed that nearly 78% of people living in KZN speak isiZulu, one of the province’s official languages, as a home language.[5] The UKZN’s bilingual language policy, approved in 2006, proposes that the university develops the use of isiZulu as a language for communication, with particular emphasis on ‘professional/vocational training for undergraduate students’.[6] In a study by Matthews,[7] the current isiZulu module for 1st-year

medical students, which provides basic vocabulary and grammar skills, has been shown to be insufficient to equip students to communicate effectively with their patients in a clinical setting. Matthews recommended that a vertically integrated isiZulu course for medical students be introduced to reinforce learning, concentrating on basic interpersonal communicative skills and appropriate terminology and vocabulary for a medical interview. Neurocognitive theory has become an important framework for planning and guiding the development of teaching material in many disciplines and has been used as such for the isiZulu video project. The Attention, Generation, Emotion and Spacing (AGES) neurocognitive model describes how hippocampal activation improves long-term retention and retrieval of learnt material.[8] The four features of the AGES model consider the following aspects of learning: getting information into the brain and encoding it; retention of memories; and retrieval of memories.[8] For learning to occur, full attention must be paid to the topic to be learnt, with concentrated focus on the task. Davachi et al.[8] suggest that one method of doing so is to make learning situations as authentic as possible, e.g. with the use of advanced simulations. These authors state that the generation of memories or recall is optimised when learners are able to ‘contextualise, retain and apply knowledge in their own way’, and that using methods which are thought to be entertaining or novel may invoke positive emotions that enhance learning.[8] The final part of the model describes how spacing (distributing learning over time) and revisiting content lead to better longterm memory.[8] The use of videos as an audiovisual component in second-language teaching has been shown to be successful and relevant to the needs of learners.[9] Further

May 2016, Vol. 8, No. 1 AJHPE

Research more, videos have been demonstrated to be useful in teaching language for a specific purpose, i.e. for the vocational requirements of medical students.[10] Other advantages of video technology in teaching communication are that it contains visual (body language) and auditory (speech patterns and rhythms) content.[9] It also has the advantage of being able to be offered on various platforms, such as learning management systems and public platforms, e.g. YouTube, at a time and place convenient to the viewer,[11] and creates a link between theoretical teaching and its practical application.

Objectives

The general objective of the study was to explore students’ views on the novel approach of using video clips as a language teaching tool in the context of the AGES model. Specific objectives were to explore how the videos improved their learning of communication skills, isiZulu language skills and awareness of the cultural context of the consultation.

Methods

The authors, being integrally involved in communication training at UKZN, identified the need to develop a means of communication teaching, where both English and isiZulu were used, which would be able to teach language and communication skills simultaneously. Where possible, scripts were developed in isiZulu by second-language speakers, relying on their experience in authentic clinical settings. These were checked for grammatial and language accuracy by an isiZulu tutor and other first-language isiZulu speakers, including the language co-ordinator of the university. The scenarios are representations of real life, although none is factual in content or person. They focus on various clinical presentations and disease entities, and use the CalgaryCambridge framework. For the purpose of this study, only one of the videos was shown – a scenario involving a young woman who presented with lower-abdominal pain and menstrual problems, symptoms located in a cultural context of the patient to prove her fertility. The ‘patients’ in the videos were actors trained in communication skills at the UKZN clinical skills department. The ‘doctors’ were authors and colleagues comfortable in second-language isiZulu, but with no formal training in the language at an academic level. The videos were filmed in a simulated setting by a professional videographer. Final-year students in their Family Medicine rotation took part in the study. They were exposed to the video, and their comments with regard to communication skills, language and cultural learning were recorded in focus group discussions (FGDs) and a self-reflection questionnaire (QUEST). Ethical permission to perform the study was granted by the UKZN Humanities and Social Sciences Research Ethics Committee (HSS/0312/013). Permission was obtained from all relevant gatekeepers and individual consent from participating students. Audio recordings of the FGDs were transcribed and thematically analysed using inductive coding with the assistance of NVivo (version 9) (USA).

Results

Responses from students with regard to the video tool were overwhelmingly positive. Only one student, who admitted to having a particularly poor understanding of isiZulu, felt that she obtained very little benefit from the video as she had not paid attention to a subject that was not examinable at

May 2016, Vol. 8, No. 1 AJHPE

the time and that she found uninteresting. In general, students embraced the idea of using technology and new learning tools to enhance learning of doctor-patient communication skills and to assist with learning of a second language. Although students were not specifically informed of the AGES model of learning, they expressed the following comments about the videos in general. ‘I think the videos are actually quite good because I do not remember half the scenarios they gave us at school because they were written. You know, we are not paying attention, we dozed off, but I mean if you have seen it and remember it, I do not know, maybe it will be better.’ (FGD2) ‘ And then if they make it available thereafter that is often a good reinforcement because it is one thing watching the video and then just going home, and then another thing watching it and then having to go over it and even watch it again when you have practised because then you actually realise how much you had forgotten though we can then pick up.’ (FGD2) ‘ It will possibly also find ways of how to make Zulu itself interesting as a language.’ (FGD2)

Communication skills

Students mentioned that they felt the videos demonstrated the importance of communication skills in terms of building the doctor-patient relationship. ‘How the doctor was communicating with the patient – it is something that we are taught … but something we overlook … if you are watching it time and time again then you learn.’ (FGD2) They also highlighted the importance of various process and non-verbal skills important in communication teaching, which were evident in the videos. ‘The encouragement.’ (FGD3) ‘… and eye contact.’ (FGD3) ‘ Your expression and body language … will determine your relationship with the patient in terms of trust.’ (FGD3)

Language skills

Students generally articulated the need to be able to communicate in isiZulu with their patients. ‘It automatically creates a bond between doctor and patient.’ (QUEST 11) The importance of the spoken language was highlighted. ‘We will not be writing notes in Zulu in as much as we will be taking a history in Zulu. So it is the speaking part that is most important.’ (FGD1) Students thought that another advantage of the videos was the learning of vocabulary and specific keywords and phrases within a specific context. The combination of hearing words in context and being able to relate them to the written word on the scripts was found to be very useful. ‘I think it is good, excellent to link it [language teaching] with video scenarios. If it is in context we can remember it better rather than getting a list of vocab.’ (FGD2)

Research ‘ Common words, common questions like “When did it start?”, “Where does it refer to?” … simple things like that really would help.’ (FGD3) In addition to second-language isiZulu speakers identifying the advantages the videos could have for them, first-language isiZulu speakers felt that the videos could benefit their understanding of the language, especially in a medical context. ‘I am Zulu. I am from a Zulu background. But there are certain things that I cannot explain to patients in Zulu because I do not know the terminology.’ (FGD2)

Enhancing cultural awareness

Students identified another objective as enhancing cultural awareness in consultations. They related many episodes where they had been exposed to situations in which an awareness of the patient’s cultural beliefs was vital to the consultation and reflected their need to improve their own cultural sensitivity. ‘I think you should be teaching [cultural awareness] because it is so common. It comes up so much wherever you are.’ (FGD2) The scenario shown to students highlighted a young woman who wanted to prove her fertility to her betrothed. One student accurately summarised the meaning of the scenario portrayed: ‘I think she was scared that if she does [not] have babies her husband is not going to marry her and I think the doctor acknowledged that and said she will try her best. And she did not judge her because her husband was going to leave her. She did not say what kind of husband is that who is going to leave you when you need him the most and stuff like that.’ (FGD3)

Technical aspects of the videos

To develop the videos for future teaching, students were specifically asked about recommendations on technical aspects of development, utilisation and availability. ‘[the video] needs to be very clear and quite slow.’ (FGD3) ‘ Something like tutorials where students are broken into small groups and are given a scenario which two people simulate – one is a patient and the other one a doctor.’ (FGD1) ‘ … the videos should be easily provided to the students that they can download from the website or to take home, watch on the computer and actually read the words … that would be a much easier and faster way of learning.’ (FGD1) ‘ To have them available on your cell phone. Oh! That would be amazing!’ (FGD2) Students debated whether the videos should have subtitles on the screen, or if a separate script, with translation, should be made available. ‘If the subtitles are there you will end up focusing on them because you will end up not listening, just reading.’ (FGD3) ‘ I think maybe the video and the printed [script]. If you have both of them you can hear how the word sounds and then you know.’ (FGD2)

Discussion

The main objective of this article was to present and discuss students’ views on the use of video clips of simulations of authentic clinical situations to enhance communication and language skills in isiZulu and develop cultural awareness for a medical consultation in line with UKZN’s Language Policy and Plan. Results of this exploratory study suggest that the use of such videos has many benefits. Students indicated that the videos captured their attention and most expressed their interest in and enthusiasm for this teaching method. Further more, students of the 21st century are technoliterate and very familiar with the use of audiovisual content, which many of them interact with and share on a daily basis. They articulated that hearing the language and observing visual cues while watching the relevant material in simulations of an authentic clinical setting would probably assist in the generation of new knowledge compared with traditional didactic teaching. The use of emotion to capture and hold a student’s attention is highlighted as a trigger to make learning more interesting and relevant – a concept supported by other authors.[8,12] The notion of spacing of content is well supported in the literature[8,13] and by students, who remarked on the advantages of being able to revisit content on the videos as frequently as required. By revisiting and reflecting on past experiences, students are able to make meaning of what they have learnt in the preclinical years and adapt it for future use in clinical settings. Using video technology to teach communication skills through simulated scenarios has been reported to be successful.[14-16] However, there is not much literature on the use of videos in second-language teaching. Students sampled in our study noted that using the videos to identify and highlight process skills is particularly beneficial in a second-language context. Students became sensitised to and remarked on the effectiveness of body language in engaging the patient, and realised that a willingness to use the patient’s language and simple techniques in the interview can improve the doctor-patient relationship and interaction. Students highlighted the importance of being able to listen to isiZulu as a spoken language, which would benefit them in learning pronunciation and specific keywords and phrases for their context. Second-language teaching places much emphasis on listening and speaking compared with writing or reading the language. Video technology has been shown to be very effective for this purpose.[9,17] Moreover, the use of audiovisual media to enhance pronunciation, especially with regard to vocational-specific learning, has been documented in other international studies.[18,19] Of particular interest was that first-language isiZulu speakers felt they would also derive learning from exposure to the videos, especially with regard to medical terminology. In addition to the teaching of language skills, the videos provided a valuable opportunity for sensitising students to cultural influences in a medical consultation. Part of understanding the patient’s perspective involves an appreciation of the patient’s life-world, which includes his understanding, cultural beliefs and past personal experiences.[20] Students indicated that an awareness of their patients’ cultural perspectives will contribute to a shared understanding of concepts in the consultation and its outcome.

Study limitations

As this was an exploratory study, it is limited in nature. Only one video clip was shown to final-year students in the Family Medicine rotation. No formal evaluation has been conducted yet, although such plans are in place for 2014. (These evaluations have been commenced, but are still ongoing at

May 2016, Vol. 8, No. 1 AJHPE

Research the time of publication.) With regard to the use of the video in the learning environment, it has been noted that a basic knowledge of the isiZulu language and some technical ability are essential.

representations of authentic clinical scenarios to fulfil these requirements requires further investigation and evaluation.

Recommendations

References

Owing to much interest and enthusiasm that this study has generated among students, the videos were presented to other role players including the Medical Students Representative Committee, Information and Communication Technology Services, language experts and members of the College of Health Sciences, UKZN. As a consequence of engagement with these role players, it was decided to formally include the series of videos in the curriculum. We have suggested that further research in implementing and evaluating the use of videos as a resource to enhance teaching and learning in the programme is undertaken. Additional funding should be made available for the development of the project. It is hoped that this study will trigger dissemination of the teaching tool to various disciplines, schools and other institutions in SA.

Conclusion

The value of teaching isiZulu as a second language in vocational training is not disputed and communication skills have become a core competency of health education. Because of current resource limitations, innovative teaching methods are required to deliver the necessary content. Learning should also be relevant to the 21st century learner in a format that triggers an emotive and lasting response in the brain. The use of recorded, simulated

May 2016, Vol. 8, No. 1 AJHPE

1. Frank JR, Danoff D. The CanMEDS initiative: Implementing an outcomes-based framework of physician competencies. Med Teach 2007;29(7):642-647. [http://dx.doi.org/10.1080/01421590701746983] 2. Health Professions Council of South Africa (HPCSA). Core Competencies for Undergraduate Students in the Clinical Associate, Dentistry and Medical Teaching and Learning Programmes in South Africa. Pretoria: HPCSA, 2012. 3. Silverman J, Kurtz S, Draper J. Skills For Communicating with Patients. 2nd ed. London, UK: Radcliffe Publishing, 2005. 4. Kurtz S, Silverman J, Benson J, Draper J. Marrying content and process in clinical method teaching: Enhancing the Calgary-Cambridge guides. Acad Med 2003;78(8):802-809. 5. Statistics SA. Census 2011 – The languages of South Africa, 2011. http://www.southafrica.info/pls/procs/iac (accessed 8 April 2013). 6. University of KwaZulu-Natal. Language Policy and Plan of the University of KwaZulu-Natal. Durban: UKZN, 2013. 7. Matthews M. Vocation-specific isiZulu Language Teaching and Learning for Medical Students at the University of KwaZulu-Natal. Durban: UKZN, 2013. 8. Davachi L, Kiefer T, Rock D, Rock L. Learning that lasts through AGES. NeuroLeadership Journal 2010;3:53-63. 9. Canning-Wilson C, Wallace J. Practical aspects of using video in the foreign language classroom. Internet TESL Journal 2000;6(11):36-1. 10. Belcher DD. Trends in teaching English for specific purposes. Ann Rev Appl Linguistics 2004;24:165-186. 11. Topps D, Helmer J, Ellaway R. YouTube as a platform for publishing clinical skills training videos. Acad Med 2013;88(2):192-197. [http://dx.doi.org/10.1097/ACM.0b013e31827c5352] 12. LeDoux JE. Emotion, memory and the brain. Sci Am 1994;270(6):50-57. 13. Litman L, Davachi L. Distributed learning enhances relational memory consolidation. Learn Mem 2008;15(9):711-716. 14. Hurtubise L, Martin B, Gilliland A, Mahan J. To play or not to play: Leveraging video in medical education. J Grad Med Educ 2013;5(1):13-18. [http://dx.doi.org/10.4300/JGME-05-01-32] 15. Fertleman C, Gibbs J, Eisen S. Video improved role play for teaching communication skills. Med Educ 2005;39(11):1155-1156. [http://dx.doi.org/10.1111/j.1365-2929.2005.02283.x] 16. Seif GA, Brown D. Video-recorded simulated patient interactions: Can they help develop clinical and communication skills in today’s learning environment? J Allied Health 2013;42(2):e37-e44. 17. Levy M. Technologies in use for second language learning. Modern Language Journal 2009;93(s1):769-782. 18. Al-Jarf R. Online videos for specific purposes. J Educ Social Res 2012;2:17-21. 19. Cakir I. The use of video as an audio-visual material in foreign language teaching classroom. Turkish Online J Educ Technol 2006;5(4). 20. Candlin S, Roger P. Communication and professional relationships in healthcare practice. Equinox 2013:201.

Research Effect of curriculum changes to enhance generic skills proficiency of 1st-year medical students D Murdoch-Eaton,1 MBBS, MD, FRCPCH; A J N Louw,2 BEd, MEd, PhD; J Bezuidenhout,2 MB ChB, MMed, PhD 1

Medical Education, Medical School, University of Sheffield, UK

Centre for Health Professions Education, Stellenbosch University, Cape Town, South Africa

Corresponding author: A J N Louw (ajnlouw@sun.ac.za)

Background. Curriculum review is a dynamic, iterative process, and the effect of change may not always be wholly predictable. At Stellenbosch University, Cape Town, South Africa, revision of the MB,ChB curriculum was undertaken to meet enhanced and changing educational and medical practice, and to provide opportunities to enhance optimal generic skills underpinning effective learning, implemented in 2008. Objective. To determine the extent to which the newly implemented revised curriculum had an effect on experience in necessary generic skills of students in their first year of study. Methods. Students provided annual formal end-of-module evaluation in addition to focus group interviews. Evaluation by teaching staff was conducted by individual in-depth interviews. A validated generic skills questionnaire completed at the end of each academic year monitored the effect on students’ generic learning skills experience. Results. Feedback from these different evaluation methods identified specific needs in the newly implemented revised curriculum, including contextualisation of interventions, unnecessary duplication of content and malalignment of assessment. This led to minor curriculum changes and an educational capacity-building programme. These responsive curriculum changes after evaluation had the intended positive effect on students’ selfreported acquisition of generic learning skills. Conclusion. The objective of the curriculum evaluation was to monitor content output and the acquisition of crucial generic learning skills. Implementation of a revised curriculum combined with ongoing responsive changes aligned with careful multimodality evaluation can ensure that, in addition to scientific knowledge and skills, generic learning skills development of students is facilitated. Afr J Health Professions Educ 2016;8(1):15-19. DOI:10.7196/AJHPE.2016.v8i1.414

In a constantly and rapidly changing environment, the training of competent, caring and committed healthcare professionals requires a continuous cycle of curriculum revision, implementation and evaluation. This is crucial to keep up to date with changes in educational practice and advances in medical knowledge. Society expects that medical graduates function as healthcare change agents.[1] There is an increasing awareness of the centrality of generic learning skills underpinning success at university and fostering effective life-long learning.[2] Many higher educational courses have introduced explicit generic skills training in their programmes to ensure that these skills are embedded early.[3] Globally, school curricula strive to ensure that such skills are introduced. It is recognised that students are most vulnerable during the crucial transi tion period from high school to university.[4] The skills profiles of medical entrants are not static; Whittle et al.[5] identified a changing profile of the key skills that underpin learning. Furthermore, a widening access agenda means that students come from a diverse range of educational backgrounds, some of which almost certainly mean that the students are potentially educationally disadvantaged.[6] This necessitates sufficient support within a curriculum designed to ensure knowledge and skills acquisition relevant to the course, and identifies and addresses deficiencies in students’ learning and other generic skills. Curricula need to ensure incorporation of opportunities to practise these skills, and importantly for students, to obtain feedback.[2,6] Comprehensive evaluation is essential to understand the effect of the curriculum, including aspects introduced specifically for generic learning skills acquisition. These should include review of the needs of students,

ensuring that the necessary generic capabilities and sufficient practice are provided.[7] This requires a regular, cyclical process of curriculum evaluation, especially after significant change, to allow embedding and re-evaluation of whether the desired effect has been achieved and further changes are needed. The purpose of this study was to evaluate the effect of a newly revised curriculum, including the acquisition of generic skills underpinning effective learning, on 1st-year students at the Faculty of Medicine and Health Sciences (FMHS), Stellenbosch University (SU), Cape Town, South Africa (SA).

Methods Context

The study population consisted of all 1st-year medical students at the FMHS, SU from 2007 to 2011. Major curriculum revisions were undertaken in the early years of the 6-year MB,ChB programme and implemented from the beginning of 2008 (Table 1). The main purpose of the revision was to introduce – in the first semester after arrival at university – curriculum elements focusing on a combination of generic skills and basic practice skills integrated within the sciences in an interdisciplinary, SA medical education context. Modules emphasised generic skills, i.e. that students need not only succeed in their studies, but also be responsible professionals. These skills comprise inter alia academic literacy, acquisition of an additional language, stress management, study skills, and how these are relevant in the context of ethics, professionalism, biostatistics, epistemology, and interdisciplinary behaviour.

May 2016, Vol. 8, No. 1 AJHPE

Research Table 1. Summary of medical curriculum changes during the study period 2007

2008

2009

2010

2011

First-semester modules

Chemistry Biology Data management Physics

Chemistry for health sciences Personal and professional development Health in context Life forms and functions of clinical importance

Second-semester modules

Cell and tissue haematology and immunology Respiratory system Cardiovascular system Ethics

Essentials of disease processes Principles of therapy

Essentials of disease processes Principles of therapy Introduction to clinical medicine

Position natural sciences in a health context Reduce science content Introduce two generic skills modules Replace systemic modules in second semester with basic principles of pathological and therapeutic processes

Reduce content overlap and duplication between modules Decrease number of lectures Emphasise contextualising of modules within the health sciences Improve quality of study guides Change to continuous assessment in two of the first-semester modules

Restructure and contextualise chemistry for health sciences Implement computer skills training Contextualise assessments across modules (horizontal integration) Dedicated time slots for teamwork Implement introduction to clinical medicine module Staff changes and capacitybuilding short courses

Minor refinements in all modules

Significant changes

Curriculum evaluation

Formal end-of-module evaluation by students was undertaken using a standard questionnaire, including closed questions (a 3-point Likert scale) and open-ended questions. Additionally, two focus group discussions (FGDs) were conducted annually (end of first and second semesters; 8 - 10 students per FGD). Teaching staff conducted evaluation by individual in-depth interviews (IDIs). At the end of 2008, an accreditation visit by the Health Professions Council South Africa (HPCSA) provided external evaluation.

Generic skills evaluation

The effect of the studentsâ&#x20AC;&#x2122; generic learning skills experience was monitored using a validated generic skills questionnaire completed at the end of each academic year.[8] Students rated their frequency of practice (1 = never, to 4 = every week) of 31 key generic skills, grouped into six categories, i.e. information handling, technical and numeracy, information technology (IT), time management, managing own learning and presentation skills (Table 2). The questionnaire

has been validated and shown to be reliable in the SA context, with stability of items within the skills categories verified by factor analysis.[9] Data were analysed with the Statistical Package for the Social Sciences (SPSS) 17 (USA), using analysis of variance (ANOVA) to investigate differences between year groups. While the generic skills questionnaire includes evaluation of confidence, for the purpose of this curriculum experience study the emphasis was on practice. The generic skills profile of students entering SA medical schools, including SU, over the period of this study, seems according to studies in progress not to be significantly different across and within institutions.

Ethical approval

Ethical approval was obtained from the Human Research Ethics Committee, FMHS, SU (N07/03/05). Informed consent was obtained from all the participants before participation in the study.

Results

Enrolment ranged from 230 students in 2007 to 264 in 2011 (total number of students over

May 2016, Vol. 8, No. 1 AJHPE

the 5 years: 1 161). Eighty-five percent were firsttime students (direct entrants) aged 18 years, the remainder having undertaken prior graduate studies. The ethnic diversity of the population did not alter significantly over the study period.

Curriculum evaluation and modifications A comprehensive semester evaluation report is compiled annually, including key aspects of good practice, in addition to areas of concern identified from student and staff feedback, and was used to inform minor curriculum interventions in 2009, which required immediate implementation. Full impact evaluation over 2 years informed more substantive curricular change, implemented in 2010 (Table 1).

Contextualisation, alignment and assessment From the FGDs it emerged that students were not satisfied with the relevance of the modules with regard to their future professions and considered that assessments were not wholly aligned with outcomes.

Research

Table 2. Thirty-two generic skills evaluated by the questionnaire, grouped into six main categories[8] Categories

Skills

Information handling

Researching a new topic using library resources Selecting information Interpreting information Using information to solve problems or answer questions

Technical and numeracy skills

Performing laboratory experiments Designing your own experiments Analysing experimental data, e.g. graphs Drawing conclusions from your data Calculations Statistics

IT skills

Word processing Spreadsheets Databases Using the internet to find information Using email

Organisational skills

Managing your time/meeting deadlines Planning tasks Thoroughness/accuracy Teamwork

Managing your own learning

Coping with stress Learning from other students Receiving feedback positively Giving constructive feedback Taking responsibility for your own learning Evaluating your strengths and weaknesses

Presentation skills

Essay writing Writing laboratory reports Explaining ideas Giving oral presentations Communicating with other scientists/doctors Communicating scientific/medical ideas to non-scientists

‘ It doesn’t make sense! It feels if we learn nothing, and nothing is applicable to us … the testing is dumb. Parrot learning teaches you very little and all these assessments required parrot learning instead of application of knowledge.’ (FGD10B St4 2008) ‘ There are too many themes that waste our time’ and ‘… the relevance of some parts of this work was not clear to me.’ (FGD1 St3 2009) ‘ … generally it’s been said that they [students] feel like some of the modules are useless because we are focusing on the main thing that we are here for, and that is medicine.’ (FGD10 St5 2008) Students’ comments during FGDs and the formal module feedback at the end of each module contributed to recognition of a need for several changes. The key interventions focused around improving the contextualising topics, especially chemistry and generic learning skills; alignment of assessment and outcomes; use of integrated single assignments to assess more than one

theme; and change from end-of-semester to continuous assessment in two modules. Methods for enhanced contextualisation included starting the semester with role-play by teaching staff and senior faculty management demonstrating the relevance of all four modules, followed by small group discussions and a debriefing, an explanation at the start of each theme of the relevance of that specific theme, and the continuous use of real-life examples.

Curriculum mapping

Reducing overlap, fostering integration, and eliminating conflicting messages reflected in the study guides were at the heart of curriculum mapping interventions. Prior to the curriculum interventions students commented: ‘Some content between the modules and different themes overlap and repeats.’ (FGD5 St5 2009) ‘ The themes in the module were very random and the information wasn’t continuous and it was frequently repeated and didn’t seem relevant.’ (FGD8 St7 2009)

Educational capacity development

The need for educational capacity development was illustrated by the following quote from a module chair: ‘Some lecturers feel overwhelmed by large class groups and experienced it as dominating.’ (IDI3) Some students felt as follows: ‘… the lecturers I think can still be a bit more interactive with us, and not just stand there and do their lecture slides.’ (FGD1 St4 2009) This was echoed by other teaching staff and resulted in the development and implementation of a very successful short course on interactive teaching, with an emphasis on large classes.

HPCSA evaluation

In 2008 the external evaluation panel of the HPCSA highlighted a number of positive aspects of the newly implemented 1st-year curriculum: the interdisiplinary nature of this phase was commended, especially as it does not exist elsewhere in SA; the emphasis on teamwork and establishing the concept of a health team in the first year; the development of commu nications skills, stress management and academic literacy; and the use of case studies on the ‘Essentials of disease processes’ module in the second semester. Recommendations were made to improve the extent of integration of content in the four modules, align assessment with outcomes, and be vigilent for ‘content overload’.

Generic skills evaluation

Responses were obtained from 1 002 students at the end of each academic year over the study period (86.2% response rate). The new curriculum, from implementation in 2008, showed a sustained and positive effect on students’ practice in information handling skills (Fig. 1). Technical and numeracy skills, however, initially showed a negative effect, with significantly reduced reported practice; curriculum changes implemented in 2010 in response to this led to a reversal. Similarly,

May 2016, Vol. 8, No. 1 AJHPE

3.35

3.30

3.25 Mean rating

Mean rating

Research

3.20 3.15

3.25 3.20

3.15

3.10 3.05

3.10

End 2007

End 2008

End 2009

End 2010

End 2011

End 2007

End 2008

Academic year

End 2009

End 2010

End 2011

Academic year

Fig. 1. Information-handling skills (practice).

Fig. 2. Information technology skills (practice).

3.75

3.45

3.65

Mean rating

3.70

3.60

3.40

3.35

3.55

3.30 3.50

3.25

3.45 End 2007

End 2008

End 2009

End 2010

End 2011

Academic year

End 2007

End 2008

End 2009

End 2010

End 2011

Academic year

Fig. 3. Organisational skills (practice).

Fig. 4. Managing own learning skills (practice).

significantly enhanced information technology skills, managing own learning skills and organisational skills practice were only reported from 2010 (Figs 2 - 4). No effect was identified with presentation skills practice. Interventions in the individual curriculum changes that contributed to enhanced practice are illustrated in Table 1.

‘Managing your own learning’ underpins much of the intended curriculum effect, initially introduced in 2008 and reinforced in 2010. The effect of changes in assessment approach on students’ learning is well recognised.[11] Assessment changes in the curriculum review included enhanced alignment with learning outcomes, identifying clearly for students the importance of skills and content, e.g. around application of academic literacy. Some significant increases in the generic skills practice level, especially from 2010, indicate the influence of a change to continuous assessment in the modules that particularly require generic learning skills around information handling (academic literacy), technical and numeracy (statistics), IT skills (word processing and spreadsheets) and organisational skills. One particularly significant intervention was recognised as being necessary after consideration of the 2008 and 2009 curriculum evaluation results, including feedback from students and review of teaching staff interviews, i.e. the full effect on student learning had not been realised. Enhancement of teaching skills was clearly identified as an area of particular need to fully implement the intended curriculum changes. A programme of staff development was therefore implemented before the start of 2010. The subsequent effect on students’ generic learning skills

Discussion

Curriculum review is a dynamic, iterative process and the effect of change may not always be entirely predictable. Curriculum content and process should be continually reviewed, recognising that the student population changes and that delivery is influenced by a multitude of factors ranging from the educational environment through to teachers. Bitzer[10] highlights that a curriculum must maintain relevance in the context of societal, university and student changes, and to improve it must be evaluated regularly. The evaluation process at SU is strengthened by a formal annual process of module evaluation, including both student and teaching staff perceptions. This study is additionally strengthened by information provided by the generic skills practice evaluation, which collectively contributed to a responsive curriculum review evaluation, resulting in appropriate and responsive considered interventions.[8] Evidence of enhanced skills around

May 2016, Vol. 8, No. 1 AJHPE

Research experience was evidenced by further enhancement of skills practice in 2010 and 2011. The sustained increase in information handling skills practice reflected the new curriculum requirement for participation in ‘research’ from admission. The revised course required students to actively use and identify a range of resources to select, interpret and apply information to problem solving. These include diverse activities ranging from, e.g. interviewing people to gain information, to reading up on topics, through to a more scientific enquiry from conventional written texts. Moreover, the increase in experience was also driven by a continuous process of formative and summative assessments and feedback, including assignments, presentations and online quizzes.[7] The initial reported reduction in practice in technical and numeracy skills in 2008 and 2009, followed by an improvement with an increase in experience in these skills in 2010 and 2011, is interesting. This is an example of a potentially unintended consequence of curriculum changes and why monitoring of skills practice crucially contributed to the curriculum evaluation. The changes can be explained by the removal of physics as a pure scientific module in the first year, and the down-scaling of biostatistics in the new revised curriculum. Curriculum interventions subsequently introduced included contextualisation and redesign of modules (such as chemistry and biostatistics), which might account for the enhanced reported practice from 2010. No single intervention is likely to wholly account for the enhanced skills, which illustrates the value of integrated, collaborative discussion of multiple sources of evidence and feedback. The World Federation for Medical Education identified a minimum basic standard for effective use of e-learning skills during training. An often-made assumption is that all students have sufficient practice in IT skills during their high-school years.[5] In SA, however, it is possible that some students enrolling at universities lack these skills.[12] As part of the curriculum monitoring, students and staff indicated a potential deficiency in IT skills in some 1st-year students. The implementation in 2010 of a specific IT intervention may be interpreted as the reason for the resulting reported increase in IT skills. There has been an increasing trend in the level of experience of students’ organisational skills since 2008, but a definite and significant positive change occurred in 2010 and 2011. Organisational skills include aspects of self-management, such as meeting deadlines. To meet deadlines, students have to plan and manage their time. From 2010, two of the modules used continuous assessment, which required students to take responsibility for managing different assignments and tasks on a variety of topics, some involving teamwork and meeting different submission dates. Students therefore have to learn to work in a group, and learn from, receive and give feedback to one another. While working in a team, students evaluate their own strengths and weaknesses, and soon realise the important role of each individual group member for the group to function optimally.[13] This method of teaching and assessing students forced them to take responsibility for their own learning, which consequently had an effect on students’ ‘Managing your own learning skills’, evidenced by the positive, sustained enhanced practice in this skill. These skills are important for success in high-pressure environments, particularly for later success at university, and subsequent future medical practice. It is also possible that other areas of change management, including enhanced staff communication, engagement with collaborative decision-

making (at all levels – from administrative support staff to teaching staff), scrutinising study guides to enhance transparency, and other subtle changes in approach that are difficult to measure and articulate, are almost certainly cumulative and contributory. This approach to curriculum review reinforces the importance of a scholarly approach to all aspects of educational practice, and aligns with Van der Vleuten’s appeal that educational research, which informs practice, should be evidence based.[14]

Conclusion

Contextual skills development in medicine is vital and more effective when embedded in the curriculum. This underpins the approach taken in this curriculum review, ensuring that key generic learning skills are addressed. Our study indicates the value of curriculum evaluation that goes beyond monitoring output only in terms of content outcomes. It also evaluated the effect on student learning, specifically how they are equipped with generic learning skills to support their expected success at university. Such a comprehensive methodological approach not only takes into account feedback and evaluation from staff and students, but, importantly, evaluates effect, particularly on skills development. This ensures identification of how the curriculum meets the diversity of learning needs and skills present on entry into higher education.

Practice points

• Scientific research is a tool to evaluate a curriculum and enhance the validity of the evaluation process, and results in appropriate and responsive considered interventions. • Change to the type of assessments results in significant increased changes to the generic skills practice levels of students. • Curriculum changes have to be accompanied by the necessary and appropriate staff development initiatives. • Key generic skills have to be embedded in a curriculum, and the contributions of the curriculum have to support student success by means of methodological evaluation processes. References 1. Frenk J, Chen L, Bhutta ZA, et al. Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. Lancet 2010;376:1923-1958. [http://dx.doi.org/10.1016/S01406736(10)61854-5] 2. Murdoch-Eaton DG, Whittle S. Generic skills in medical education: Developing the tools for successful lifelong learning. Med Educ 2012;46:120-128. [http://dx.doi:org/10.1111/j.1365-2923.2011.04065.x] 3. Barrie S. A conceptual framework for the teaching and learning of generic graduate attributes. Studies in Higher Education 2007;32:439-458. [http://dx.doi.org/10.1080/03075070701476100] 4. Paul G, Hinman G, Dottl S, Passon J. Academic development: A survey of academic difficulties experienced by medical students and support services provided. Teach Learn Med 2009;21:254-260. [http://dx.doi. org/10.1080/10401330903021041] 5. Whittle SR, Pell G, Murdoch-Eaton DG. Recent changes to students’ perceptions of their key skills on entry to higher education. J Further Higher Educ 2010;34:557-570. [http://dx.doi.org/10.1080/030987 7X.2010.512082] 6. Burch VC, Sikakana CNT, Gunston G, Shamley DR, Murdoch-Eaton D. Generic learning skills in academicallyat-risk medical students: A development programme bridges the gap. Med Teach 2013;35:671-677. [http://dx.doi. org/10.3109/0142159X.2013.801551] 7. Kember D. 2009. Nurturing generic capabilities through a teaching and learning environment which provides practice in their use. Higher Educ 2009;57:37-55. [http://dx.doi.org/10.1007/s10734-008-9131-7] 8. Whittle SR, Murdoch-Eaton DG. Curriculum 2000: Have changes in sixth-form curricula affected students’ key skills? J Further Higher Educ 2005;29:61-71. [http://dx.doi.org/10.1080/03098770500037762] 9. Murdoch-Eaton D, Manning D, Kwizera E, Burch V, Pell G Whittle S. Profiling undergraduates’ generic learning skills on entry to medical school: An international study. Med Teach 2012;34:1033-1046. [http://dx.doi.org/10.3 109/0142159X.2012.706338] 10. Bitzer E. First-year students’ perceptions of generic skills competence and academic performance: A case study at one university. S Afr J Higher Educ 2005;19(3):172-187. [http://dx.doi.org/10.4314/sajhe.v19i3.25512] 11. Cilliers FJ, Schuwirth LWT, Herman N, Adendorff HJ, van der Vleuten CPM. A model of the pre-assessment learning effects of summative assessment in medical education. Adv Health Science Educ 2012;17:39-53. [http:// dx.doi.org/10.1007/s10459-011-9292-5] 12. Boveé C, Voogt J, Meelissen M. Computer attitudes of primary and secondary students in South Africa. Computers Human Behav 2007;23:1762-1776. [http://dx.doi.org/10.1016/j.chb.2005.10.004] 13. Hrynchak P, Batty H. The educational theory basis of team-based learning. Med Teach 2012;34:796-801. [http:// dx.doi.org/10.3109/0142159X.2012.687120] 14. Fincher RME, Simpson DE, Mennin SP, et al. Scholarship in teaching: An imperative for the 21st century. Acad Med 2000;75:887-894.

May 2016, Vol. 8, No. 1 AJHPE

Research Perceived stressors of oral hygiene students in the dental environment N A Gordon,1 DipOH, BA, MPH, Dip Adult Education; C A Rayner,1 DipOH, BA Hons, MA; V J Wilson,2 BChD, MChD; K Crombie,3 Dip Diagnostic Radiography, HDE, MSc Dent; A B Shaikh,4 BChD, MSc Dent, MChD; S Yasin-Harnekar,5 BChD, MSc Dent, PDD 1

Department of Oral Hygiene, Faculty of Dentistry, University of the Western Cape, Cape Town, South Africa

Department of Restorative Dentistry, Faculty of Dentistry, University of the Western Cape, Cape Town, South Africa

Department of Diagnostics and Radiology, Faculty of Dentistry, University of the Western Cape, Cape Town, South Africa

Department of Orthodontics, Faculty of Dentistry, University of the Western Cape, Cape Town, South Africa

Department of Paediatric Dentistry, Faculty of Dentistry, University of the Western Cape, Cape Town, South Africa

Corresponding author: C A Rayner (crayner@uwc.ac.za)

Background. University students are exposed to a multitude of stressors that may impact on their performance. The nature of health sciences education generally involves early engagement with patients and communities, which may add to the stressors inherent to university life. There is sparse infor mation on stressors in the oral hygiene educational environment. Objective. To determine perceived stressors and the level of burnout among oral hygiene students at the University of the Western Cape, Cape Town, South Africa. Method. A descriptive, cross-sectional study design was used. The study sample included all students in the Bachelor of Oral Health (BOH) degree during 2012 (N=89). A self-administered questionnaire was used to gather data. Three parameters were measured, i.e. (i) demographic characteristics; (ii) perceived sources of stress, using a modified Dental Environment Stress (DES) questionnaire; and (iii) burnout, using the Maslach Burnout Inventory (MBI). Results. Respondents were mostly female (74%) and primarily in the 18 - 25-year age group (92%). First- and 2nd-year students identified fear of failing and study load as major stressors. Stressors related to a lack of basic needs were identified as major stressors by 25% of 1st-year students. Third-year students identified clinical quotas, supervision and patients being late as major stressors. MBI scores indicated that students were not at risk for burnout; however, most students (66.2%) scored high on emotional exhaustion (EE). Conclusion. Oral hygiene students identified stressors in their learning environment. There was a progressive increase in EE across academic years. The results suggest that interventions should be tailored for specific academic year groups. Afr J Health Professions Educ 2016;8(1):20-24. DOI:10.7196/AJHPE.2016.v8i1.422

Stress is part of daily life and may be the stimulus for individual achievement. Therefore, stress can serve as a motivational factor for students to perform at their peak or reduce their level of effectiveness.[1] Individuals experience stress to a greater or lesser degree, depending on their perception of the demands in their environment and the resources to cope with these demands.[2] University entry for students is a transitional period, as they are exposed to a multitude of changes in their personal, social and academic environment. Conditions and events inherent to university life induce experiences of stress, which may lead to difficulty in adjusting to this new environment.[3] The health sciences educational milieu is unique, as students are exposed to further stressors such as early engagement with patients and communities. Stress among students in various disciplines in the health sciences is well documented.[1,4-21] However, the literature on stressors among students and qualified dental hygienists is sparse.[7,14,20-22] Sources of environmental stress among dental students have been identified and quantified by means of the Dental Environment Stress (DES) questionnaire.[4,5] Stressors identified include the learning environment, fear of failure, heavy workload, difficulty in dealing with patients, per forming non-reversible procedures in a confined space, difficulty in dealing with transitions in curricula and difficult relationships with academic staff. Gorter et al.[11] suggested that stress among dental students has been

May 2016, Vol. 8, No. 1 AJHPE

reported at length and that it would currently be more useful to focus on interventions to address this concern. Long-term exposure to stress in the learning and working environ ment may result in burnout,[23] also referred to as a syndrome found among professionals doing ‘service work’.[24] Burnout includes the domains of emotion al exhaustion (EE), becoming emotionally exhausted; depersonalisation (DP), the development of a negative, cynical attitude to patients; and a sense of diminished personal accomplishment (PA), evaluating oneself and one’s own accomplishments negatively. Roberts and Ellingson[20] reported that signs and symptoms of burnout include ‘emotional, cognitive, behavioural and physical aspects’. These may be seen as loss of humour, a persistent sense of failure, anger, resentment and bitterness, postponement of contact with patients, constant feeling of tiredness, increased use of sick leave, rigid thinking, and difficulty concentrating.[25] Although burnout has not been reported as being prevalent, EE, the key dimension of burnout, has been reported among students and professionals in the dental field.[7,11,14,15,23,26] There are no published reports on stress among South African (SA) oral hygiene students. As qualifications in both oral hygiene and dentistry are offered at the Faculty of Dentistry, University of the Western Cape (UWC), Cape Town, SA, it would be premature to develop student interventions without identifying the stressors and their effect. The objective of this study was to determine perceived stressors and the level of burnout among oral hygiene students at UWC.

Research Methods

Study design

A descriptive, cross-sectional study design was used.

Study population

All students registered for the 3-year Bachelor of Oral Health (BOH) programme during 2012 were included (N=89).

Instrument and data collection

Data were collected by means of a self-administered questionnaire. The following three parameters were measured: (i) demographic characteristics; (ii) perceived sources of stress, using a modified DES questionnaire; and (iii) burnout, using the Maslach Burnout Inventory (MBI). The DES[4] and MBI[24] questionnaires are validated instruments. Demographic characteristics included home language as a proxy of ethnicity, in view of the 11 official languages in SA. The DES consisted of 79 statements categorised into the following areas of study: study environment (n=27), theoretical aspects (n=14), preclinical aspects (n=13), and clinical aspects (n=25). Students were asked to respond to each statement by indicating whether it posed ‘no problem’, ‘a small problem’ or ‘a huge problem’ that might interfere with their studies. The following statements were added to suit the local context: discrimination due to race, nationality, gender or social class; transport to the university; accommodation; safety; and having enough food to eat. The MBI consisted of 22 statements. Each statement was scored on a 7-point Likert scale ranging from 0 (‘never’ experienced) to 6 (experienced ‘every day’). The MBI was divided into three subscales, i.e. EE, PA and DP. Statements in the MBI were adapted to include ‘other students or people’. This was in view of the teaching methodologies that encompass engagement with students and communities throughout the programme. The questionnaire was piloted with 10 students to assure validity, and modified accordingly. It was distributed to students for completion in their classrooms. Completed questionnaires were submitted to the researchers. The study was conducted at the end of the first semester.

Data analysis

Data were entered and analysed using descriptive statistics (IBM SPSS Statistics for Windows, Version 21.0, USA: IBM Corp.). Frequency distributions were used to identify stressors posing a ‘huge problem’ to students. The MBI manual[24] was used to categorise the student groups into high, average and low risk for burnout. Burnout is indicated in high scores of EE (≥27) and DP (≥10) and low scores of PA (≥40) in the Human Services Survey (MBI HSS).[24]

Ethical considerations

Ethical approval was obtained from the Faculty of Dentistry and University Research and Ethics committees, UWC. Prior to distributing the questionnaires, students were informed verbally and in writing of the purpose of the study. Informed consent was obtained.

Results

Demographics

The response rate was 85%. Respondents were mostly female (74%) and primarily in the 18 - 25-year age group (92%). Six of the 11 official languages

were reported as home languages, with 38% having English, the medium of education at the university, as their home language. More than half (58%) resided in the Western Cape – the province where UWC is located; 76% had attended public schools and 24% private schools; 47% lived with families and the remainder stayed in university residences (30%) or private residences (17%).

Student response to the DES statements

Table 1 illustrates the top five stressors reported as a ‘huge problem’ for each category of the DES. The ‘study environment’ scored lowest overall of the four categories. Only 3rd-year students completed the clinical category and the majority experienced this as a ‘huge problem’. Table 2 illustrates the top five stressors by year group, indicating that stressors vary across the academic years. First- and 2nd-year students identified the theoretical aspect of their studies as most stressful, whereas the 3rd-year group reported the clinical category as most stressful. Table 1. The top five perceived stressors in each category of the DES Responses to a ‘huge problem’, %

Perceived stressors Study environment 1. Fear of being unemployed in future

2. Lack of time for relaxation

3. Neglect of personal life

4. Treated as being immature

5. Lack of confidence to be a successful hygienist

Theoretical problems 1. Heavy study load

2. Fear of failing a module or year

3. Overloaded feeling due to the large number of modules in the programme

4. Having a lecture or clinic before a scheduled assessment

5. Lack of self-motivation to study

Preclinical problems (BOH II and III) 1. Fear of making mistakes

2. Lack of time to practise preclinical procedures

3. Limited co-operation from laboratory technicians/staff

4. Meeting preclinical requirements

5. Number of supervisors in relation to students; inconsistency between supervisors

Clinical problems (BOH III only) 1. Number of assigned quotas

2. Number of clinical supervisors in relation to number of students

3. Fear of being criticised

4. Patients being late/missing appointment

5. Fear of being unable to catch up with clinical requirements

May 2016, Vol. 8, No. 1 AJHPE

Research

Table 2. Top five perceived stressors of the DES per year group

Table 3. The Maslach Burnout Inventory

Study year

Potential stressor

Frequency, %

Statements describing student feelings

Mean (SD)

BOH I

Fear of failing a module Overloaded feeling due to large number of modules Heavy study load Fear of being unemployed in the future Having financial responsibilities

61 55

Emotional exhaustion

3.28 (1.75)

1. I feel used up/worn out at the end of a day at university

4.25 (1.53)

2. I feel emotionally drained/exhausted from my studies

4.24 (1.56)

3. I feel fatigued/tired when I get up in the morning and have to face another day at university

4.11 (1.63)

Heavy study load Overloaded feeling due to large number of modules Fear of failing a module or year Having a lecture before an assessment Fear of making mistakes

91 82

4. I feel frustrated by my studies

3.96 (1.65)

5. I feel burnt out from my studies

3.86 (1.65)

6. I feel that I am working too hard on my studies

3.01 (1.94)

7. I feel that I am at the end of my rope

2.32 (2.23)

8. Interacting with people all day is really a strain for me

1.96 (1.84)

Number of assigned clinical quotas Number of clinical supervisors in relation to students Fear of being criticised by supervisors Patients being late/missing appointments Fear of being unable to catch up with clinical requirements

95 74

9. Interacting with people directly puts too much stress on me

1.85 (1.74)

Depersonalisation

1.29 (1.58)

1. I worry that my studies are hardening me emotionally

2.53 (2.19)

2. I have become more callous/uncaring towards people since I started my studies

1.35 (1.82)

3. I feel that I treat some patients and other students as if they were impersonal objects

0.99 (1.35)

4. I don’t really care what happens to some patients and other students

0.93 (1.44)

5. I feel that patients and other students blame me for some of their problems

0.65 (1.12)

Personal achievement

3.71 (1.66)

1. I feel I’m positively influencing other people’s lives through my studies

4.18 (1.46)

2. I can easily create a relaxed atmosphere with my patients and other students

4.17 (1.63)

3. I can easily understand how my patients and other students feel about things

3.93 (1.67)

4. I feel exhilarated/inspired after working closely with my patients and other students

3.77 (1.68)

5. I deal very effectively with the problems of my patients and other students

3.70 (1.73)

6. I have accomplished many worthwhile things in my studies

3.68 (1.70)

7. In my studies, I deal with emotional problems very calmly

3.47 (1.74)

8. I feel very energetic

2.85 (1.70)

BOH II

BOH III

52 44 38

77 68 66

74 74 74

Although not in the top five stressors, the statements added to the DES questionnaire to suit the local context indicated the following: 1st-year students reported transport (29%) and safety (27%) to and from the university, accommodation (29%), not being able to study in their living environment (24%) and not having enough to eat (24%) as a ‘huge problem’. Discrimination due to race, nationality, gender or social class was reported as a ‘huge problem’ by 2nd-year (32%) and 3rd-year (37%) students.

Student response to the MBI

Table 3 shows overall means and standard deviations for EE, PA and DP. The mean and standard deviations for individual statements were ranked in descending order. Statements referring to ‘self ’ were reportedly experienced more frequently in EE (‘I feel used up at the end of my day at university’) and DP (‘I worry that my studies are hardening me emotionally’) subscales. Table 4 shows the categorisation of year groups according to their risk for burnout. The means and standard deviations of the MBI score for each subscale and the percentage of students in the respective year group are indicated. Most (66.2%) students scored high on EE, the key dimension for burnout. However, there were significant differences between the three year groups (p=0.039). In terms of burnout, 1st-year scores were seen as ‘indicative of engagement with work’, with 76.5% of the class scoring high on PA and 14.7% scoring average on EE. Second-year scores were high on EE and DP but average on PA, suggesting a risk for burnout. Third-year scores showed a reversal on the DP and PA scores; yet, EE remained high. There was no significance in student demographics and EE, DP and PA. There was considerable variation in student experiences in the academic year groups, as seen by the percentage of students in each category.

Discussion

Demographic characteristics

Oral hygiene is a predominantly female-orientated profession globally.[27] The gender distribution in this study is indicative of a changing student

May 2016, Vol. 8, No. 1 AJHPE

profile, with more males entering the profession. Further diversity of the study population is evident in the home language distribution and schooling background. Bojuwoye[3] reported that factors associated with financial difficulties, demands of the university environment and administrative processes were experienced as stressful by 1st-year university students. The current study suggests that 1st-year oral hygiene students may experience similar stressors. Reports of discrimination due to race, nationality or gender as a ‘huge problem’ are cause for concern and warrant further enquiry. The decision to include additional stressors to the DES was supported by the results, suggesting that a validated tool should be adapted to the local context to accommodate the social, cultural, economic and historical factors.

Research

Table 4. Categorisation of MBI subscales by student year group, BOH MBI subscales

Overall score

1st year

2nd year

3rd year

EE category EE score % within group

High 29.04 (11.00) 66.2

Average 25.79 (10.56) 14.7

High 33.09 (10.10) 81

High 30.16 (11.51) 63.2

DP category DP score % within group

Average 6.28 (5.24) 27.8

Low 5.14 (4.97) 56.3

High 8.13 (4.86) 36.4

Average 6.15 (5.77) 27.8

PA category PA score % within group

High 28.89 (8.23) 76

High 29.02 (10.05) 76.5

Average 26.44 (6.35) 13.6

High 31.47 (5.66) 63.2

Student response to the DES statements

There were a number of similarities between the top stressors identified in this study and those in the international literature.[1,3-5,9,10,12,13,16,17] The study load, financial responsibilities, patients being late or missing appointments, and fear of being unable to catch up with clinical requirements were also noted among US dental hygiene students.[20] At least two stressors in the top five of each component of the DES questionnaire posing a ‘huge problem’ in this study were also reported by Saudi Arabian dental students.[4] These stressors were: lack of time for relaxation, being treated as immature, study load, feeling overloaded due to the large number of modules in the programme, lack of time to practise a preclinical procedure, number of supervisors in relation to students, inconsistency between supervisors, fear of being criticised, and patients being late or missing appointments (Table 1). Fijian dental students also reported the following stressors: feeling overloaded, fear of failure, criticism from clinical supervisors in the presence of patients, amount of assigned work, financial resources, and fear of unemployment after graduation.[12] Of concern is that stressors identified by Garbee et al.[1] in 1980 are still reported in the current literature and were also found in our study. Considering that the stressors were known, the authors questioned whether demands by departments were realistic and in the interest of students or whether departments competed for students’ time.[1] Stressors in the abovementioned studies could be categorised as student, staff, curriculum and/or educational system related. It may be expedient to use categories to guide universities to the type and level of intervention required for a less stressful dental environment. Final-year oral hygiene students reported signi ficantly higher (p<0.01) stress levels than 1st-year students in three DES items, i.e. atmosphere created by clinical faculty, lack of input into decision-

making processes at the faculty and inconsistency of feedback between different instructors.[20] A number of items identified were also noted by students in the current study. The authors questioned whether different experiences to stressors between class years were a result of changing demands of the programme or the unique personality of a class.[20] The results of this study did not indicate a lack of input into decision-making processes at university as a ‘huge problem’, contrary to those reported by Roberts and Ellingson.[20] A possible explanation is that UWC students have representation on faculty structures. The manner in which stress is defined by the researcher informs the research approach and ultimately the answers gained. Hamill,[28] in a qualitative study of student nurses’ perceptions of stress, used Cox’s interactionist model of stress. This model advocates that ‘stress should not be seen as either a stimulus or a set of responses but rather a person’s interpretation of the significance of a threatening event (the stimulus) and his or her resources to cope with it (the response)’. The DES and MBI questionnaires are quantitative instruments and may not be useful on their own. Future studies using these instruments should consider using a mixed-method approach, where qualitative aspects are included to allow for clarification and elaboration of student experiences. Polychronopoulou and Divaris[16] grouped stressors into seven categories to facilitate targeted interventions. These are self-efficacy beliefs, faculty and administration, workload, patient treatment, clinical training, performance pressure, and other. A substantial number of stressors identified by students in this study were in the ‘self-efficacy beliefs’ category, suggesting that further enquiry may be needed. In considering interventions, programmes may also have

limited control over stressors, such as patient co-operation,[20] also identified as a stressor in this study. In such instances student stress can be reduced through training to develop inter personal relationships with patients to foster understanding of the patient’s life context and so improve co-operation.[20] Longitudinal studies have been suggested to better the understanding of stressors identified and to monitor at-risk students to inform appro priate interventions.[11,12,21,23,29] This position is supported by the current study in view of stressors appearing to vary across the academic years.

Response to the MBI

The scores for each subscale of the MBI show a trend (Table 3), with statements referring to ‘engagement with others’ reported at a lower frequency on the EE and DP subscales and at a higher frequency on the PA subscales. This observation suggests that students may feel better about themselves when interacting with others, which supports the view that early engagement with patients is ‘protective’ in terms of stress and burnout.[19] The opposite, where engagement with ‘self ’ was reported at a higher frequency on EE and a lower frequency on PA, may indicate that students are challenged to cope in an academic environment. This finding may be consistent with the fact that a number of stressors noted in the DES were located in the ‘self-efficacy beliefs’ category.[16] The overall scores (Table 4) indicate that the group is not at risk of burnout. Although the mean scores for the programme may be favourable, considerable variations across the academic years were noted. The 1st-year class started off positively, showing ‘engagement with work’, the 2nd-year class appeared to be at risk of burnout, and at the 3rd-year level students appeared to be coping better. Of concern is that EE, the key dimension of burnout, increased progressively over the 3 academic years, with 62% of students falling into the ‘high’ category in the 3rd year. Dimensions of burnout were also found among qualified dental hygienists, with high levels of EE (14%) and DP (15%) and high levels of diminished PA (29%).[14] Hinshaw et al.[7] reported on stress and burnout experienced by dental hygiene educators. The authors highlighted institutional responsibility to reduce stress experienced as a result of educators’ roles and responsibilities. The results of this study cannot be generalised with regard to the broader oral hygiene student

May 2016, Vol. 8, No. 1 AJHPE

Research population. However, the findings provide insight into the perceptions and experiences of UWC oral hygiene students.

Conclusion

This study found that stressors were identified within the oral hygiene student population. Stressors were generally similar to those reported by dental hygiene and dental students in the international literature. The fact that EE increased progressively across the 3 years indicated a need for intervention to improve the experiences of students in the dental learning environment. The results suggest that interventions should address student stressors at a generic student level and at the level of the academic year. References 1. Garbee WH, Zucker SB, Selby GR. Perceived sources of stress among dental students. J Am Dent Assoc 1980;100:853-857. [http://dx.doi.org/10.14219/jada.archive.1980.0279] 2. Taylor SE. Health Psychology. 3rd ed. New York: McGraw-Hill, 1995. 3. Bojuwoye O. Stressful experiences of first year students of selected universities in South Africa. Counselling Psychology Quarterly 2002;15(3):277-290. 4. Al-Saleh SA, Al-Madi EM, Al-Angari NS, Al-Shehri HA, Shukri MM. Survey of perceived stress-inducing problems among dental students, Saudi Arabia. Saudi Dent J 2010;22:83-88. [http://dx.doi.org/10.1016/j.sdentj.2010.02.007] 5. Alzahem AM, van der Molen HT, Alaujan AH, Schmidt G, Zamakhshary MH. Stress amongst dental students: A systematic review. Eur J Dent Educ 2011;15:8-18. [http://dx.doi.org/10.1111/j.1600-0579.2010.00640.x] 6. Dahan H, Bedos C. A typology of dental students according to their experience of stress: A qualitative study. J Dent Educ 2010;74(2):95-103. 7. Hinshaw KJ, Richter LT, Kramer GA. Stress, burnout, and renewal activities of dental hygiene education administrators in six US Midwestern States. J Dent Educ 2010;74(3):235-250. 8. Silverstein ST, Kritz-Silverstein D. A longitudinal study of stress in first-year dental students. J Dent Educ 2010;74(8):836-848. 9. Kumar S, Dagli RJ, Mathur A, Jain M, Prabu D, Kulkarni S. Perceived sources of stress among Indian dental students. Eur J Dent Educ 2009;13:39-45. [http://dx.doi.org/10.1111/j.1600-0579.2008.00535.x]

May 2016, Vol. 8, No. 1 AJHPE

10. Polychronopoulou A, Divaris K. Dental studentsâ&#x20AC;&#x2122; perceived sources of stress: A multi-country study. J Dent Educ 2009;73(5):631-639. 11. Gorter R, Freeman R, Hammen S, Murtomaa H, Blinkhorn A, Humphris G. Psychological stress and health in undergraduate dental students: Fifth year outcomes compared with first year baseline results from five European dental schools. Eur J Dent Educ 2008;12:61-68. [http://dx.doi.org/10.1111/j.1600-0579.2008.00468.x] 12. Morse Z, Dravo U. Stress levels of dental students at the Fiji School of Medicine. Eur J Dent Educ 2007;11:99-103. [http://dx.doi.org/10.1111/j.1600-0579.2007.00435.x] 13. Sofola OO, Jeboda SO. Perceived sources of stress in Nigerian dental students. Eur J Dent Educ 2006;10:20-23. [http://dx.doi.org/10.1111/j.1600-0579.2006.00391.x] 14. Gorter RC. Work stress and burnout among dental hygienists. Int J Dent Hyg 2005;3(2):88-92. [http://dx.doi. org/10.1111/j.1601-5037.2005.00130.x] 15. Pohlmann K, Jonas I, Ruf S, Harzer W. Stress, burnout and health in the clinical period of dental education. Eur J Dent Educ 2005;9(2):78-84.[http://dx.doi.org/10.1111/j.1600-0579.2004.00359.x] 16. Polychronopoulou A, Divaris K. Perceived sources of stress among Greek dental students. J Dent Educ 2005;69(6):687-692. 17. Naidu RS, Adams JS, Simeon D, Persad S. Sources of stress and psychological disturbance among dental students in the West Indies. J Dent Educ 2002;66(9):1021-1030. 18. Sanders E, Lushington K. Effect of perceived stress on student performance in dental school. J Dent Educ 2002;66(1):75-81. 19. Heath JR, Macfarlane TV, Umar MS. Perceived sources of stress in dental students. Dent Update 1999;26:94-100. 20. Roberts RA, Ellingson PL. Perceived environmental stressors for dental hygiene students. J Dent Educ 1996;60(10):836-841. 21. Hendricks SJH, Joshi A, Crombie K, Moola MH. Perceived sources of stress among black dental students in South Africa. J Dent Educ 1994;58(6):406-410. 22. Lopresti S. Stress and the oral hygiene profession. Can J Dent Hyg 2014;48(2): 63-69. 23. Humphris G, Blinkhorn A, Freeman R, et al. Psychological stress in undergraduate dental students: Baseline results from seven European dental schools. Eur J Dent Educ 2002;6:22-29. [http://dx.doi.org/10.1034/j.1600-0579.2002.060105.x] 24. Maslach C, Jackson SE, Leiter MP. Maslach Burnout Inventory Manual. 3rd ed. Palo Alto, CA: Consulting Psychologists Press, 1996. 25. Scutter S, Goold M. Burnout in recently qualified physiotherapists in South Australia. Austr Physiother 1995;41(2):115-118. [http://dx.doi.org/10.1016/S0004-9514(14)60425-6] 26. Gorter RC, Storm MK, te Brake JHM, Kersten HW, Eijkman MAJ. Outcome of career expectancies and early professional burnout among newly qualified dentists. Int Dent J 2007;57(4):279-285. 27. Johnson PM. International profiles of dental hygiene 1987 - 2006: A 21-nation comparative study. Int Dent J 2009;59:63-77. 28. Hamill C. The phenomenon of stress as perceived by Project 2000 student nurses: A case study. J Adv Nursing 1995;21:528-536. [http://dx.doi.org/10.1111/j.1365-2648.1995.tb02737.x] 29. Burk DT, Bender DJ. Use and perceived effectiveness of student support services in a first-year dental student population. J Dent Educ 2005;69(10):1148-1160.

Research Balancing the educational choices in the decision-making of a dean of medicine: Fission or fusion? J E Wolvaardt,1 BCur, MPH, PGCHE, PhD; B G Lindeque,2 MB ChB, MMed, MD, GKOG; P H du Toit,3 BA, HED, BA Hons, MEd, DTI, PhD School of Health Systems and Public Health, Faculty of Health Sciences, University of Pretoria, South Africa

School of Medicine, Faculty of Health Sciences, University of Pretoria, South Africa

Department of Humanities Education, Faculty of Education, University of Pretoria, South Africa

Corresponding author: J E Wolvaardt (liz.wolvaardt@up.ac.za)

Background. The literature on deans of medicine focuses mostly on the qualifications, roles, abilities, management and leadership competence of the deans. The gap between theory and practice is, however, the nucleus of the position. Objectives. To describe insights into the educational forces that act on a dean of medicine and the implications for those who wish to bring about change – in this case, changes in the inclusion of public health in the medical curriculum. Methods. A series of in-depth interviews of a vice dean (VD) of medicine was conducted over a period of a year. The interviews were transcribed. Initial in-depth analysis of the transcriptions was done using open coding, prior to a second round of coding that resulted in themes. Results. The interviews revealed a serendipitous aspect, namely the ontological realities of the VD’s practice. This practice is characterised by balancing multiple internal and external forces, such as the breadth and depth of the curriculum that acts on the medical curriculum. Conclusion. The ontological realities of the VD bring to life the qualifications and leadership, and management competence, roles and abilities described in the literature. The multiple – often opposing – educational choices that deans face are an inescapable reality of deanship. Medical deans must balance these opposing forces to ensure fusion within the curriculum, and those interested in changes, such as strengthening the teaching of public health in this curriculum, need to plan on how to overcome this. Afr J Health Professions Educ 2016;8(1):25-29. DOI:10.7196/AJHPE.2016.v8i1.474

If one can compare deanship to an atom, then the focus in the literature is on the physical manifestations of deanship as a source of support, funding and problem solving rather than the nucleus.[1-3] This article outlines some insights into deanship that emerged as a result of the first author’s exploration of teaching public health in the medical curriculum. In exploring the inclusion of public health, the serendipitous understanding of what it entails to hold a medical curriculum together for a vice dean (VD) of medicine in a South African (SA) university surfaced. At this particular university, the chairperson of the school of medicine (similar to dentistry) also holds the position of the VD of medicine. The two remaining schools, healthcare sciences and public health, do not have VDs. It is the second of these two schools that, although a postgraduate school, is responsible for the inclusion of public health in the undergraduate medical curriculum. One consequence of this separation between a school of public health and a school of medicine is that although the responsibility for the inclusion of public health rests with one school, the students and the curriculum in which public health is included are those of another school. The series of interviews with the VD were conducted with the primary intention of better understanding the original curricular intentions, curricular development and current inclusion of the public health curriculum in the medical curriculum. Among the journal articles that focus on deans of medicine, the emphasis is on the qualifications, competence, abilities and personal qualities.

Qualifications and experience

Deans of medicine are usually described as being medical specialists with a reputation in clinical expertise and research in their field of expertise.

Deans usually come from mainstream specialties such as internal medicine, surgery, and obstetrics and gynaecology, and are often appointed in their fifties.[2] The professional pathway followed by the dean is immaterial (academic, clinical or administrative).[2] However, the most common criteria used in the selection of deans (excellence in clinical practice and research) are insufficient in themselves to fulfil the leadership role.

Leadership and management competencies

Various understandings exist on what constitutes leadership and manage ment practice. The purpose in this article is not to engage in the debate on leadership and management, but rather to highlight how both are considered desirable attributes. The management and leadership competencies des cribed by Rich et al.[4] are based on a comprehensive review of 33 sources, but it is the need to create a common vision that is of particular interest.[1,2,4] Medicine focuses on decision-making at the individual physician-patient level. Leadership necessarily involves stepping away from this relationship and examining problems at a systems level, requiring the ability to view issues broadly and systemically.[5] At the nucleus of systems thinking is the ability ‘to balance the interests of several conflicting interests with an ultimate focus on the benefit of the institution as a whole’.[2] Also popular in the literature are the desirable skills and abilities required.

Skills and abilities

The differences between skills or abilities are, again, better debated elsewhere, but what is clear is that a wide range of both is required. The literature suggests that the skills required to create common goals of communal achievement include patience, persuasion, communication and

May 2016, Vol. 8, No. 1 AJHPE

Research interpersonal skills.[2,6] Patience is required not only in the act of persuading others, but as a more generic ability to tolerate ambiguity – an inescapable reality of management, but not necessarily of doctors in the diagnostic process.[2] Despite working within a management environment that has few diagnostic tests, the dean is nevertheless expected to ‘act decisively, and achieve an appropriate balance between flexibility and assertiveness’.[2] Interpersonal skills identified include diplomacy, and communication skills including listening skills, facilitation and spiritual sensitivity.[2] Among the professional and technical management skills, fiscal expertise, planning, problem-solving, organising, and administering are just a few.[2] These skills and abilities of the envisaged successful dean overlap with the personal qualities that are needed to fulfil the role(s).

Roles and personal qualities

Among the plethora of roles, the primary roles of the dean are to ensure that the medical school flourishes and to foster a deeper understanding of why it is important to flourish.[2] Successful deans are described as having high professional competence combined with human relation skills and the ability to administer a complex programme, and personal qualities, such as openness and objectivity.[2,6] The personal qualities of honesty and integrity, and the ability to address conflict of interest so that decisions are fair, consistent and transparent, are valued, as well as ‘tenacity, stamina, longterm wisdom, emotional intelligence … equanimity’.[1] It is surprising that we have any who meet these criteria listed in the literature. The aim of this article is to highlight the often opposing educational choices an experienced dean of medicine faces – a key consi deration for those who wish to bring about change in the curriculum.

Methods

A constructive grounded approach using qualitative methods, as described by Charmaz,[7] was used. The longitudinal series of interviews formed part of the first author’s action research design,[8] aimed at her professional development.

Ethical considerations

Ethical approval for the overarching study of exploring the inclusion of public health in the medical curriculum was granted by the University of Pretoria’s Faculty of Education ethics committee (HS10/05/01). Written informed consent for the interviews was obtained from the VD.

Data gathering

Qualitative data were obtained through a series of three semistructured in-depth interviews over the period of a year. Each interview with the VD started with a trigger question: • Interview one: What is the next big idea in medical education? (Academic Medicine Question of the Year, 2011) • Interview two: The Health Professions Council of South Africa (HPCSA) regulations refer to having medical public health as a prominent curriculum theme. So what do you think they envisage/want with this inclusion? • Interview three: So what roles (for our medical students) do we consider ideal? The interviews were exploratory and dialectic in nature and were videotaped for transcription by an expert. The transcription was checked by the first

May 2016, Vol. 8, No. 1 AJHPE

author as the interviewer, and member-checked by the VD. An initial in-depth analysis of the transcriptions was done using open coding, and a constant comparative method suggested by Tesch[9] was used to form and delineate categories, and to discover patterns in the data. The second round of coding included axial coding that resulted in themes. A software package for qualitative data (NVivo 8) was used for data management. Agreement of the themes was reached through discussion between the authors.

Crystallisation

In this research, crystallisation replaced triangulation, as triangulation implies that the outcome is a fixed position. In contrast, crystallisation allows a ‘more complex and deeper understanding of the phenomenon’.[10] Crystallisation was ensured by inviting other specialists in deanship in health sciences to peer review this article. These experts were a current dean of medicine from another African university, a VD of research and a director of a school of medicine from two other SA universities, and a senior experienced academic from another institution who had never been a dean or VD. The participants were asked to critically read the article and to reflect whether the account is an authentic reflection of their own reality, and also to identify any additional educational forces that influence their decision-making in education.

Results

The core theme that emerged from the text is that of constant pressure of often-conflicting interests. This main theme is supported by six subthemes: • Balancing those who use blocking strokes and those who hit for the boundary in education • Balancing the breadth and depth of the curriculum • Balancing the interests of the many and the interests of the few • Balancing the individual focus and population focus • Balancing teaching in the academic complexes and in the field • Balancing international and local realities. To represent the complexity of medical education, extensive use is made of verbatim text as the authentic voice of the VD.[11]

Balancing those who use blocking strokes and those who hit for the boundary in education The profiles of academic staff include those who model their teaching practice on their own experience as undergraduate students so that ‘we still have some people who read from a textbook’. These members of the academic team are ‘just … blocking like we were taught before …’ using predictable serviceable classroom techniques of lecturing. But the milieu has changed and as a result there are those who have been included in the team who are engaged in their educational practice as well as their clinical practice, and who are using educational strategies that, in effect, have changed the game: ‘In the modern game of cricket everything has changed. Because I think you are right. I think that some of the people who are out there batting, who are teaching the students, who were taught in a conventional sense and their game … is no longer adequate for the challenges at hand. We have to devise new strokes … .’ But the challenge of combining traditional and modern styles of teaching fades in the face of the major challenge of deciding the breadth and depth of the curriculum.

Research Balancing the breadth and depth of the curriculum

A primary intention of the medical curriculum is that the learning experience will prepare students for professional practice, and while not all can be achieved within the curriculum alone, the curriculum needs to anticipate a future reality. Balancing the known with the unknown: ‘… so, the preparing for the hardness of a community service, we expose them to that but in a controlled way and we hope that the concept of width with a varying degree of depth with increasing competencies in the intern year, will allow them to be properly armed for the whole thing.’

Although the curriculum is no longer characterised by what the academic staff consider important on the day, there is still a balance to be struck between the breadth and the depth. ‘In the US you have all these [clinical] offices next to one another … but here we have hundreds of kilometres between people, so that is quite difficult. So wide, wide knowledge, but not necessarily deep, means that they [the students] must be able to recognise and not necessarily be able to deal with everything … We use what we call the Nijmegen criteria.’

Balancing the interests of the many and the interests of the few

Each school of medicine has to decide on what must be included in the curriculum, and how much time each inclusion will have. ‘Historically, when I was a medical student, um, the curriculum was driven by what the lecturers thought the students should know. So it was totally a lecturercentred thing, giving us a story and, in fact, they dictated so we had to write down. So that has soon been replaced then by a sort of a student-centred thing where we … identify themes and the students must solve problems.’

The Nijmegen criteria that provide a typology for the levels of knowledge and skills required by a student are useful to curtail dominance of any particular academic staff member’s personal research or clinical interest within the curriculum. ‘Our motto says, a wide but not necessarily deep … and the depth varies with the different conditions. It’s a bit like the seabed. Wide, however, is essential because they are out there in the field. So that is why we expose them to 18 blocks, 13 special activities and 12 exit rotations as a final-year student and that defines the width.’ But this decision also comes at a price: ‘… that means that we spread the marmite very thin on the bread.’ The decision to focus on ‘messages of the reality and not necessarily the messages of complete academic theory’ is not necessarily met with universal approval, in that ‘in most people’s hearts there is a desire to cover everything.’ Clearly, these curricular decisions have to be reinforced to work against the desire to have saturation of theory as ‘the not … not covering everything is also a new thing that we have to work on’ and this constant balancing results in a changing stability ‘so there is evolution all the time’. But it is not the constant struggle for balance between breadth and depth of the curriculum alone that is the cause of the VD’s insomnia: ‘My difficulty does not come with the departments or the blocks that have a standardised programme. My difficulty is those who still make use of whatever patient is in the ward.’ The pull of the past curriculum can still be felt in the present curriculum: ‘My personal favourite is that while in this curriculum, we still have one lecture for prostate cancer, which is the leading cancer of men and one lecture of cervix cancer, which is the leading cancer of women and while we have one lecture for heart attack … we spend a week on the Krebs cycle, which nobody has actually seen.’ The push and pull continues also with placing blocks within the curriculum, with slots within the senior years more desirable than the early years: ‘Not everybody can have the students for the last block … everybody would like that.’

The global consensus on social accountability considers the priority health concerns of the community as the departure point for education, research and service delivery.[12] However, the priority needs of the community (or the many) are not necessarily in line with the scientific interests of the academic staff members (or the few). Even within the lecture halls, this tension between what students need to know (the common) v. what is cutting-edge science, is played out: ‘Listen, the wheel keeps on turning and … now we know a lot of things. We know about the human genome. We know about multidrug-resistant [tuberculosis] TB. We know about all those things but your training should reflect what’s happening in your community.’ This strain is not felt by academic staff alone: ‘And the student on the other side sits there and I think everything weighs kind of equally. Gynaecology – we see one new patient with cervix cancer every day and most days more than one new patient with cervix cancer, which is, which is … a lot. And we see about one patient with cancer of the fallopian tube every third of a year. So we might see three a year. Yet the students will take the two chapters in the book as equivalent.’ The central anchoring point is to ensure a ‘new commitment for a gain for relevance [to undergraduates]’ and part of this striving for relevance is the inclusion of public health in the curriculum.

Balancing the individual focus and population focus

The debate surrounding the inclusion of a population focus, such as public health, in the curriculum that focuses on individuals, is well summarised.[13] The decision of the HPCSA to include public health in the medical curriculum renders the decision moot. What remains, however, is not ‘should we’ but questions of ‘how’, ‘when’ and ‘why’? It was this inclusion that was of primary interest for the interviews and it became clear that there has been some evolution in the curriculum: ‘But that is basic sort of old-style public health, where you get health inspectors and that sort of stuff … we have moved on.’ The status of the health of the public now serves to inform the medical curriculum: ‘We have an epidemic of trauma and, uh, violence … that can only be known if you look at what is happening in the community and from there on, the competencies should be designed to deal with crises and so on, uh, with lesser events. Which doesn’t mean that we shouldn’t tell them about Addison’s disease or something that happens rarely.’ This role of public health, not only to inform, but also to interpret, is elaborated on:

May 2016, Vol. 8, No. 1 AJHPE

Research ‘ … public health medicine must tell us what is happening, what it’s prioritising and what is the context and hopefully that will lead to a complete understanding of where we are.’ Among the myriad decisions that surround the inclusion of public health, none are so central than whether to embed public health in mainstream medical teaching or encapsulate public health in one or more stand-alone modules/blocks/activities. The inclusion to embed public health relies on the academic staff from the school of medicine feeling able to include a population perspective. ‘So if we talk about cervix cancer, I’m supposed to give a public perspective as well as not.’ One negative design effect of embedding public health is that it is then virtually impossible to evaluate the inclusion and this in turn is problematic in an environment that requires evidence of the inclusion for accreditation purposes: ‘On the other hand, one of the things that we have to be comfortable with is whether … it’s … this behind the scenes and visible work … so to be rated on something that is behind the scenes will lead to a limited yield.’

Balancing teaching in the academic complexes and in the field

In SA, filling clinical posts outside the metropolitan areas is notoriously difficult and finding those who are then also prepared to support students at these teaching sites is a constant headache: ‘So at least there is one teacher per hospital where the students go who have actually said, “I am prepared to take part in this process as a teacher”. Better than that in our social economic situation, and our staff situation, is pretty hard to get.’ Difficult or not, this imperative of widening the training platforms is unavoidable, and is driven by trends in medical education, rising student numbers and the need to reduce the dominance of hospital-centric training for students. This dispersed model of training brings about an additional tension, that of involving others: ‘Ah, the hard thing is that we then delegate our in loco parentis responsibility to people who are very far from us, and who are not necessarily buying into the system of being a teacher.’ In many cases, the health professionals in question are not fellow medical colleagues: ‘… so the sisters in the clinics don’t see these visits that the kids come and do every two weeks and … they say “Oh, here they are again and we already have 300 people waiting here.” So that’s a hard match at the moment.’ An additional strain is that of content in the field: ‘Community-based teaching, where the students are predominantly trained in the community, depends on a principle that they will see what they need to see because they are there.’ But the ability to handle ambiguity is a key ability for a dean, not the average medical student. The counterweight argument of student support, through guided reflection of these community experiences, is then yet another factor to consider.

May 2016, Vol. 8, No. 1 AJHPE

If support of the students in training is not problematic enough, then preparation of students to cope beyond the reach of support is almost unimaginable. Much of the clinical practice and experience required for final professional registration takes place in the community-service years. But reality is somewhat different from expectations in that ‘… they work then in the state sector which was supposed to be a comm[unity] serv[ice] year under supervision but that has been abandoned because there is no supervision. So they now work within the constraints of what they are presented with … and that means that there is not necessarily an expected growth in procedural competencies.’

Balancing international and local realities

The commissioned Lancet article outlined what international authors consider the third reform period in medical education.[14] What is impor tant in the context of this article is not the content of that reform, but the acknowledgement that these sentinel theories expand our local thinking about medical education. One example is the question of whether to make use of interprofessional education: ‘I’m not sure that that’s the ideal myself because … any programme that you enter, you have to have … a prime knowledge. So, if the knowledge base of all the professions is the same, then it’s pretty easy to go into a joint programme. If the knowledge bases are different or if the expectations or the objectives are different, you will have a staggered, non-equal outcome of a joint thing.’ Resisting the attraction of opting for the simple adoption of international educational strategies is the messy complexity of our local reality. We are: ‘… still developing [as] an evolutionary system [where] patient numbers is a strength, it is not a problem. We all have our own problems, it is just ours seem less exotic so that it is a dangerous phenomenon to think that if you talk about great things that they see on the TV, then they think that this is the real thing, whereas the rest, the bread and buttery sort of stuff, might not be regarded as important.’ Feedback from the critical readers supports this authentic account of the experience of being a dean or VD of medicine, with one respondent who added two additional sources of tension: the demands of the Department of Health as the employer and the demands of the regulatory bodies.

Summary

This article outlines a complex picture of forces and counterforces on the medical curriculum in general, and the energy required by the VD to bind these forces into a stable whole. There are subthemes of balancing: those who use blocking strokes and those who hit for the boundary in education; the breadth and depth of the curriculum; the interests of the many and the interests of the few; the individual focus and population focus; teaching in the academic complexes and in the field; and international and local realities. These emphasise the required systems thinking ability of the VD, who needs to balance multiple conflicting interests to benefit the institution.[2] The ontological reality of the VD in this research brings to life the qualifications, experience, leadership and management skills, and roles and abilities, described in the literature. While leadership and support of the dean are necessary for enhancing the teaching of public health,[15] the complex forces of often-opposing educational choices is a significant factor to effect change. The literature suggests that within this reality of multiple

Research educational choices, the development of a common educational vision is fundamental to effecting the required changes.[1,2,4]

Conclusion

This article is an in-depth description of the ontological reality of one VD of medicine at one medical school. Through crystallisation, it became apparent that this reality is a common one for those who occupy similar positions. The forces that exert pressure on the curriculum, such as international or local realities, are an inescapable aspect of practice. Similarly, deans of medicine have to strike a balance in teaching in the academic complexes and teaching in the field. The debate on the inclusion of a population focus in the individual focus of medicine should never end, and the interests of the many do not have to annihilate the interests of the few. Those who wish to bring about change in any medical curriculum – in this case strengthening the teaching of public health – need to be cognizant of the pressure of educational choices that act on the dean and need to be able to create a common educational vision to justify the changes. Acknowledgements. The authors would like to acknowledge the contribution of the following critical readers for their valuable input: Prof. M. Chidzonga, College of Health Sciences, University of Zimbabwe; Prof. N. Ebrahim, Director

of the School of Medicine, University of Limpopo (MEDUNSA), South Africa; and Prof. J. Frantz, Deputy Dean of Research, Faculty of Community and Health Sciences, University of the Western Cape, South Africa. References 1. Bassaw B. Determinants of successful deanship. Med Teach 2010;32(12):1002-1006. [http://dx.doi.org/10.3109 /0142159X.2010.497821] 2. Lee A, Hoyle E. Who would become a successful dean of faculty of medicine: Academic or clinician or administrator? Med Teach 2002;24(6);637-641. [http://dx.doi.org/10.1080/0142159021000063970] 3. Chapman J. Reflections on the medical deanship. Acad Med 1998;73(6):654-656. [http://dx.doi.org/10.1097/00001888199806000-00011] 4. Rich E, Magrane D, Kirch DG. Qualities of the medical school dean: Insights from the literature. Acad Med 2008;83(5):483-487. [http://dx.doi.org/10.1097/ACM.0b013e31816becc9] 5. Collins-Nakai R. Leadership in medicine. Mcgill J Med 2006;9(1):68-73. 6. Yedidia MJ. Challenges to effective medical school leadership: Perspectives of 22 current and former deans. Acad Med 1998;73(6):631-639. [http://dx.doi.org/10.1097/00001888-199806000-00007] 7. Charmaz K. Constructing grounded theory: A Practical Guide Through Qualitative Analysis. Thousand Oaks, California: Sage Publications, 2006:123-150. 8. Whitehead J, McNiff J. Action Research: Living Theory. London: Sage Publications, 2006. 9. Tesch R. Qualitative Research. Analysis Types and Software. London: Falmer Press, 1990. 10. Nieuwenhuis J. Qualitative research designs and data gathering techniques. In: Maree K, ed. First Steps in Research. Pretoria: Van Schaik Publishers, 2007:70-97. 11. Regehr G. It’s NOT rocket science: Rethinking our metaphors for research in health professions education. Med Educ 2010;44(1):31-39. [http://dx.doi.org/10.1111/j.1365-2923.2009.03418.x] 12. Global Consensus for Social Accountability of Medical Schools, 2010. http://my.ibpinitiative.org/Community. aspx?c=c5357538-ce2a-4627-94f6-6110addbe047 (accessed 22 April 2013). 13. Woodward A. Public health has no place in undergraduate medical education. J Public Health Med 2004;16(4):389-392. 14. Frenk J, Chen L, Bhutta Z, et al. Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. Lancet 2010;376(9756):1923-1958. [http://dx.doi.org/10.1016/S01406736(10)61854-5] 15. Johnson I, Donovan D, Parboosingh J. Steps to improve the teaching of public health to undergraduate medical students in Canada. Acad Med 2008;83(4):414-418. [http://dx.doi.org/10.1097/ACM.0b013e318166a8e4]

May 2016, Vol. 8, No. 1 AJHPE

Research Relationship between student preparedness, learning experiences and agency: Perspectives from a South African university N V Roman,1 PhD; S Titus,2 MA; A Dison,3 PhD 1

Department of Social Work, Faculty of Community and Health Sciences, University of the Western Cape, Cape Town, South Africa

Department of Sport, Recreation and Exercise Science, Faculty of Community and Health Sciences, University of the Western Cape, Cape Town, South Africa

Teaching and Learning, Faculty of Community and Health Sciences, University of the Western Cape, Cape Town, South Africa

Corresponding author: S Titus (sititus@uwc.ac.za)

Background. One of the more discernible needs that challenges universities is addressing the level of preparedness of students entering the higher education environment. Students expect to participate in active learning, while at the same time adopting a certain level of agency to successfully pass through higher education. Objective. To determine the relationship between student preparedness, learning experiences and agency of students in the Faculty of Community and Health Sciences (FCHS), University of the Western Cape (UWC), Cape Town, South Africa. Methods. A cross-sectional study was conducted on 266 (N=578) convenience sampled 3rd-year students in the FCHS. Data were collected with an instrument constructed from items of evaluation from the departments in the FCHS and other validated instruments. Results. Findings suggest that 3rd-year students perceive themselves as moderately prepared on enrolling at UWC (mean (SD) 13.74 (1.86)); current learning experiences are favourably indicated (94.04 (15.32)). On average, students perceive themselves to be agents of their own learning (51.56 (8.79)). Furthermore, a significantly positive relationship was found between learning experiences and agency. Conclusion. This study broadens our understanding of the Vygotskian perspective of the zone of proximal development, where students bring their own knowledge, interact with lecturers who scaffold their learning, and then become agents in their own learning. Afr J Health Professions Educ 2016;8(1):30-32. DOI:10.7196/AJHPE.2016.v8i1.490

This article explores how Vygotsky’s zone of proximal development (ZPD) can be applied to the teaching and learning of health science professionals in higher education. ZPD provides a conceptual understanding of how developmental potential might be understood within health science education in South Africa (SA). Vygotsky [1] defined ZPD as ‘the distance between the actual developmental level as determined by independent problem-solving and the level of potential development determined under adult guidance or in collaboration with more capable peers’. Furthermore, it celebrates the importance of the ‘social other’. According to Vygotsky,[1] social interaction precedes development, and consciousness and cognition are the end-products of socialisation and social behaviour. ZPD refers to a conceptual space/ gap between what students know and what they need to know.[1] This space generates a unique opportunity for academics to design learning activities that may facilitate the development of student agency and preparedness. Lecturers are therefore mediators in ZPD for student learning to occur and for students to become more active as learners. Thus the interaction between lecturer and student encourages more positive outcomes for the latter. In SA, the education needs at universities include addressing a general lack of academic preparedness, multilingual needs in English-medium settings, large class sizes and inadequate curriculum design.[2] The challenge for higher education institutions not only relates to increasing the throughput of students and diversity of the student population, but also involves the provision of quality education. Many university courses are theory driven, without much thought to students’ real-world experiences. Therefore, their ability to link theory and practice may be compromised.[2] A higher education report on student experiences posits that the existing cohort of students is not necessarily underprepared, and that failure to succeed lies more in systemic weaknesses in higher education.[3] Therefore, there is a need for academics to fully understand students’ thinking to deliver educational practices that will

May 2016, Vol. 8, No. 1 AJHPE

allow them to achieve their full potential, while bearing in mind that learning takes place on the basis of social activity.[4] While SA’s higher education system leads that of its African peers in research and postgraduate attainment, it is deeply affected by the same educational inequalities and poor educational performance that characterise the school system.[5] Furthermore, there is an indication that students who are not sufficiently prepared are not equipped to deal with self-directed approaches.[6] In a crosscultural study, considerable variation is observed between students in four settings with regard to perceptions of preparedness for higher education.[7] Lecturers are important role-players in the learning context for students, and the latter, in turn, are important role-players in the teaching environment for lecturers.[8] One of the principal objectives of a health science course is that students should progressively gain the ability to identify the relevant learning issues. These should encompass all disciplines to facilitate an adequate understanding of the clinical situation in need of improvement and to enable students to formulate clinical judgements and action plans. Each student’s prior knowledge helps to inform other students in the class by identifying essential learning needs.[9] Therefore, educational experiences are only as effective as students’ engagement with them,[10] because students determine how much effort is required to engage in the learning process, and real learning takes place through this engagement. ‘Agentic’ has been used to describe students who assert agency in their learning. Billet[11] describes them as ‘learners who are pro-active and engaged in making meaningful and developing capacities in ways that are intentional, effortful and are actively critical in constructing their knowledge’. Therefore, students’ readiness to take up and engage with the invitations being offered to them is central to their learning.[11] Guided by Vygotsky’s ZPD, the purpose of this study was to examine the relationship between students’ perceptions of their preparedness, learning experiences and agency in the Faculty of Community and Health Sciences (FCHS), University of the Western Cape (UWC), Cape Town, South Africa. The context for this study is an interprofessional health sciences faculty. Approximately

Research 2 700 undergraduate students are enrolled across nine departments/schools (Nursing; Physiotherapy; Occupational Therapy; Social Work; Psychology; Dietetics; Natural Medicine; Public Health; and Sport, Recreation and Exercise Science). We are not aware of studies that have examined the relationship between the variables for this study.

Methods

This article forms part of a larger study that sought to investigate the learning needs of health science students at a university in southern Africa. A cross-sectional study was conducted on 266 (N=578) 3rd-year students from eight departments/schools in the FCHS.

Sample

Participants for this study were a convenience sample of 266 students in their 3rd year of study towards a degree registered in a health science faculty. Academic co-ordinators for 3rd-year programmes in the faculty allowed the research assistants access to their 3rd-year classes. Data were collected using a modified self-administered instrument completed by students from eight different departments/ schools that offer undergraduate programmes.

Research instrument

The questionnaire was constructed from other relevant ones, including that by V Bozalek et al. (unpublished report on the Competitive Research Grant received from the Council for Higher Education, 2008). The instrument consists of scales and subscales that measure preparedness, learning experience, prior learning experience, self-esteem, perceptions of academic literacy and numeracy skills, and student agency. The instrument was piloted before implementation to determine the time it would take to complete, and to ascertain whether there were any items that would require amendments. The instrument showed high reliability with a Cronbach α of 0.85.

Procedures

Departments and schools in the health sciences faculty were invited to participate in the study. All departments assigned a 3rd-year cohort to the project. Students from these cohorts were invited to participate and provided informed consent. Data were collected face-to-face by research assistants assigned to the project. These assistants informed participants of the objectives of the project, the nature of the questionnaire, that participation was voluntary, and that they could withdraw at any time without undue consequences. The questionnaire took approximately 15 - 20 minutes to complete.

Permission to conduct the research was obtained from the Senate Research Grants Committee and the Ethics Committee at UWC. All information was strictly confidential and pseudonyms were used to protect anonymity of the participants.

Data analysis

Data were double captured in Microsoft Excel 2010. After matching both sets of data, the clean data were imported into SPSS Statistics version 20 (IBM, USA) for analysis by means of descriptive and inferential statistics. Descriptive statistics included percentages, means and standard deviations. Crosstabulations were conducted in terms of gender. Inferential statistics included a correlation and linear regression analysis. An R-value of 0.0 - 0.2 suggests a weak relationship, 0.3 - 0.5 a moderate relationship and 0.6 - 1.0 a strong relationship.

Results

Four hundred questionnaires were distributed to the 3rd-year students (N=578), of which 266 were returned, indicating a response rate of 67%. Seventytwo percent of the respondents were female. The majority of respondents were from the psychology department (Table 1). The mean (SD) age of the participants was 23.57 (4.91) years. English was the first language of the majority of participants (48%); 61% did not have work experience before entering university; 47% considered their socioeconomic status as average; 41% had previously attended a historically disadvantaged school (41%); and 58% identified themselves as coloured. For the purpose of this study, historically disadvantaged schools are those situated in poverty-stricken areas, mostly townships, rural and farm areas, which are characterised by poor socioeconomic conditions and poor educational infrastructure and resources.[12] The results in Table 2 show that the majority of students considered themselves to be moderately prepared (13.74 (1.86)) and their learning experien ces to be favourable (94.04 (15.32)). On average, students perceived themselves to be agents of their own learning (51.56 (8.79)). Further crosstabulations were conducted in terms of gender. The results suggest that 43% of females and 38% of males were moderately prepared for university, even though the majority of participants were female. Moreover, a correlation indicated significant posi tive relationships between student learning experiences and student preparedness (r=0.16; p<0.01), as well as student learning experiences and student agency (r=0.34; p<0.05). No relationship was found between student preparedness and student agency. Two linear regression analyses were conducted (Table 3) to predict student agency. Student learn

Table 1. Demographic profile of students in the Faculty of Community and Health Sciences, UWC Variables

Students, n (%)

Gender Female

187 (72.5)

Male

71 (27.5)

Faculty of Community and Health Sciences Psychology

80 (30.1)

Nursing

55 (20.7)

S port, Recreation and Exercise Science

35 (13.2)

Physiotherapy

30 (11.3)

Social work

24 (9.0)

Dietetics

15 (5.6)

Natural Medicine

16 (6.0)

Occupational Therapy

9 (3.4)

Ethnicity Coloured

151 (58.1)

Black

75 (28.8)

Indian

17 (6.4)

White

17 (6.4)

Living arrangements Both parents

123 (47.7)

Only mother

49 (19.0)

In residence

33 (12.8)

Only father

1 (0.4)

Language English

123 (47.5)

Afrikaans

66 (25.5)

isiXhosa

52 (20.1)

Type of school previously attended Historically disadvantaged school

107 (41.3)

Ex-model C school

98 (37.8)

Independent school

30 (11.6)

Work experience prior to entering university No

161 (61.0)

Yes

103 (39.0)

Perceived current socioeconomic status Very disadvantaged

13 (5.0)

Disadvantaged

38 (14.7)

Average

122 (47.3)

Advantaged

73 (28.3)

Very advantaged

12 (4.7)

May 2016, Vol. 8, No. 1 AJHPE

Research ing experiences in the classroom accounted for 11.3% of the variance for student agency. Student preparedness did not predict student agency.

enhanced. However, it is a huge undertaking for a higher education institution to attempt to redress inequalities in the formal education system to address the underpreparedness of students. These findings are similar to those of Brüssow and Wilkinson [15] with regard to learning experiences and underprepared students, where students also had favourable learning experiences. Many university courses are theory driven, and assume that students have knowledge of real-world experiences that lead to them linking theory and practice.[2] The higher education report on student experiences suggests that students are not necessarily underprepared, and that failure to succeed lies more in systemic weakness than in higher education.[3] This implies a need for a deeper understanding of students who strive to achieve their full potential. This study showed a positive relationship between learning experiences and student agency. Much has been said about poor graduation rates and a diminished learning culture among students. This study negates these arguments, proving that there is an indication that students who have a better learning experience take more responsibility for their own learning. The results also provide an understanding of the Vygotskian perspective of the role of the learning experience for underprepared students to become agents of their own learning, i.e. that lecturers provide the learning experience for students to function optimally and be self-directed within the space of the ZPD. Student agency cannot and should not be ignored in the teaching and learning process.[16] Students’ power (their agency) cannot be ignored as they negotiate their needs, which can be overdetermined by their social background,

Discussion

This study provides the first known information with regard to the relationship between student preparedness, learning experiences and agency. A large proportion of students are enrolled in the psychology and nursing fields – two of the largest departments in the faculty; however, the responses are a representation of a range of views from students across the faculty. Participants reported that they were moderately prepared for university. This is not surprising, as a significant number of students are underprepared when entering higher education settings,[13] regardless of gender. While the current challenge facing higher education institutions is not only about increasing throughput in terms of numbers and diversity of student population, it also involves ensuring quality education. Therefore, educational needs at universities should include addressing a general lack of academic preparedness, multilingual needs in English-medium settings, large class sizes and inadequate curriculum design.[2] Despite the moderate level of preparedness, a positive relationship between perceptions of student preparedness and learning experiences was observed. One of the major factors relating to low graduation rates in SA higher education is underpreparedness of students.[14] Therefore, if students are better prepared for higher education before entering university, their experience in higher education should be greatly Table 2. Associations between variables Variables

Minimum

Maximum

Mean (SD)

Student learning experiences

Student preparedness

13.74 (1.86)

0.16*

Student agency

51.56 (8.79)

0.34†

Student learning experiences

128

94.04 (15.32)

*Correlation significant at the 0.05 level (two-tailed t-test). † Correlation significant at the 0.01 level (two-tailed t-test).

Table 3. Regression analyses predicting student agency Predictor

SE B

Constant (student agency)

45.52

4.55

Student preparedness

0.44

0.33

Constant (student agency)

33.04

3.56

Student learning experiences

0.20

0.04

ΔR²

1 0.09

*Regression is significant at p<0.05 level.

May 2016, Vol. 8, No. 1 AJHPE

0.34*

11.3

available resources, campus organisations and external pressures. Agency is seen as an important empowering and disempowering factor, as it enables and constrains the interpretation of actions, society and social interactions with people.[14]

Conclusion

This study was limited to one of the 24 higher education institutions in SA. The sample included a larger proportion of females, which may suggest a bias, but moderate preparedness was found for all students regardless of gender. The study indicates that there is a very enmeshed and complex relationship between teaching and learning, which may further be complicated by other factors, which were not the focus of this study. However, our study showed that students who are better prepared for the higher education setting have better learning experiences. These settings may provide higher education specialists with a unique opportunity in ZPD to provide academic activities that enhance a positive learning experience. Acknowledgements. The authors would like to acknowledge the participants in this study for their meaningful contribution, and the Directorate of Teaching and Learning at the University of the Western Cape for their funding assistance. References 1. Vygotsky LS. Mind in Society: The Development of the Higher Psychological Processes. Cambridge, Mass: Harvard University Press, 1978. 2. Jaffer S, Ng’ambi D, Czerniewicz L. The role of ICTs in higher education in South Africa: One strategy for addressing teaching and learning challenges. International Journal of Education and Development using Information and Communication Technology 2007;3(4):131-142. 3. Council on Higher Education (CHE). Access and Throughput in South African Higher Education: Three Case Studies. Pretoria: CHE, 2010. 4. Kinginger C. Defining the zone of proximal development in US foreign language education. Applied Linguistics 2002;23(2):240-261. 5. Fisher G, Scott I. Closing the Skills and Technology Gap in South Africa. Background Paper 3: The Role of Higher Education in Closing the Skills Gap in South Africa. Cape Town: World Bank, 2011. 6. McMillan VK, Parke S. Remedial/developmental education approaches for the current community college environment. New Directions for Community Colleges 1997;25(4):21-32. 7. Byrne M, Flood B. A study of accounting students’ motives, expectations and preparedness for higher education. Journal of Further and Higher Education 2005;29(2):111-124. [http://dx.doi.org/10.1080/03098770500103176] 8. Richardson JTE. Students’ approaches to learning and teachers’ approaches to teaching in higher education. Educ Psychol 2005;25(6):673-680. [http://dx.doi.org/10.1080/01443410500344720] 9. Cooke M. Integrating knowledge in clinical practice. In: Alavi C, ed. Problem-Based Learning in Health Science Curriculum. New York: Routledge, 1995:104-115. 10. Richards J, Sweet L, Billett S. Preparing medical students as agentic learners through enhancing student engagement in clinical education. Asia-Pacific. J Cooperative Educ 2013;14(4):251-263. 11. Billett S. Developing Agentic Professionals Through Practice-Based Pedagogies. Final Report for Australian Learning and Teaching Council (ALTC) Associate Fellowship. Strawberry Hills, NSW: ALTC, 2009. 12. Xaba M, Malindi M. Entrepreneurial orientation and practice: Three case examples of historically disadvantaged primary schools. S Afr J Educ 2010;30(1):75-89. 13. Slominsky L, Shalem Y. Reading below the surface. Students’ organization of content and form. Council for Higher Education Monitor 2010;10:81-109. 14. Council on Higher Education (CHE). The State of Higher Education in South Africa. Pretoria: CHE, 2009. 15. Brüssow S, Wilkinson A. Engaged learning: A pathway to better teaching. S Afr J Higher Educ 2010;24(3):374-391. [http://dx.doi. org/10.4314/sajhe.v24i3.63444] 16. Luckett K, Luckett T. The development of agency in first generation learners in higher education: A social realist analysis. Teaching in Higher Education 2009;4(5):469-481. [http://dx.doi.org/10.1080/13562510903186618]

Research Perceptions of undergraduate dental students at Makerere College of Health Sciences, Kampala, Uganda towards patient record keeping A M Kutesa,1 BDS, MSc (Dent); J Frantz,2 PhD 1

Department of Dentistry, College of Health Sciences, Makerere University, Kampala, Uganda Department of Physiotherapy, Faculty of Community and Health Sciences, University of the Western Cape, Cape Town, South Africa

Corresponding author: A M Kutesa (akutesa@chs.mak.ac.ug)

Background. The creating, maintenance and storage of patients’ medical records is an important competence for the professional training of a dental student. Objective. Owing to the unsatisfactory state of dental records at the students’ clinic, the objective of this study was to obtain information from undergraduate dental students on the factors that affect this process and elicit recommendations for improvement. Methods. This qualitative cross-sectional study used focus group discussions with 4th- and 5th-year dental students for data collection. Data were captured through a written transcript and an audio recorder. The data were transcribed and analysed manually through developing themes, which were compared with the literature and interpreted. Results. Three themes emerged: (i) Poorly designed clerking forms. The clerking forms were deemed to have a poor design with inadequate space for clinical notes. It was recommended that they be redesigned. (ii) Inadequate storage space. Space for storing patient records was deemed inadequate and a referencing system for file retrieval was lacking. It was recommended that more space be allocated for storage, with a referencing system for easy file retrieval. (iii) Poor maintenance of records. Patients’ records, especially radiographs, were not well labelled and stored. It was recommended that drug envelopes be utilised to store radiographs. An electronic system was deemed the ultimate solution to this problem. Conclusion. The general perception was that the current paper-based record system at the clinic was unsatisfactory. Therefore, there is a need to improve the maintenance and storage of records, and to change to a more efficient electronic system. The students’ attitude towards record keeping was found to be questionable, with a need to be addressed as part of teaching and learning in the curriculum. Lecturers were deemed to have a bigger role to play in the record-keeping process. Afr J Health Professions Educ 2016;8(1):33-36. DOI:10.7196/AJHPE.2016.v8i1.521

The creating of clear and accurate patient medical records, and their maintenance and storage, are an important part of the professional training of a dental student. The records of an individual patient in a dental setting constitute the clinical notes, radiographs and plaster models, as well as other information. These records present an in-depth and continuous documentation of the patient’s condition, which helps to contribute to both diagnosis and proper patient care.[1,2] In addition, these records have various other purposes, including research, administration, quality assurance, teaching and learning, and can be used as evidence during legal proceedings.[3,4] Patient records have traditionally been paper-based but electronic documentation, in addition to residual paper-based records, is being increasingly used because of the inherent disadvantages of paper-based systems. Documented problems of paper-based systems in the literature[5] include difficulty in deciphering clinician handwriting, difficulty in accessing patient information from remote centres, time-consuming patient chart retrieval, and questionable security and confidentiality of records.[6] The other challenge identified with paper-based records is their incompleteness,[6,7] i.e. the omission of vital information, such as laboratory results or radiography reports, from the patient record. Missing information has been shown to have consequences, including delays in patient care, repeated laboratory testing and additional patient visits, which also adversely affect the wellbeing of patients. In addition to the pressure to reduce patient length of stay in hospital, this highlights the need for investigation into

the use of electronic records. The pace of modern practice dictates the use of technology, but the lack of direct clinical input in the development of information technology is a major factor in the failure of many clinical information systems.[8] Replacing paper records with electronic records will assist in changing medical practice, including relationships between doctors and nurses, responsibilities, and work distribution. However, clinicians need to involve themselves in ensuring that software for documenting patient encounters complements the way they work. Documentation and note taking continue to be an area of concern because quality documentation is an important aspect of communication by health professionals. The notes of any health professional serve as a window into their thought processes.[9] During student training, communication through note taking is an important aspect of exchange of information with colleagues and patients. Therefore, teaching and learning of students in written and verbal communication skills is a key component of health professional curricula. Some academics[10] have advocated that it is now more important than ever that education and practice in professional writing and speaking should be included as part of medical training and residency. Furthermore, national quality-regulating bodies such as the Accreditation Council for Graduate Medical Education in the USA, a body that establishes educational standards, have included interpersonal skills and communication as one of six core competencies for all training programmes.[11] At the Makerere College of Health Sciences (MakCHS), Kampala, Uganda, teaching and learning in interpersonal and communication skills are included in the undergraduate dental curriculum. These skills are expected to be

May 2016, Vol. 8, No. 1 AJHPE

Research demonstrated by the students, particularly during the clinical years, when they interact with other professionals and patients through record keeping. At MakCHS dental clinic, the record-keeping system is paper based and patient records have recently had many challenges, which include the loss of patient records, poor status and incomplete records. Moreover, the storage of the patient records is inadequate, using cardboard boxes as a filing system, making it difficult to trace patient information when they come for recall visits. These challenges have had an effect on student learning and patient care. Despite these concerns, no study has been done among the faculty and students to document their perceptions of the status quo. The objective of this study was to document the perceptions of undergraduate dental students at MakCHS of factors that affect the keeping of clear and accurate patient records, and their maintenance and storage. It also elicits recommendations to improve the system for better teaching and learning at MakCHS.

similar to others and whether the findings can be applied to other settings. Dependability has been achieved by describing the methodology clearly to enable a future investigator to repeat the study. To achieve conﬁrmability, the ﬁndings that emerged from the data were supported by quoting the participants’ words during the discussions.

Data analysis

This was a cross-sectional study using descriptive qualitative methodology, conducted at the Department of Dentistry, MakCHS.

After the interviews, the data were transcribed and checked for correctness by cross-checking with the three data sources (the notes of the researcher, the research assistant and the recorder). Data were coded by the author using the open code system.[13] Coding was done by first using a start list of preset codes. The initial preset codes were three: making, maintenance and storage, which were derived from the study objective. Themes were developed from the coded data with the help of an expert in qualitative analysis, and the emerging themes were compared with the data and finally interpreted. The participants’ quotes were identified and labelled using a method devised by the authors. As an example, a quote would be labelled with the year of study as the first digit and the participant number as the second digit. Y3P1 would signify a 3rd-year student, participant 1. The emerging themes were checked by one of the authors (JF).

Population and sampling

Ethical considerations

Data collection methods

Results

Methods

Research setting and design

The study population comprised 4th- and 5th-year dental students (n=20) and faculty members (n=2). Fourth- and 5th-year students were selected to participate because, at this time, they were in their clinical clerkships and actively used medical records as part of patient care. Of the 18 faculty members at the Department of Dentistry, two were purposefully selected to participate in the study, as they were actively involved in student training for this competence and supervision at the clinic.

The instrument used for data collection comprised a structured focus group discussion exploring the creating, maintenance and storage of patient records. The interview guide was developed by the authors to ensure that participants were allowed to discuss their perceptions concerning different aspects of patient record-keeping competence, which included knowledge on patient charting, its existence in the curriculum, gaps identified, and recommendations to improve the curriculum. Two focus group discussions were conducted. One group comprised 4th-year dental students and the other 5th-year dental students. The dental classes are small, with 10 - 15 students per year, and thus all students were invited to participate in the study. A total of 10 students per focus group were interviewed, and the discussions took place in a lecture room at the Department of Dentistry for an hour each. Focus group discussions were conducted by one of the researchers (AK) using a structured interview guide. A 5th-year dental student was used as a research assistant and captured the discussions through a written script and a voice recorder. To establish the trustworthiness of this research, the concepts of credibility, transferability, dependability and confirmability were considered as decisive factors for quality, based on Guba’s four criteria for trustworthiness.[12] Credibility was addressed by describing the methods in detail in the protocol to convey the actual situations that would be investigated. To allow transferability, sufficient detail of the study context has been reported in this article to enable other researchers to decide whether the setting is

May 2016, Vol. 8, No. 1 AJHPE

Ethical clearance to carry out the study was obtained from the School of Medicine Review and Ethics Committee. Prior to commencement, gatekeeper permission was obtained from the Department of Dentistry and the Faculty. Written consent was obtained from the selected participants in accordance with the Helsinki Declaration. The investigators informed the participants of the procedure, its objectives and benefits, and any possible risks involved. Confidentiality was ensured by avoiding the use of any person identifiers.

The two focus group discussions consisted of a total of 20 participants, of whom 10 were female and 10 were male. The 5th-year group had 7 females and 3 males, and the 4th-year group had 4 females and 6 males. The transcribed data from the interviews produced three themes: poorly designed clerking forms, inadequate storage space and poor maintenance of records. The following issues within the themes were highlighted: inadequacy of the currently used clerking forms, patient files, labelling and the electronic system.

Poorly designed clerking forms

It emerged that the current clerking forms were in need of a redesign as they have insufficient space to make clinical notes, and seemed disorganised in design. In addition, owing to the texture of the paper used, they were not easy to store. ‘Clerk forms do not have enough space for patient recall notes.’ (Y5P2) ‘The clerk form is disorganised, they need to be re-designed.’ (Y5P1) ‘Record keeping should be improved.’ (Y4P6)

Inadequate storage space

The participants felt that the current space for storing patient records was inadequate. In addition, referencing to facilitate patient file retrieval needed updating. It was recommended that management use box files to store

Research patient records and provide a bigger storage area with a reference system for ease of file retrieval. ‘We need more room to store the patient records, our lockers are too small.’ (Y5P3)

‘Provide box files for each of us to keep the patient records.’ (Y4P2)

among other details. These records should be kept securely and in line with dataprotection requirements of the institution for a period of not less than 10 years from when the patient last attended the clinic for treatment.[17] Owing to the inadequacies observed by the authors with patient records at MakCHS dental school clinic, the next step was to obtain information and create awareness among the undergraduate dental students and the faculty of the factors that affect this process, with the overall objective of improving the quality of service rendered by dental professionals. In view of the international guidelines discussed above,[15,16] we found that the students felt that the dental records were being inadequately stored and maintained at the clinic. We thought this was unfortunate because accurate records, in part, are considered important for good clinical decisions[13] and, therefore, better treatment outcomes. The students expressed the need for more training in record keeping and guidance from the supervisors. This information was beneficial to educators and will guide us in improving student training at the clinic, especially supervision. This finding is corroborated elsewhere, where students have been found to be a source of information that can provide formative feedback to faculty for improving teaching, course content and structure.[18] We also found that the students were able to give recommendations that are practical and easy to implement. As an example, they suggested the use of drug envelopes and box files to store patient radiographs and records. These recommendations we believe will be taken into consideration as we plan to improve patient record storage and maintenance. The reaction of the students in this study was interesting, showing that they expected the challenges to be solved by someone else. They felt that the responsibility for good record keeping lay entirely with others rather than with the students. They commented that someone should be hired for record keeping and that lecturers had a bigger role to play in record keeping. This showed us that the students were aware of inadequate recordkeeping practices at the clinic, but that they took no initiative to effect change to improve the situation. This implies that the students cannot easily show initiative to change challenging situations, even when they are aware of them, expecting others to take the lead. Therefore, we realise that students need to be taught to be agents of change, and advocate change in their work environment to improve practice.[19] This remains a challenge, as the problem will not be solved if students do not accept this responsibility during their training. Educators will need to ensure that this aspect in the curriculum is highlighted and receives the attention it deserves.

‘Someone should be hired for record keeping.’ (Y4P2)

Conclusion

‘ We need a reference system to be able to retrieve old patient records.’ (Y4P9) ‘ We don’t know how to get the records, apart from using patient names.’ (Y4P9) ‘ Sometimes we end up reclerking or asking the patient their previous treatment.’ (Y4P9)

Maintenance

It emerged that patient records, especially radiographs, were not well labelled. Furthermore, their storage was poor. It was unanimously recommended that drug-dispensing envelopes should be labelled and used to store patient radiographs; the introduction of an electronic system to improve storage was also recommended. ‘X-rays should be kept in labelled envelopes and stored in files.’ (Y5P6) ‘We need the electronic system for better storage.’ (Y5P7) The consensus was that there was an urgent need to introduce an electronic system for better record keeping. Respondents also felt that the lecturers had a bigger role to play in record keeping but had not emphasised the aspects of maintenance and storage of patient records. The students also felt that they needed more training in record keeping and guidance from the supervisors. ‘All lecturers at the clinic should emphasise record keeping.’ (Y5P1) ‘Supervisors at the clinic do not put in enough.’ (Y4P2) The recommendations that were given were practical and easy to implement. The students also thought that the responsibility of good record keeping lay with their supervisors and not with themselves. ‘X-rays should be kept in labelled envelopes and stored in files.’ (Y5P6)

‘ Lecturers have a bigger role to play in record keeping but they have not put a lot of emphasis on it.’ (Y4P1)

Discussion

Record keeping is an important competence that, as a key responsibility of the dentist in clinical practice, must be developed during the professional training of a dental student. A dentist is obliged, ethically and legally, to keep patient records that are accurate, legible, comprehensive and organised because appropriate decisions are based on accurate and complete facts retrievable from a patient’s record.[14] This is emphasised through various guidelines from medical and dental councils internationally,[15-17] which state that patient records must be clear and accessible. According to these guidelines, records must formally record patient details, clinical findings and treatment given,

The general perception was that the current paper-based record system at the clinic was unsatisfactory. Therefore, there is a need to improve the storage of records and their maintenance, and to change to a more efficient electronic system. The students’ attitude to record keeping was also found to be questionable, emphasising a need to be addressed as part of teaching and learning in the curriculum. Supervisors need to take on a more supportive role and guide the students towards good record-keeping practices. Medical record-keeping clinical skills are a core part of the training of a dental student and should be given the platform they deserve in the curriculum. Acknowledgements. We highly appreciate the support of both the Medical Educational Partnership Initiative/Medical Education for Equitable Services for All Ugandans (MEPI/MESAU) and the Sub-Saharan Africa-FAIMER Regional Institute (SAFRI) accorded during the course of this study.

May 2016, Vol. 8, No. 1 AJHPE

Research References 1. Cole A, McMichael A. Audit of dental practice record-keeping: A PCT-coordinated clinical audit by Worcestershire dentists. Prim Dent Care 2009;16(3):85-93. [http://dx.doi.org/10.1308/135576109788634296] 2. Osborn JB, Stoltenberg JL, Newell KJ, Osborn SC. Adequacy of dental records in clinical practice: A survey of dentists. J Dent Hyg 2000;74(4):297-306. 3. Charangowda BK. Dental records: An overview. J Forensic Dent Sci 2010;2(1):5-10. [http://dx.doi. org/10.4103/0974-2948.71050] 4. Soisson EL, van de Creek L, Knapp S. Thorough record keeping: A good defense in a litigious era. Professional Psychology: Research and Practice 1987;18(5):498. 5. Zandieh SO, Yoon-Flannery K, Kuperman GJ, Langsam DJ, Hyman D, Kaushal R. Challenges to EHR implementation in electronic versus paper-based office practices. J Gen Intern Med 2008;23(6):755-761. [http:// dx.doi.org/10.1007/s11606-008-0573-5] 6. Hersh WR. The electronic medical record: Promises and problems. J Am Soc Inf Sci 1995;46(10):772-776. 7. Smith PC, Araya-Guerra R, Bublitz C, et al. Missing clinical information during primary care visits. J Am Med Ass 2005;293(5):565-571. [http://dx.doi.org/10.1001/jama.293.5.565] 8. Walsh S. The clinician’s perspective on electronic health records and how they can affect patient care. Br Med J 2004;328(7449):1184-1187. [http://dx.doi.org/10.1136/bmj.328.7449.1184] 9. Cegala DJ, Broz SL. Physician communication skills training: A review of theoretical backgrounds, objectives and skills. Med Educ 2002;36(11):1004-1016.

May 2016, Vol. 8, No. 1 AJHPE

10. Simonson JA. Why we must teach written and verbal communication skills to medical students and residents. Acad Med 2013;88(4):435. [http://dx.doi.org/10.1097/ACM.0b013e3182854f57] 11. Accreditation Council for Graduate Medical Education (ACGME). Core competences. 2003. http://www.ecfmg. org/echo/acgme-core-competencies.html (accessed 14 July 2014). 12. Guba EG. Criteria for assessing the trustworthiness of naturalistic inquiries. Educ Commun Tech J 1981;29(2):75-91. 13. Lofland J, Lofland LH. Analyzing Social Settings: A Guide to Qualitative Observation and Analysis. Belmont, USA: Wadsworth Publishing Company, 1995. 14. Record Keeping Outline – Oregon.gov. www.oregon.gov/OBCE/publications/record_keeping_ch_final_jan06. pdf (accessed 14 July 2014). 15. The Guidelines of the Royal College of Dental Surgeons, May 2008. www.rcdso.org (accessed 8 July 2014). 16. Good medical practice. 2013. http://www.gmc-uk.org/guidance/ (accessed 14 July 2014). 17. Record keeping in the UK. 2012. http://www.dentalprotection.org/UK/RiskManagement/RecordKeeping/ (accessed 8 July 2014). 18. Marsh HW, Roche L. The use of students’ evaluations and an individually structured intervention to enhance university teaching effectiveness. Am Educ Res J 1993;30(1):217-251. [http://dx.doi.org/10.3102/00028312030001217] 19. Dunne E, Roos Z, Tony Brown T, Nurser T. Students as change agents: New ways of engaging with learning and teaching in higher education. York, UK: ESCalate/Higher Education Academy Publication, 2011.

Research Fieldwork practice for learning: Lessons from occupational therapy students and their supervisors D Naidoo,1 MOT; J van Wyk,2 PhD Education 1

Discipline of Occupational Therapy, School of Health Sciences, University of KwaZulu-Natal, Durban, South Africa

Clinical and Professional Education, Nelson R Mandela School of Medicine, School of Clinical Medicine, University of KwaZulu-Natal, Durban, South Africa

Corresponding author: D Naidoo (naidoodes@ukzn.ac.za)

Background. Fieldwork practice forms a vital part of occupational therapy (OT) education and contributes significantly to competent practice and students’ clinical reasoning. Students’ learning is positively or negatively influenced by their fieldwork experience. Objective. To explore the views and experiences of final-year OT students, site-based clinicians and university-based academic supervisors to identify strategies that influenced students’ learning during fieldwork practice. Methods. This descriptive qualitative study used a purposeful sampling technique. Data collection strategies included focus group discussions with clinical and academic supervisors and semistructured interviews with final-year students. Each set of data was analysed according to the research questions. The researcher analysed the data into themes, which were corroborated by a supervisor. Data source and analyst triangulation ensured trustworthiness of the study. Results. Two themes, i.e. difficulties experienced by students during fieldwork and supervision strategies that they found beneficial for learning, are described. Guidance and mentoring from experienced therapists helped students to link observations from assessments and intervention plans. Observations of treatment sessions, peer learning and practice in the skills laboratories were beneficial for learning, competence and confidence. Guided questions from supervisors to enhance reflexive practice and peer learning strengthened the students’ confidence and ability to give feedback to their peers. The students also benefited from sessions that allowed them the freedom and space to work autonomously. Conclusion. This study provides insight into the difficulties that students experienced when engaging with fieldwork and offers some strategies that have been found to advance their learning. Afr J Health Professions Educ 2016;8(1):37-40. DOI:10.7196/AJHPE.2016.v8i1.536

Students who enrol in occupational therapy (OT) at the University of KwaZulu-Natal (UKZN), Durban, South Africa (SA) follow a 4-year degree programme, which includes theoretical and practical components. Fieldwork placement constitutes the main practical com ponent of the undergraduate training programme[1] and involves rotation of students in groups of 2 - 4 through four 6-week placements at government hospitals or non-governmental organisations (NGOs). Fieldwork placement allows students opportunities for learning by integrating their theory into practice, conceptualising what professionalism entails and providing a chance to practise their professional skills in a supportive environ ment. Furthermore, it affords them the opportunity to engage with members of communities through service delivery.[2] They can also gain knowledge about their profession while learning how to practise as health professionals. In this way they learn to identify the strengths and shortfalls of their interventions, while developing a repertoire of professional practice and clinical reasoning skills.[2] Learning during fieldwork encompasses a multimodal and complex experience where students become engaged in socially authentic workplace experiences in environments that shape their learning.[3] Generally, on commencing fieldwork practice, they enter a complex and relatively unknown world, with differing social and cultural realities. There is therefore a need not only to prepare students for the variety of contexts where they will work, but also to ensure that they can benefit from the quality of the fieldwork exposures. The latter may include becoming familiar with the disintegrating social systems in primary healthcare settings; exposure to crime; and abuse, poverty and the realities and limitations of resources in their local areas.[4] The transition from student to practitioner requires that the theories and techniques initially acquired in the classroom context be transferred to the clinical context. Furthermore, planning and implementing a comprehensive contextualised client treatment

programme entails clinical reasoning and decision-making skills. A lack of adequate preparation may render the students’ therapy ineffective. With the entry-level OT practitioners’ learning firmly embedded in their ability to practise effectively in the community, the quest remains for academic institutions to ensure that students make a successful transition to the role of a novice practitioner, notwithstanding the challenging circumstances. In recognising the value of fieldwork for learning, the Health Professions Council of South Africa (HPCSA) has mandated that a minimum of 1 000 fieldwork hours be completed by OT students before their graduation.[1] The need to reach consensus on the concept of professional behaviour and for clinicians to guide students to attain the required competencies has been highlighted.[5] Clinical supervisors’ personal beliefs and prior experiences have been found to influence their assessment of students’ performance.[5] The need for training to improve the validity of student performance assessments is noted by Snyman.[6] Incongruence in rating students’ clinical performance is exacerbated in the absence of collaboration between site-based clinical supervisors and university academics. There is therefore a need for academic institutions to review the implementation of fieldwork practice, guidance and assessment of students during this important practical work-based experience. In the Discipline of Occupational Therapy at UKZN, final-year students rotate in groups of 2 - 4 through four 6-week fieldwork placements in their final year, forming the bulk of the 1 000 hours of clinical practice required for graduation. The sites for the fieldwork include placements at district or tertiary-level hospitals, NGOs and one peri-urban site. Given the severe staff shortages at a number of placement sites, some students may be expected to perform the duties of a qualified OT while being supervised by the only remaining OT at the hospital. The students are supervised primarily by the OT at the site; however, they receive additional supervision from an allocated academic supervisor once a week. This highlights the need for efficient

May 2016, Vol. 8, No. 1 AJHPE

Research supervision strategies to facilitate the student’s acquisition of professional behaviour and learning of practice skills. One of the key challenges for supervisors is fostering and developing professional behaviour of students,[6] which entails introducing new theoretical ideas and knowledge as a necessary foundation for practice. It also comprises experimenting with and evaluating the success of interventions in the practical and clinical context.[7] While authors agree that ethics, cultural competence, empathy and therapeutic use of self are important aspects of professional behaviour, limited literature exists to describe how these attributes can be taught to students. A considerable amount of literature has acknowledged the impact and difficulty brought about by the rapid expansion of tertiary education and the increased diversity of students because of broadened access and redress policies in the SA setting. The changes in students’ preparedness for tertiary training and characteristics of millennial students who seek training have implications for the skills of supervisors and effectiveness of instructional methods for training, including fieldwork.[8] In the absence of theories to explain how OT students negotiate fieldwork and supervision strategies that best assist their learning, this article reports on a study that explored the opinions of final-year OT students and their academic and clinical supervisors on the difficulties that students encounter during their fieldwork experience and the supervision strategies that they found beneficial during their various placements.

Methods

A descriptive qualitative study was used to gain an understanding of the perceptions and concerns of final-year OT students and their academic and clinical supervisors with regard to their fieldwork experience at an SA institution of higher learning.[9] Purposeful sampling was used to identify potential participants. All final-year students (N=21) who had studied at UKZN for the full duration of their degree, clinical supervisors (N=7) and academic supervisors (N=5) who supervised for ≥2 years were invited via email to participate in the study. All potential participants were sent information documents and the informed consent letter with a request to volunteer. Seventeen final-year students who were in the final semester of their undergraduate programme, 7 of their clinical supervisors and 5 academic supervisors agreed to share their views and experiences about fieldwork. The student sample consisted mainly of females (n=16) aged between 21 and 27 years, from all racial groups. The supervisors in the sample were aged from 25 to 59 years, with clinical supervisory experience from 2 to 36 years. The supervisors were white and Indian and only one was a male. There were three data sources, i.e. the students, supervisors and data gained from the curriculum documents. Two focus groups (n=13) and four semi structured interviews were conducted with students by two practising experienced OTs who were trained as research assistants. (The schedule of questions is given in Appendix A.) This allowed for student anonymity and provided greater reliability by reducing potential research bias, as the principal investigator was a member of the academic staff in the Discipline of Occupational Therapy at UKZN, which could have impacted on the veracity and integrity of the feedback. Two focus groups were facilitated by the principal investigator to collect data from the clinical supervisors and the academic supervisors, respectively. The focus group sessions and the interviews were audiorecorded and transcribed. Each set of data was analysed in answer to the research question. The researcher analysed the data into themes; these were corroborated by a super visor. Triangulation of data collection methods (focus groups, semistructured interviews) and the use of two data analysts enhanced the trustworthiness of the research process and the generated data. Informed consent, voluntary

May 2016, Vol. 8, No. 1 AJHPE

withdrawal and potential bias were reduced by an experienced research assistant who is a qualified OT practitioner. Ethical clearance was obtained from the UKZN’s Health and Social Sciences Ethics Committee (HSS/0761/012M).

Results

The two themes outlined in this article relate to the difficulties experienced by students during fieldwork, and to the supervision strategies that were identified as beneficial for learning. Subthemes are further reported within each respective theme.

Difficulties experienced by students during fieldwork

The difficulties that students encountered during fieldwork practice include anxiety about failing because they are mark driven, a need for clear guidelines, difficulty with clinical reasoning and a lack of understanding of the diverse contexts of their patients. Mark driven and anxious about failing Both the clinical site-based and academic supervisors identified the students’ desire to obtain high marks and its impact on their motivation to learn. They noted how a desire to score well inspired the students and how fear of failure increased anxiety. The supervisors also noted that students employed very limited initiative and self-directed learning, which increased their dependence on the supervisors. ‘They lack confidence in approaching their supervisor to ask for help and to discuss things … , they expect the supervisor to organise their day and structure supervision times for them which can help.’ (Clinical supervisor focus group, Participant 6) ‘ I think the initiative to go and find out more is lacking … they need to be told what to do.’ (Academic supervisors, Participant A) Need for clear guidelines The students and clinical supervisors reported a need for clear guidelines or expectations to ensure a successful fieldwork experience. One student discussed it in the following way: ‘They must also like, … tell me from the very beginning what they want … and okay, … such as you not doing the right thing and you gonna fail and all … . So that it gives me a chance to improve.’ (Student focus group, Participant 3) ‘ Set expectations so student know what you want … so they come into their prac sessions more prepared on a daily basis.’ (Clinical supervisor focus group, Participant 2) Difficulty with clinical reasoning The clinical and academic supervisors reported that students had difficulty linking their observation of a patient to their assessment. Moreover, students experienced difficulty when constructing a patient problem list and linking it to appropriate intervention strategies. Students requested generic steps (recipes) relating to therapies and experienced difficulty in planning alter native sessions. They also lacked the skills needed to adapt sessions and failed to demonstrate adequate problem-solving during sessions. ‘They can’t interpret it [assessment] and … their observation skills are lacking.’ (Academic supervisor focus group, Participant D) ‘ There’s no link between the assessment to the problem list to the treatment, you don’t see that follow through.’ (Clinical supervisor focus group, Participant 4)

Research ‘ Want to do the treatment based on … the basic recipes that they’ve been taught … so lack of focus of it, difficulty to see that what’s working, what’s not.’ (Academic supervisor focus group, Participant B) ‘ They struggle with the grading and adapting, they’re afraid to change the activity if it’s not working at that point because they plan this and they gonna stick to that. And they don’t have back up plans.’ (Clinical supervisor focus group, Participant 3) Lack of understanding of the diverse contexts of their patients Students’ lack of understanding of the diverse contexts of their patients’ lived experiences impacted on their ability to provide effective therapy. ‘Most of our students are middle-classed, they don’t have to take the bus and the taxi … how do we then expect them to then problem solve for our clients who live in those circumstance.’ (Academic supervisor focus group, Participant D) ‘ They actually have no idea of the context which the client is … is going back to so it becomes a programme that’s really not of benefit to the client.’ (Clinical supervisor focus group, Participant 2)

Supervision strategies found beneficial

All participants found guidance and mentoring beneficial for understanding links between observations from assessments and patients’ intervention plans. Furthermore, the use of active teaching strategies, such as reflective journals, peer learning and the skills laboratory, enhanced learning. Allowing students freedom and autonomy with intervention sessions and ensuring that there was constructive feedback was the final subtheme. Guidance and mentoring Clinical and academic supervisors found that verbalising and explaining their clinical reasoning process with regard to the problem list was a useful intervention, which seemingly improved students’ professional skills. ‘Explaining what’s going on in my mind out loud, I feel that they then latch onto that and they will engage in conversation, will learn from it.’ (Academic supervisor focus group, Participant A) ‘ Sit with them … drawing up a problem list … they can start seeing where things [patients problem areas] were clustering and plan intervention from there.’ (Academic supervisor focus group, Participant C) ‘ The supervision and the guidance from a supervisor, it’s really helpful cause it also makes you feel … umm … you going somewhere, like you actually doing something right.’ (Student focus group, Participant 8) Active teaching strategies The use of reflective journals with prompts in the form of questions allowed students to reflect on their clinical reasoning and actions during the fieldwork session. This included observations of the scope of the therapist’s treatment plan, the impact of peer learning on their understanding and skills, and the use of skills laboratories to practise skills in a safe environment. ‘Practical sessions, demonstration, watching … watching an experienced therapist handle a situation. Uhm, we’re not confident in the way we do it and we need to know that someone else does it that kind of way.’ (Students focus group, Participant 6)

‘ I also found that getting them to sit in on each other’s sessions have helped them inside building for their own activity … umm … also giving positive or negative feedback to their colleague or their friends.’ (Clinical supervisor focus group, Participant 3) Students’ freedom and autonomy, and providing constructive feedback Students further benefited from flexible planning during sessions and freedom and space to work autonomously. Supervisors found that encou raging students to be flexible in their sessions helped to build their confidence to cope with problems arising in the session. Constructive feedback and evaluation of sessions with students were valued. ‘What I’ve asked my students to do is to do their plan session and then do something that they haven’t tried before … so just try another activity, see how it would work.’ (Clinical supervisor focus group, Participant 5) ‘ I’ve had the confidence to … explore. Let me do my own thing and then she [supervisor] is there to say, “Hey, that didn’t work. This or that worked.” But … you grew in those situations where you were left alone.’ (Student focus group, Participant 9) ‘ You do feel confident … especially once you’ve had feedback on the prac, especially in 4th year. It’s quite nice to hear that you’re doing what’s expected of you, ... it helps if they give constructive feedback.’ (Student focus group, Participant 1)

Discussion

The literature supports the findings that the students are mark driven and that they fear failure. This behaviour often results in the lack of persistence, as noted with the millennium generation, owing to their need to receive constant praise and being unfamiliar with receiving and/or coping with negative feedback.[8] These students’ unrealistic perceptions of themselves, continuous access to the internet and technology, and need for high marks, which is not always well correlated with their effort, were also noticed by their supervisors. As a result of these peculiar generational characteristics, several authors suggest that millennium students, although being more technologically savvy, have difficulty communicating in traditional formats. They therefore prefer group learning and peer work, where their interactions demonstrate their drive for high achievement. The students’ strategic approaches to studying, linked especially to tasks involving high stakes, necessitate the need to link every learning activity to clear educational goals and outcomes that are directly relevant to their learning.[8] The difficulty with clinical reasoning can partially be explained by the students’ limited experience as novice therapists, who may lack the ability to integrate theory and practice. Completing a comprehensive assessment, and creating a prioritised problem list and contextualised client treatment regimen require clinical reasoning and clinical decision-making skills. The view that clinical reasoning skills develop over time is supported by the literature. Furthermore, clinical reasoning reportedly improves as students gain experience in the field, which allows them to acquire a repertoire of clinical practice skills to draw on.[9,10] Students thus become more confident as they master the clinical decision-making process.[2,11] Students’ ability to engage with clinical reasoning is further hindered by a lack of understanding of the diverse contexts of their patients. A possible explanation for the restricted world view may be found in Amosun et al.’s[12] observation relating to the predominance of students from middle-class families in health science programmes. In SA this observation is supported

May 2016, Vol. 8, No. 1 AJHPE

Research in that these students enjoy greater access, as they are more likely to have studied at better-resourced secondary schools. Leibowitz et al.[13] suggest that educators need to be aware of their own assumptions and should create safe spaces and opportunities for students to improve their cultural sensitivity and exposure to lifestyles that differ from their own lived experiences. Cultural sensitivity is vital for health professional education, given the diverse nature of the population for whom the graduates would be required to deliver a service.[14] Buchanan and Cloete[4] indicate that students need to be prepared for the experiences they will face during fieldwork to facilitate more effective planning for patient intervention.[15] As supervisors, we need to ensure that students access psychological services for debriefing and counselling for emotional stress that placements could generate. But how can supervisors and educators assist student learning? The findings suggest that clear guidelines and mentoring were found to be beneficial to learning. Students follow rule-based procedural reasoning to guide their actions when implementing an intervention.[10] Sinclair’s[10] matrix for clinical reasoning and Schell and Schell[11] suggest that students straddle the levels from novice to advanced beginner in clinical reasoning upon starting out as therapists. This study found that clinical supervisors can aid students’ learning by verbalising the clinical reasoning processes when treating more complex cases. This practice helps to model more advanced clinical reasoning in context, which illustrates the relevance to the case being discussed. Furthermore, feedback from students and supervisors suggests that reflexive practice and peer learning have been noted to start the process of building students’ confidence with regard to their observational skills. These also enhance their ability to give feedback to their peers. This is supported by the literature, which shows that working with peers creates support, as students often experience the same tensions when trying to apply and transfer knowledge to new settings.[7] In this way students are able to explore their professional reasoning and thoughts and evaluate their own interventions. The findings from this study concur with the social constructivist learning experience in which students observe the supervisor providing intervention and improve their repertoire of practice skills though this modelled behaviour.[16] The supervisor and student would then work on an intervention session together, thereby eliciting cognitive changes through guidance and prompting. Finally, students would try intervention sessions using prior knowledge and constructing new knowledge for their professional practice through integrating the observed learned experiences with their own experiences gained during fieldwork.[16] This would help them to learn to use client-generated cues to guide intervention, the ability to anticipate and recognise client strengths and weaknesses based on the experience of other clients, and be flexible to alter and adapt their intervention plans.[10] This feedback from the students and supervisors has shown that during

fieldwork the latter are better able to facilitate graded learning experiences, which would create a store of professional reasoning and practice experiences for the student to use in future professional practice.

Conclusion

Fieldwork is a necessary and essential practical component of the OT under graduate programme. This study provides insight into the difficulties that students experience when engaging in fieldwork. Students benefited from increased guidance and mentoring and the use of active teaching strategies, which allowed them to accept greater responsibility for their learning. Both academic and clinical supervisors play a vital role in modelling therapy, demonstrating clinical reasoning and providing structured learning experiences where students can be supported to master concepts and be empowered to practise independently. In the context of the current diverse student intake at tertiary institutions, it is necessary that we acknowledge our cultural identities and be reminded of students’ inexperience in understanding the cultural contexts of their clients. It is therefore advisable for educational programmes to allow opportunities for supervised debriefing after fieldwork experiences for learners and practitioners to learn from one another. References 1. Health Professions Council of South Africa (HPCSA). Professional Board for Occupational Therapy, Medical Orthotics and Prosthetics and Arts Therapy. The Minimum Standards for Training of Occupational Therapists. Form123B. HPCSA: Pretoria, 2009. 2. Gray M, Clark M, Penman M, et al. New graduate occupational therapists’ feelings of preparedness for practice in Australia and Aotearoa/New Zealand. Aust Occup Ther J 2012;59(6):445-455. 3. Billet S, Choy S. Learning through work: Emerging perspectives and new challenges. Journal of Workplace Learning 2013;25(4):264-276. [http://dx.doi.org/10.1108/13665621311316447] 4. Buchanan H, Cloete L. Preparing students for the complexities of practice learning. In: Lorenzo T, Duncan M, Buchanan H, Alsop A, eds. Practice and Service Learning in Occupational Therapy. New York: John Wiley, 2006:73-87. 5. Emslie B. Clinical occupational therapists’ experience of their role as clinical educators during the fieldwork experience of occupational therapy students. MPhil. Cape Town: Stellenbosch University, 2012. http//:scholar. sun.ac.za/bitstream/handle/10019/Emslie_clinical_2012.pdf (accessed 9 October 2015). 6. Snyman MA. Assessment of professional behaviour in occupational therapy education: Investigating assessors’ understanding of constructs and expectations of levels of competence. MPhil. Cape Town: Stellenbosch University, 2012. http://scholar.sun.ac.za/handle/10019.1/20037 (accessed 9 October 2015). 7. Rutter L. ‘Theory’ and ‘Practice’ within HE professional courses – integration of academic knowledge and experiential knowledge. 6th LDHEN symposium, Bournemouth, UK, 2009. http://eprints.bournemouth.ac.uk/10130/2/LDHEN_LR_paper.final.pdf (accessed 16 August 2014). 8. Hills C, Ryan S, Smith DR, Warren-Forward H. The impact of ‘generation Y’ occupational therapy students on practice education. Aust Occup Ther J 2012;59(2):156-163. 9. Carpenter C, Suto M. Qualitative Research for Occupational and Physical Therapists: A Practical Guide. UK: Blackwell Publishing, 2008. 10. Sinclair K. Exploring the facets of clinical reasoning. In: Creek J, Lawson-Porter A, eds. Contemporary Issues in Occupational Therapy: Reasoning and Reflection. 2nd ed. UK: Wiley and Sons, 2007:143-160. 11. Schell BA, Schell J. Clinical and Professional Reasoning in Occupational Therapy. Philadelphia, USA: Lippincott Williams and Wilkins, 2008:1-433. 12. Amosun S, Hartman N, Janse van Rensburg V, Duncan E, Badenhorst E. Processes in widening access to undergraduate allied health sciences education in South Africa. Afr J Health Professions Educ 2012;4(1):34-39. [http://dx.doi.org/10.7196/AJHPE.138] 13. Leibowitz B, Bozalek V, Carollissen R, Nicholls N, Rohleder P, Swartz L. Bringing the social into pedagogy: Unsafe learning in an uncertain world. Teaching in Higher Education 2010;15(2):123-133. 14. Frenk J, Chen L, Bhutta ZA, et al. Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. Lancet 2010;6 (9756):1923-1958. [http://dx.doi.org/10.1016/S01406736(10)61854-5] 15. Rodger S, Fitzgerald C, Davila W, Millar F, Allison H. What makes a quality occupational therapy practice placement? Students and practice educators’ perspectives. Aust Occup Ther J 2011;58:195-202. 16. Schunk DH. Learning Theories: An Educational Perspective. 6th ed. Boston, USA: Pearson, 2012.

Appendix A. Schedule of questions for student semistructured interviews 1.

Describe your experiences of the OT programme during your training. (Probe: Negative or positive experiences?)

Do you think that clinical fieldwork placements are valuable and why? (Probe: Please give examples of positive experiences that contributed to your learning. Did you have any negative experiences?) (What do you feel makes a good supervisor?)

If you had an opportunity to change anything about fieldwork, what would you change?

How do you feel about assessing and treating clients in the various fields of OT? (Probe: What do you think helped or would have helped with this?)

Do you feel prepared for the planning of treatment programmes and sequencing of treatment sessions? What would assist with this?

Do you feel you are able to evaluate the outcome of your treatment intervention? What helped or would have helped you with this?

The purpose of this interview was to explore your perceptions and concerns related to the fieldwork experience. Is there anything you anticipated me asking that I did not? Please feel free to bring up any topic that might have been missed.

May 2016, Vol. 8, No. 1 AJHPE

Research On being agents of change: A qualitative study of elective experiences of medical students at the Faculty of Health Sciences, University of Cape Town, South Africa J Irlam,1 BSc, BSc (Med) Hons, MPhil (Epidemiology); L Pienaar,2 BSc (Physio), MSc (Physio); S Reid,1 BSc (Med), MB ChB, MFamMed, PhD Primary Health Care Directorate, Faculty of Health Sciences, University of Cape Town, South Africa

Health Education Development Unit, Faculty of Health Sciences, University of Cape Town, South Africa

Corresponding author: J Irlam (james.irlam@uct.ac.za)

Background. Student electives during the 5th year of the University of Cape Town (UCT) medical curriculum provide a 4-week work experience in the health system. The reflective reports of past students indicate that the electives may significantly shape their developing identities as health professionals and agents of change. Objective. To better understand how 5th-year medical elective students perceive themselves as agents of change to strengthen the elective programme in the Faculty of Health Sciences, UCT. The hypothesis was that the more choice that students are given over their learning, the greater is their sense of agency. Methods. Thirteen 5th-year student volunteers participated in four focus group discussions soon after completing their electives in district, regional or tertiary health facilities in the South African health system. Thematic analysis of the transcripts was performed independently by two of the authors. Results. Key themes were the importance of providing holistic patient-centred care, becoming a competent health professional, working within the health team and advocating for a better health system. The elective experience helped students to be more confident in their abilities and to better understand how to effect change at a clinical and health system level. Conclusion. This study supported the hypothesis that the more choice students have over their learning, the greater is their sense of agency. The electives are appreciated as opportunities to develop clinical skills and competencies and to better understand the role of future doctors within the health team and health system. The value of the UCT elective programme could be enhanced by greater promotion, funding for rural electives, and post-elective peer-topeer feedback sessions. This study will inform planning for an extended 2016 medical elective programme in the Faculty. Afr J Health Professions Educ 2016;8(1):41-44. DOI:10.7196/AJHPE.2016.v8i1.540

In the 21st century, health science faculties need to be more than institutions of education where health science students develop the knowledge, skills and competencies for becoming health professionals. It is essential that students are equipped to meet global health challenges and that their skills are relevant to the needs of local communities.[1,2] This means working with communities to improve their health and access to health services, especially communities that struggle for health against multiple social and economic barriers.[3] In South Africa (SA), poor rural communities have least access to healthcare.[4] Doctors are considered the ‘attorneys of the poor’, an advocacy role that requires a strong sense of social responsibility in medical education and practice. According to Dharamsi et al.,[5] developing and nurturing a moral sense of responsibility depends largely on the curriculum and the pedagogical approaches adopted by an institution. In striving to entrench these values of social accountability, medical schools should engage with national and regional stakeholders, such as medical educators, health managers, doctors, communities and policy makers.[6] This engagement should inform the educational model, research agenda and health services offered by medical schools in order to align medical education with local health needs. Woollard and Boelen[6] assert that ‘the medical school must provide ample and appropriate learning opportunities for medical students to grasp the complexity of socioeconomic determinants in health. It must explicitly adopt a preferential model of practice that integrates the biomedical aspects of diseases into a holistic approach to health and wellbeing, and it must offer role models to reinforce this approach.’

Developing future medical doctors to be advocates for greater health equity and quality patient care may require health science faculties to re-orientate their educational activities and contexts of learning. Traditional curricula have not kept pace with rising inequities in health and emerging health risks and continue to graduate students who lack the competencies to address these challenges.[7] It is therefore essential that medical students learn and practise in settings that provide authentic community exposure, to develop their capacity for social change.[8,9] We define this capacity in terms of transformative education of students as agents of change towards better patient care and greater health equity. Student electives can contribute to the development of social conscience and the knowledge, skills and attitudes needed for addressing the health needs of diverse communities.[10] Elective students may have opportunities for deeper engagement with community and social structures that allow application of their altruistic ideals.[11] We therefore hypothesised that the more choice that students are given over their learning, the greater is their development of a sense of agency. The Faculty of Health Sciences, University of Cape Town (UCT) aspires to promote quality patient care and social justice within the lead theme of the equitydriven primary healthcare (PHC) approach.[12] The 5th-year medical elective, introduced in its present form into the curriculum in 2006, is an opportunity for students to work for 4 weeks in the health system to develop their clinical skills and better understand the principles of the PHC approach. The vast majority of students choose clinical electives in general medical practices, community health centres or hospitals in the SA public sector. A minority work in the for-profit and not-for-profit private sector and a handful undertake research electives.

May 2016, Vol. 8, No. 1 AJHPE

Research The electives are assessed based on a written reflective report by the student, accompanied by a brief evaluation from the elective supervisor on site. Students’ clinical elective reports over the past 9 years indicate that the electives raise their awareness of the social determinants of health and may significantly shape their developing identities as health professionals with agency. The purpose of this study is to better understand how 5th-year medical elective students perceive themselves as agents of change, to strengthen the elective programme in the Faculty of Health Sciences, UCT.

Methods

Fifth-year medical students in four elective rotations of the class of 2014 (76 students of a class total of 184) were invited to participate in the study within 2 months of completing their electives. Only those who had completed their electives in district (primary-level care), regional or tertiary health facilities in the SA health system were eligible for inclusion. All those who accepted the invitation were included in the study. One of the authors (LP) facilitated four focus group discussions (FGDs) in a private venue in the Faculty of Health Sciences, UCT. Five questions (Table 1) were discussed, which were drafted by JI, based on review of past elective reports. The questions were refined in consultation with colleagues, three of whom teach on the Faculty’s medical programme and two on the allied health sciences programme. The questions were pretested by LP on three students, who were also participants in the study, and were well understood. The FGDs were recorded by LP and transcribed by a professional trans cription agency. JI and LP analysed the transcripts independently without use of a software package and then discussed their analyses to reach consensus on the key themes. Ethical approval for the study was obtained from the Human Research Ethics Committee (HREC #133/2014) of the Faculty. The UCT Department of Student Affairs gave permission for student participation and participants signed a standard consent form.

Table 1. Focus group discussion questions 1.

Please share what you learnt from your elective about being an agent of change as a student doctor. Provide some examples.

Please tell me how you think you can be an agent of change as a future doctor in the SA healthcare setting.

How do you think the elective programme could better prepare you to be an agent of change in the SA healthcare setting?

How do you think the Faculty could better prepare you to be an agent of change in the SA healthcare setting?

What does being an ‘agent of change’ mean to you?

Providing holistic patient-centred care

The PHC approach emphasises the principle of holistic patient-centred care, which includes biopsychosocial assessments of patients, good communication, effective health promotion, and involvement of families and communities in care. Participants reflected on the multiple aspects of this principle and how it strengthens the agency of health professionals in the lives of their patients. The relationships of health professionals with their patients are strength ened by demonstrating concern for the whole person: ‘The [general practitioner] GP was able to actually diagnose and treat the person fully and at the same time he had such a good relationship with his patients because he is their family doctor and he treated their mothers, their grandfathers and their great grandmother … and that connection that you have with your patient is so much better.’ (FGD, 21 May) Speaking the language of one’s patient enhances the patient encounter and promotes their health more effectively: ‘I just like the fact that we get taught … different languages during our studies. I realised how important it is … I don’t think I maximise my exposure to the language to the best of my ability, and I think that one thing I would change personally is to be able to speak the language of the people that I’m going to be treating.’ (FGD, 2 June) Being a good listener demonstrates humility and empathy towards your patients: ‘… just being kind and listening to people and … getting yourself out of this role and off this pedestal that you’re sitting on and just being human and approaching people on that level.’ (FGD, 31 July) Making time to counsel and educate patients about their health is important for early prevention of disease: ‘It made me feel like the simplest things could actually make a difference because I think the way lifestyle conditions have taken over, that our focus shouldn’t be more on treating complications; you should go a few steps back and prevent them from happening in the first place.’ (FGD, 31 July) Respecting your patient and the importance of each consultation makes a real difference in their lives: ‘I think it’s about taking responsibility for what you know, and taking responsibility for what you’re learning and making a difference in patient’s lives … That patient made an effort to see you, you should make an equal effort to treat them well and not just see them as like, “oh number three, number four, number five”. Realise the importance of each patient encounter.’ (FGD, 31 July)

Becoming a competent health professional

Results

Thirteen medical elective students (10 females and 3 males) participated in one of four FGDs with their elective peers, averaging three students per FGD. The following key themes arising from the discussions are described: the importance of providing holistic patient-centred care, becoming a competent health professional, working within the health team and advocating for a better health system.

May 2016, Vol. 8, No. 1 AJHPE

The UCT electives are seen as a valuable opportunity to develop clinical skills and competencies and to address one’s deficiencies. Having extended responsibilities for patient care develops greater self-confidence and deeper insight into what it means to be an agent of change, but it can also erode one’s empathy: ‘As agents of change, you always need to come back to the reason why you made the decision in the first place. And so when I came into med-school, I think I was possibly a lot more understanding and empathic than I am now.’ (FGD, 2 June)

Research An agent of change is someone who is able to identify problems and find feasible solutions: ‘So it’s someone who is able to identify problems, but also to identify realistic solutions and be able to come up with ways in which to implement those solutions; and more than that it’s someone that is able to network.’ (FGD, 4 June) Initiating change requires sharing your ideas and convincing others to participate in what can be a long process: ‘Yes, so that excitement again made me realise that you can’t sit back and watch things happen around you. If you identify a problem, take it upon yourself to either speak to other people who can do something about it, or start the process yourself. Initiate change because that’s why you studied medicine, to help people and you have the ability to start the process and get everybody involved.’ (FGD, 31 July) ‘ … having perseverance and dedication to whatever goals you have, as change usually takes a long time; … to make the changes you need to convince the rest of the team; … to almost prove it, sell your idea, why this needs to happen.’ (FGD, 4 June) Having greater clinical responsibility is very motivating for most students and develops a deeper appreciation of what they are able to do in the ‘real world’ of medicine: ‘Because I have been in a more practical setting where I was kind of given responsibilities, it makes me appreciate more what I learn now.’ (FGD, 31 July) ‘ I feel like studying for 6 years, we kind of lose the reason we signed up for medicine, like it’s just, it’s not really what we thought it was, and then getting this four weeks was a little taste of what life will be like, and that just gives you that light push until the end.’ (FGD, 31 July) The elective is a time for reflection on the medical profession and one’s personal learning experiences and objectives: ‘I’m starting to think more consciously about my profession, like medicine is so broad, there’s so much to know, there’s so much to learn.’ (FGD, 31 July) ‘ I only have a year and a half left of purely academic undergraduate time, and I need to make sure that I use it wisely. I want to be able to make sure that I know what I am doing, and I can manage, especially in emergency situations, until someone more senior can get there.’ (FGD, 4 June) ‘ It’s just finding that niche and I think the elective helped us realise a little bit more what path we will be going on, and knowing where we will be going.’ (FGD, 31 July)

Working in the health team

After years of being junior members of the health teams in large training hospitals, the elective experience of being respected as equals within the health team is very empowering. Respect for each other’s roles builds the confidence of the team, as observed by one student in a rural district hospital: ‘I also think that the sisters there had confidence in their position, they wouldn’t feel like the doctors are looking down on them, or feeling like

they are superior; and with everybody understanding that everyone has a different role, you are all contributing very much to the team.’ (FGD, 4 June) A well-functioning health team is essential to improving healthcare and motivating health workers: ‘It’s helpful for the patient if I work well with the team, with the nurses … if you have a good relationship, then our patients tend to get better care.’ (FGD, 4 June) ‘ And the way that they used the multidisciplinary team in psych it’s so interesting ... everyone was just discussing how we can make his life better, you know. So if you work like that, eventually everyone will be motivated.’ (FGD, 2 June) Rural electives expose students to another important principle of the PHC approach, which is collaboration between the hospital-based health team and its community partners. The partnership with a non-profit organisation called Mentor Mothers to improve neonatal care in a rural district was cited as a good example: ‘The mentor mother will come to the mother’s home, in the middle of the rural community and speak to her, and follow her up, and refer her back to the hospital if there are any problems that the doctors can sort out, or refer them to [occupational therapy] OT if that is needed. And so it’s doing things like that that really require lateral thinking, that really can make a big difference, and it’s not an extra burden on the hospital at all; in fact it’s really helpful because it means you’re not admitting 6-week-old babies that are already malnourished.’ (FGD, 4 June)

Advocating for a better health system

Several students felt that their elective experiences helped them to realise the important social determinants of health, to see what may be needed to improve healthcare in the local setting, and to understand how they can advocate for a better health system. Careful observation to identify deficiencies in the healthcare setting is seen as an essential first step: ‘… seeing things that hadn’t yet been changed, or seeing difficulties to implement change, I think part of being a student then was bringing up like why … just asking why this is not yet done, and having discussions with the doctors there, which was then finding out difficulties that they’re facing.’ (FGD, 4 June) ‘ … it’s so easy for us to judge what’s right or wrong, but the truth is we’re a new set of eyes and we should be the driving force.’ (FGD, 2 June) Although the shortage of resources is a common complaint, minimising wastage should be everyone’s responsibility: ‘So I think if you just stop wasting some of our resources and making sure that like you know whatever is done with this patient, because we’re so resource limited, it’s actually needed for that patient’. (FGD, 2 June) One student saw the maldistribution of specialists and the poor living conditions of doctors in many rural areas as important challenges for the SA health system: ‘We need to evaluate how we distribute our specialists, but more than that, we need to look at the living conditions of doctors in rural areas,

May 2016, Vol. 8, No. 1 AJHPE

Research because it’s natural for a doctor not to want to go and live somewhere where he can’t send his or her kids to school or where there isn’t a proper house, or there isn’t running water, and it is unfair to send doctors to places like that, if they don’t want to go. So we really need to look at why certain provinces are better than others, and we need to look at fixing that.’ (FGD, 4 June) The challenges of the health system have implications for the type of doctors that should be trained and for where they are trained: ‘The doctors that we need … are actually okay to go and work in a rural area where you don’t have proper referral systems, and the [antiretrovirals] ARVs are out of stock occasionally, and where you really have to make a plan.’ (FGD, 4 June) Working to change the system can seem overwhelming; some students would rather focus on doing what they can at the individual level: ‘I think it’s absolutely overwhelming, and if you’re that person where you are motivated to make change on a big level, then you have to step out of the hospital to do it. But I think for me that just loves patient contact, I’m going to do what I can for this patient, just this one particular patient. So mine is a personal agent of change.' (FGD, 2 June) Working together makes one a more effective agent of change at a health systems level: ‘One person can’t change the whole system nationally; we need to all work together and … fill in the gaps that you can.’ (FGD, 4 June)

Discussion

In this study, we sought to understand medical elective students’ perceptions of themselves as agents of change during their 5th-year electives. We found that the electives are generally perceived as good opportunities to practise holistic patient-centred care, to develop clinical skills and competencies, to work within the health team, to gain first-hand awareness of the determinants of health, and to better understand the role of future doctors within the health system. Our findings support the hypothesis that the more choice students have over their learning within the curriculum, the more opportunity they have to develop agency or self-efficacy.[13] This may contribute to a stronger internal locus of control that may result in greater agency and impact as graduate doctors.[14] These findings are consistent with the findings of Dharamsi et al.[15] that students became more aware and engaged with the needs of vulnerable communities during international service-learning programmes. Smith and Weaver[16] found that students became more aware of primary care and public health issues while undertaking their electives. Our study participants recommended that the Faculty promote the electives more (an extended pre-elective session on the range of elective opportunities was suggested), restore the funding for rural electives (discontinued due to budget constraints), and introduce post-elective peer-to-peer feedback sessions. Post-elective workshops with faculty staff

May 2016, Vol. 8, No. 1 AJHPE

and student peers have enhanced the educational value of student electives at Newcastle University, UK, and would be feasible within the Faculty.[17] Offering training opportunities beyond relatively well-resourced teaching hospitals and health centres was seen as essential for ensuring that UCT medical graduates are equipped to practise in an under-resourced health system such as that in SA. A limitation of this study is the small group of volunteer participants, whose perceptions about the electives may be more positive than those of their non-participating peers. Nevertheless, our findings on the value of the electives are consistent with the vast majority of elective reports assessed by JI since the inception of the elective programme in 2006. We conclude that the UCT medical elective programme makes a valuable contribution to the development of agency among students. This study provides timely and relevant evidence for an extended 2016 medical elective programme at UCT and indicates the value of health science electives in other faculties. Declaration of funding. We received support from the UCT Internal Research Award. Acknowledgements. We thank our colleagues, C S Naidu, H Talberg, M Set shedi and S Amosun, for their assistance in developing the protocol, the 5thyear UCT students for their participation in the focus group discussions, Ms Elloise Kennell for administrative support, and On Time Transcribers for the FGD transcriptions. References 1. McMenamin R, McGrath M, Cantillon P, MacFarlane A. Training socially responsive health care graduates: Is service learning an effective educational approach? Med Teach 2014;36(4):291-307. [http://dx.doi.org/10.3109/ 0142159X.2013.873118] 2. World Health Organization (WHO). WHO transformative scale up of health professional education. An effort to increase the numbers of health professionals and to strengthen their impact on population health. Geneva: World Health Organization, 2011. http://whqlibdoc.who.int/hq/2011/WHO_HSS_HRH_HEP2011.01_eng.pdf (accessed 16 October 2014). 3. Coria A, McKelvey TG, Charlton P, Woodworth M, Lahey T. The design of a medical school social justice curriculum. Acad Med 2013;88(10):1442-1449. [http://dx.doi.org/10.1097/ACM.0b013e3182a325be] 4. Harris B, Goudge J, Ataguba JE, et al. Inequities in access to health care in South Africa. J Public Health Policy 2011;32(Suppl 1):S102-S123. [http://dx.doi.org/10.1057/jphp.2011.35] 5. Dharamsi S, Ho A, Spadafora SM, Woollard R. The physician as health advocate: Translating the quest for social responsibility into medical education and practice. Acad Med 2011;86(9):1108-1113. [http://dx.doi.org/10.1097/ acm.0b013e318226b43b] 6. Woollard B, Boelen C. Seeking impact of medical schools on health: Meeting the challenges of social accountability. Med Educ 2012;46(1):21-27. [http://dx.doi.org/10.1111/j.1365-2923.2011.04081.x] 7. Frenk J, Chen L, Bhutta Z, et al. Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. Lancet 2010;376(9756):1923-1958. [http://dx.doi.org/10.1016/ S0140-6736(10)61854-5] 8. Papadimos TJ, Murray SJ. Foucault’s ‘fearless speech’ and the transformation and mentoring of medical students. Philos Ethics Humanit Med 2008;17(3):12. [http://dx.doi.org/10.1186/1747-5341-3-12] 9. Elam CL, Sauer MJ, Stratton TD, Skelton J, Crocker D, Musick DW. Service learning in the medical curriculum: Developing and evaluating an elective experience. Teach Learn Med 2003;15(3):194-203. [http://dx.doi. org/10.1207/S15328015TLM1503_08] 10. Murray RB, Larkins S, Russell H, Ewen S, Prideaux D. Medical schools as agents of change: Socially accountable medical education. Med J Aust 2012;196(10):653. [http://dx.doi.org/10.5694/mja11.11473] 11. Meili R, Fuller D, Lydiate J. Teaching social accountability by making the links: Qualitative evaluation of student experiences in a service-learning project. Med Teach 2011;33(8):659-666. [http://dx.doi.org/10.3109/014215 9X.2010.530308] 12. UCT Faculty of Health Sciences. Vision, Mission, Values and Goals. http://www.health.uct.ac.za/about/mission/ (accessed 18 February 2014). 13. Bandura A. Self-efficacy: The Exercise of Control. New York: Freeman, 1997:604. 14. Rotter JB. Generalized expectancies for internal versus external control of reinforcement. Psychol Monogr 1966;80(1):1-28. 15. Dharamsi S, Richards M, Louie D, et al. Enhancing medical students’ conceptions of the CanMEDS health advocate role through international service-learning and critical reflection: A phenomenological study. Med Teach 2010;32(12):977-982. [http://dx.doi.org/10.3109/01421590903394579] 16. Smith JK, Weaver DB. Capturing medical students’ idealism. Ann Fam Med 2006;4(Suppl 1):S32-S37;S58-S60. [http://dx.doi.org/10.1370/afm.543] 17. Evans R, Dotchin C, Walker R. Maximising the value from the elective experience: Post-elective workshops. Clin Teach 2013;10(6):362-367. [http://dx.doi.org/10.1111/tct.12033]

Research Mapping undergraduate exit-level assessment in a medical programme: A blueprint for clinical competence? C P L Tan,1 MBBS (Lond), MRCGP (UK), FRCGP (UK), MPhilHSE; S C van Schalkwyk,1 PhD; J Bezuidenhout,1 PhD; F Cilliers,2 MB ChB, Hons BSc (MedSc) (MedBiochem), MPhil (Higher Education), PhD 1

Centre for Health Professions Education, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, South Africa

Educational Development Unit, Faculty of Health Sciences, University of Cape Town, South Africa

Corresponding author: C P L Tan (tanplchristina@gmail.com)

Background. Assessment is an essential component of a medical curriculum. High-stakes exit-level assessment used for licensing and certification purposes needs to be sound. Even though criteria for evaluating assessment practices exist, an analysis of the nature of these practices is first required. Objective. To map current exit-level assessment practices, as described in institutional documentation. Methods. This descriptive interpretive study centred on the document analysis of final-phase study guides of the undergraduate medical programme at Stellenbosch University, Cape Town, South Africa. Results. The key findings were: (i) there is a diversity of methods and approaches to assessment in the final-phase modules; (ii) modules using similar assessment methods applied different credit weightings; (iii) similar assessment methods were described differently across the study guides; and (iv) study guides varied in the amount of information provided about the assessment methods. Conclusion. There is a diverse range of assessment practices at exit level of the MB,ChB programme at Stellenbosch University. This in-depth analysis of assessment methods has highlighted areas where current practice needs to be investigated in greater depth, and where shifts to a more coherent practice should be encouraged. Assessment mapping provides a useful reference for programme co-ordinators and is applicable to other programmes. Afr J Health Professions Educ 2016;8(1):45-49. DOI:10.7196/AJHPE.2016.v8i1.546

Assessment is an essential component of a medical curriculum and is used to measure and manage student progress. Assessment further serves as an indicator of educational efficacy to institutions and teachers.[1] Exit-level assessment is also important for reasons of public accountability and in the interest of patient protection.[1] Medical schools are increasingly being challenged to provide evidence that the assessments used can discriminate between sufficiently and insufficiently competent students.[2,3] Where exitlevel assessments are used for licensing and certification purposes, they are regarded as being ‘high-stakes’ and therefore have significant implications for the student, curriculum, institution and public.[4,5] The assessment of clinical competence is one of the most important tasks facing medical teachers and is used to certify a level of achievement at the end of a programme.[6,7] A range of methods are available to assess clinical competence. These include oral examinations, traditional long and short clinical cases, objective structured clinical examinations (OSCEs), standardised patient-based assessments, and workplace-based assessments such as the mini clinical evaluation exercise (Mini-CEX) and direct observation of procedural skills (DOPS).[4,8,9] To make meaningful decisions about competence, the assessment needs to be sound. Various standpoints have been put forward on how this soundness can be realised. For example, a programmatic approach to assessment has been advocated to achieve fitness for purpose with the assessments used.[10,11] Norcini et al.[1] suggest that validity, reproducibility, equivalence, feasibility and acceptability are essential criteria for good or sound assessment. Multiple methods, preferably in a variety of contexts to capture different aspects of performance,[7] also need to be considered. Given the existence of established criteria to guide sound assessment practices, it would seem reasonable to assume that their application in

medical education programmes is a priority for medical schools that hold themselves publicly accountable to ensure that assessments are seen as credible for all stakeholders. However, there appear to be few studies that have looked at exit-level assessment practices against such criteria.[4,8] An analysis of the assessment practices that are in place is a first step before investigating exit-level assessment against established criteria. There appear to be few studies in this area;[12,13] this study seeks to address the gap. As a starting point, the investigation concentrated on assessment in the final 18-month phase of the Bachelor of Medicine and Bachelor of Surgery (MB,ChB) programme at Stellenbosch University, Cape Town, South Africa. Currently, no overall map exists of assessments as practised during this period. Creating such a map would help to provide an overall picture of what assessment takes place. A preliminary literature search for ‘mapping’ revealed that this term is often associated with ‘curriculum mapping’, ‘concept mapping’ and ‘mind maps’, which make use of visual or diagrammatic pictures instead of written or verbal descriptions to illustrate the relationships and connections between different components of a curriculum or concepts.[14,15] Applying mapping to assessment practices or activities would appear to be a reasonable step forward. One way of analysing assessment activities is by focusing on how these are described in official faculty documents and student module study guides. The objective of the study was therefore to map current exit-level assessment practices as described in the documentation relevant to the final phase of a medical programme. The research question was: ‘What can be learned about the assessment of clinical competence at exit level of an MB,ChB programme from an analysis of how this is described in student study guides provided for each of the modules in the final phase?’

May 2016, Vol. 8, No. 1 AJHPE

Research Methods

This descriptive interpretive study centred on a process of document analysis of the 2012/2013 study guides. In the final 18-month phase (which runs from July of one year to November of the following year) of the 6-year MB,ChB program me at Stellenbosch University, students rotate through 11 clinical modules varying in length from 3 to 7 weeks. Ten of the modules represent one clinical discipline each, and the remaining module, Health, Disease and Disability in the Community, is shared by the divisions of Family Medicine and Community Medicine, and the Centre for Care and Rehabilitation (Table 1). Detailed information relating to each module, including teaching schedules, duty rosters, projects and assignments, assessment methods and resource materials, is made available in study guides that are provided to all students and relevant faculty. In each module, three components contribute to the students’ final overall mark: in-module and end-of-module assessments and final module examinations are conducted in either April or November of the final year. The analysis of the 11 final-phase module study guides was undertaken in two stages. In the first stage, any available information pertaining to assessment conducted during the module (either in-module or end-of-module) and in the final examinations was gathered from the study guides. This collection included varying combinations of information with regard to the assessment schedules, written descriptions of methods of assessment, assessment checklists and marking grids, logbooks, proportion of Table 1. Modules in the final phase of the MB,ChB programme at Stellenbosch University Anaesthesiology Health, Disease and Disability in the Community Internal Medicine Obstetrics and Gynaecology Ophthalmology Orthopaedic Surgery Otorhinolaryngology and Head and Neck Surgery Paediatrics and Child Health Psychiatry Surgery Urology Total: 11 modules

marks allocated for each assessment method and weighting (relating to the calculation of students’ final overall mark for that module). As the study guides were written in English and Afrikaans, the information provided in both language versions was compared to check whether it was the same (by investigator JB, who is fluent in both languages). The information was collated on an Excel spreadsheet and categorised by modules and assessment methods to generate an overview of exit-level assessment in the programme and facilitate comparison between the modules. As this process proceeded, it became clear that there were some gaps in information in the study guides. In the second stage of data collection, all 11 module chairs (faculty who were in charge of organising and co-ordinating the individual modules) were invited to participate in clarificatory interviews to verify and add to the correctness of assessment-related information in the study guides. The module chairs were invited by letter and email, with a follow-up email being sent to non-responders 4 weeks after the initial invitation. At the time of the interview, informed consent was obtained from study participants. An interview schedule was drawn up to serve as a prompt during the interviews. Notes were taken during the interviews, with additional notes recorded afterwards from memory recall. Where necessary, the data on the spreadsheet were amended based on the additional information obtained from these interviews. Ethical approval was obtained from the Stellenbosch University Health Research Ethics Committee (Ethics Committee Reference No. N13/01/009) and institutional permission from the Stellenbosch University Division of Insti tutional Research and Planning to conduct this study.

in the final module examinations. Assessment methods used were grouped together under three main categories, i.e. (i) written; (ii) performancebased; and (iii) other forms of assessment that did not fall under the previous two categories. In drawing up the groupings, it became evident that there was no uniformity in how assessments were described.

Results

Performance-based assessment methods included an assessment of clinical skills in a controlled setting in the form of an OSCE and/or objective structured practical examination (OSPE), which was used in four modules. The number of stations was variable. The OSCE and OSPE used in the final summative examination for Module 4 comprised 16 active stations, each of 7 minutes’ duration, whereas the OSCE for Module 8 had approximately 20 active stations, each of 5 minutes’ duration. ‘Unprepared OSCE questions’ (Study guide 4) that were used as an in-module assessment method in Module 4 were described by Module chair E to be of a written format and were used ‘to test knowledge’. The OSPE in-module assessment for Module 5 was described as including written clinical scenarios, use of videoclips

The information provided in both the English and Afrikaans versions of all 11 final-phase module study guides was confirmed to be the same. Nine of the 11 module chairs consented to participating in interviews, 1 declined and 1 was unavailable. Ultimately, 8 module chairs and 1 module team member were interviewed. Interviews, lasting between 20 and 100 minutes, were conducted over a period of 7 weeks by investigator CPLT. Twenty-one different assessment methods were identified from the study guides. The results are summarised in Tables 2 and 3 to illustrate the differences in methods used during the modules (in-module and end-of-module) and

May 2016, Vol. 8, No. 1 AJHPE

Written assessments

The most common format of written assessments was multiple-choice questions (MCQs), used by six modules. ‘Written’ and ‘slide’ tests used in five modules signified some format of shortanswer questions (SAQs), in which students were required to formulate responses to questions posed, based on a clinical scenario, clinical or laboratory investigations, or a photograph. In several instances, where information extracted from the study guides indicated similar terms being used by different modules, interviews revealed that the nature of the assessment was different. As an example, the slide test in Modules 2 and 9 referred to the projection of a PowerPoint presentation of clinical photographs on a screen while students were writing the test, whereas in Module 7, this referred to a writtenformat assessment which ‘includes clinical material as well as special investigations’ (Study guide 7), with ‘questions based around clinical scenarios’ (Module chair D). ‘Other written’ assessments were used in two modules. These included assignments that students were required to complete during the modules, such as an electronic literature search relating to a patient that the student had cared for during the module, and an evidence-based medicine presentation.

Performance-based assessments

Research

Table 2. Range of assessment methods used during the modules Assessment methods (in-module and end-of-module assessments) Written

Performance based

Contribution to final module mark, %

Module

Duration (weeks)

MCQ

SAQ

OSCE/OSPE

Clinical cases

Diverse clinical

Other

Written test (12.5)

Clinical examination (12.5)

Clinical case discussion (12.5); clinical examination method (12.5)

Continuous (P/F)

Slide test (25)

Clinical case studies (25)

Skills logbook (5); practical ability (40)

Dedication and enthusiasm (5)

General oral and simulated clinical oral (50)

(15)

OSCE (10)

Ward mark (25)

(10)

OSPE (20)

Portfolio (20)

Attitude (satisfactory/ unsatisfactory)

(15)

Clinical (17.5)

Continual (17.5)

Electronic literature search (5)

Clinical long case (40)

Clinical procedures (completed: Yes/No); X-ray presentation (5)

Professional conduct (satisfactory/ unsatisfactory)

(5)

EBM presentation (5); work rehab task (2.5); physical rehab task (2.5); community project (12.5)

Clinical portfolio (17.5)

Continuous tutor assessment (5)

(25)

Slide test (25)

Skills logbook (P/F)

Dedication and enthusiasm (satisfactory/ unsatisfactory)

(17)

Skills Clinical (20) (in skills lab) (2)

Attitude (1)

Written test (20)

Integrity assessment (5)

Other written

‘OSCE’ (clinical and oral) (25)

MCQ = multiple choice question; SAQ = short answer question; OSCE = objective structured clinical examination; OSPE = objective structured practical examination; EBM = evidence-based medicine; P/F = pass/fail. Figures in parentheses refer to the percentage contribution to the final module mark.

and interactive sessions with standardised patients (Module chair F). The inmodule OSCE in Module 11 was actually a combined oral and clinical case assessment. The use of clinical cases (involving real patients) was employed in five modules, varying from 15 to 30 minutes per case. Module chairs pointed out that the number of cases used in the final examinations varied – from 1 (Module 3) to 2 (Module 7) and 3 (Modules 1 and 6). The number of clin ical cases used in the same module also differed when used for in-rotation assessment (e.g. Modules 6 and 7 used 1 case each) compared with the final examinations (the same Modules 6 and 7 used 3 and 2 cases, respectively). In two other modules (Modules 9 and 11), there appeared to be some overlap between the use of clinical cases and oral assessment in the final examinations, as described by the respective module chairs. A number of ‘diverse clinical’ assessment methods was described in the study guides, comprising skills logbooks, portfolios, assessment of ‘practical ability’ (based broadly on history-taking and examination technique, mastery

of skills prescribed in a logbook, ability to formulate and summarise clinical problems and develop a management plan); clinical examination method (based on specific physical examination techniques in that module); clinical case discussions and X-ray presentations to ward consultants; and oral assessment. This loose grouping was made by investigator CPLT in the initial mapping of all assessment methods extracted from the study guides, as these methods shared a common clinical thread but did not fit into the two previously described groups of performance-based assessment methods.

Other assessments

The remaining category of assessment methods used in 10 of the finalphase modules, primarily as part of in-rotation assessment, is labelled ‘other’. These methods dealt mainly with various aspects of professionalism. In four modules, although this assessment did not appear to carry an actual mark, the student was required to obtain a ‘satisfactory’ judgement.

May 2016, Vol. 8, No. 1 AJHPE

Research

Table 3. Range of assessment methods used for the final module examinations Assessment methods (final module examinations) Written

Performance based

Module

Duration (weeks)

Written examination (12.5)

OSCE and OSPE (50)

Clinical (30)

Clinical (25)

OSCE (50)

OSCE (24)

MCQ

SAQ

(20)

OSCE/OSPE

Slide: written (25)

Clinical cases

Diverse clinical

Clinical examination (37.5) Clinical (17)

General oral and simulated clinical oral (33)

Oral (50)

Clinical oral examination (50)

Contribution to final module mark, %

50 Oral (36)

Clinical, oral and X-ray discussion (50)

60 50

Figures in parentheses refer to the percentage contribution to the final module mark.

Structured marking guidelines to assist the assessment of this component were provided in the study guides for Modules 7 and 8. For the remaining eight modules, module chairs confirmed that there were no guidelines and the allocation of marks was subjective.

Summary of results

Ten of the modules used at least one written and one clinical assessment method during the module, whereas Module 3 relied on one method in the form of an oral assessment (Table 2). On overall review of the final examinations (Table 3), Modules 1, 6 and 7 used a written and clinical assessment method. Three modules (3, 10 and 11) used two clinical assess ment methods. Two modules (4 and 8) used a multiple station OSCE and/ or OSPE format, and two modules (5 and 9) used an oral assessment format alone. Information relating to the final examinations for Modules 6 and 9 was not described in the study guides; this additional information was obtained only at the time of interview. There was no information available regarding the final examination in the Module 2 study guide. The students’ final overall mark for each module was based on two components: the total marks awarded for the rotation (from in-rotation and end-of-rotation assessments) and those from the final examinations. In 10 of 11 modules, the weighting for these two components was equal. In the remaining Module 10, 40% of the final overall mark was derived from the rotation marks and 60% from the final examination marks. As indicated by the figures in parentheses in Tables 2 and 3, the weighting of individual assessment methods varies considerably between modules.

Discussion

Four key findings emerged from this study. Firstly, there was a diversity of assessment methods and approaches in the final-phase modules. Secondly, modules using similar assessment methods applied different weightings. A third finding was that the information provided about similar assessment

May 2016, Vol. 8, No. 1 AJHPE

methods was described differently in the various module study guides. These are not necessarily synonymous with what is described in the literature. And fourthly, study guides varied in the amount and detail of information provided about the assessment methods used in the respective modules.

Range of methods used

The diversity of methods and approaches to assessment across the finalphase modules is similar to that reported in McCrorie and Boursicot’s[12] UK study and by Ingham[13] in Australia. Conversely, a single assessment method was used in several modules. The question is whether the (mix of) methods are utilised in a way that is appropriate to exit-level assessment. Miller’s[16] ‘pyramid’, often used to illustrate the multidimensional complexity of assessing clinical competence, moving upwards from reproduction or factual recall in the lower tiers of the ‘pyramid’ to demonstration and application at the summit, provides a useful framework for responding to this question. The study findings indicate that a substantial proportion of assessment still takes place at the ‘lower’ tiers of the pyramid. This finding raises questions about how this might influence the validity of decisions on the clinical competence of the student. Analysis of how assessment is described in the student study guides does not provide sufficient information to draw final conclusions, and further research is required in this area. Other questions deserving further study include whether the range of methods used is appropriate to the outcomes of the relevant exit-level modules and what the findings reveal about the validity of the opinions offered by external examiners.

Weighting of assessment methods

Modules using similar assessment methods applied different weightings, suggesting that the emphasis placed on the assessment method varied across modules. Possible explanations include resource constraints

Research (such as available assessors and space to conduct assessment), and the opinion of assessment conveners about the perceived merits of the chosen methods. Wass et al.[8] have shown that weighting accorded to items per test or total test time can significantly affect reliability, but this has to be considered carefully with other established criteria for good or sound assessment in a high-stakes context. The reasons behind these decisions were beyond the ambit of this study, and these too warrant further investigation.

Description of assessment methods

The study guides serve primarily as a reference for students and faculty to provide official information relating to each module. There was little uniformity in how assessments were described. The varying use of terms, such as OSCE and OSPE, suggests that faculty in different modules may have a different understanding of similar assessment methods, which could impact on reliability and fairness. The absence of clear descriptions of what individual assessment methods entail could potentially lead to confusion and incorrect assumptions by students. Defining and providing consistent and adequate information in the module study guides and official faculty documents regarding the assessment methods used would reduce any possible misunderstanding. Incorporating this detail into faculty development programmes would also promote consistency in the future practices of assessors. Variable in-rotation assessment practice without any descriptions or guidelines of how the marks are determined was noted in several modules, which could result in subjective interpretation and impact on fairness. These in-rotation assessments dealt mainly with aspects of professionalism. The assessment of professionalism is equally complex and requires a multidimensional approach. While itemised checklists and rating scales may not necessarily be the best solution, the introduction of some form of global overall rating could be considered as an alternative and go some way to addressing the difficulties of assessing aspects of behaviour or professionalism during placements.[7] Ultimately, whether quantitative and qualitative measures are used, their utilisation in a defensible manner is key to making valid inferences.

Level of detail provided

Study guides varied in the number of assessment methods used in the respective modules and amount of detailed information provided. There were instances where there was no information regarding the final examination or the assessment methods used. Study guides have the potential to help students to manage their own learning. One of their many uses as a management tool could be for examination preparation by providing information on the format and arrangements for assessment.[17] Although the broad outlines in the Stellenbosch University study guides are similar, a structured template could be used to provide guidance for uniformity in the writing detail.

The way forward

Overall or central co-ordination of the assessment of the MB,ChB programÂ me could address some of the issues highlighted, such as the uniformity of detail and consistency of description regarding assessment methods in all the final-phase modules. One next step could be to investigate what exitlevel assessment actually takes place at Stellenbosch University, and how this relates to what is described in the final-phase module study guides. This could be further expanded to study the exit-level assessment taking place at other medical schools in a similar context, such as in sub-Saharan Africa. Exploring the reasons around choices of assessment methods, decisions on weighting, and clinical competencies considered appropriate for medical graduates could also be avenues for further research.

Conclusion

This study provides an in-depth analysis of assessment methods across an undergraduate medical programme, highlighting the range and diversity of existing assessment practices at the exit-level phase of the MB,ChB programme at Stellenbosch University. A limitation of the research is that the findings reported are not necessarily generalisable to earlier phases of the MB,ChB programme at the university. In addition, actual assessment practices and content will require separate verification. This study has highlighted potential areas where current practice needs to be investigated in greater depth, and where a shift to a more coherent practice should be encouraged. Assessment mapping provides a useful reference for programme co-ordinators and the tool has applicability for other programmes. References 1. Norcini J, Anderson B, Bollela V, et al. Criteria for good assessment: Consensus statement and recommendations from the Ottawa 2010 Conference. Med Teach 2011;33(3):206-214. [http://dx.doi.org/10.3109/0142159X.2011.551559] 2. Health Professions Council of South Africa. Health Professions Act 56 of 1974. Regulations relating to the registration of students, undergraduate curricula and professional examinations in medicine. Government Gazette 31886, 19 February 2009. 3. General Medical Council. Assessment in Undergraduate Medical Education. Advice Supplementary to Tomorrowâ&#x20AC;&#x2122;s Doctors. London: General Medical Council, 2011. http://www.gmc-uk.org/static/documents/content/ Assessment_in_undergraduate-web.pdf (accessed 20 February 2013). 4. Roberts C, Newble D, Jolly B, Reed M, Hampton K. Assuring the quality of high stakes undergraduate assessments of clinical competence. Med Teach 2006;28(6):535-543. [http://dx.doi.org/10.1080/01421590600711187] 5. Norcini JJ, Lipner RS, Grosso LJ. Assessment in the context of licensure and certification. Teach Learn Med 2013;25(S1):S62-S67. [http://dx.doi.org/10.1080/10401334.2013.842909] 6. Wass V, van der Vleuten C, Shatzer J, Jones R. Assessment of clinical competence. Lancet 2001;357:945-949. 7. Epstein RM. Assessment in medical education. N Engl J Med 2007;356:387-396. 8. Wass V, McGibbon D, van der Vleuten C. Composite undergraduate clinical examinations: How should the components be combined to maximise reliability? Med Educ 2001;35(4):326-330. 9. Wilkinson TJ, Frampton CM. Comprehensive undergraduate medical assessments improve prediction of clinical performance. Med Educ 2004;38(10):1111-1116. [http://dx.doi.org/10.1111/J.1365-2929.2004.01962.X] 10. Dijkstra J, van der Vleuten CPM, Schuwirth LWT. A new framework for designing programmes of assessment. Adv Health Sci Educ Theory Pract 2010;15(3):379-393. [http://dx.doi.org/10.1007/s10459-009-9205-z] 11. Van der Vleuten CPM, Schuwirth LWT, Driessen EW, et al. A model for programmatic assessment fit for purpose. Med Teach 2012;34(3):205-214. [http://dx.doi.org/10.3109/0142159X.2012.652239] 12. McCrorie P, Boursicot KAM. Variations in medical school graduating examinations in the United Kingdom: Are clinical competence standards comparable? Med Teach 2009;31(3):223-229. [http://dx.doi. org/10.1080/01421590802574581] 13. Ingham AI. The great wall of medical school: A comparison of barrier examinations across Australian medical schools. Australian Medical Student Journal 2011;2(2):5-8. 14. Harden RM. AMEE Guide no. 21. Curriculum mapping: A tool for transparent and authentic teaching and learning. Med Teach 2001;23(2):123-137. [http://dx.doi.org/10.1080/01421590120036547] 15. Daley BJ, Torre DM. Concept maps in medical education: An analytical literature review. Med Educ 2010;44(5):440-448. [http://dx.doi.org/10.1111/j.1365-2923.2010.03628.x] 16. Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990;65(9 Suppl):S63-S67. 17. Harden RM, Laidlaw JM, Hesketh EA. AMEE Medical Education Guide No. 16: Study guides â&#x20AC;&#x201C; their use and preparation. Med Teach 1999;21(3):248-265. [http://dx.doi.org/10.1080/01421599979491]

May 2016, Vol. 8, No. 1 AJHPE

Research The Umthombo Youth Development Foundation, South Africa: Lessons towards community involvement in health professional education L M Campbell,1 PhD, MB ChB, MFamMed, MMedSci, MPhil, FRACGP; A J Ross,1 MB ChB, DCh, MMed (Family Medicine), FCFP; R G MacGregor,2 PhD, MSc (Agric), BSc Hons, BSc Agric 1

Department of Family Medicine, College of Health Sciences, School of Nursing and Public Health, University of KwaZulu-Natal, Durban, South Africa

Umthombo Youth Development Foundation, Hillcrest, South Africa

Corresponding author: L M Campbell (campbella@ukzn.ac.za)

Background. Internationally, the development of partnerships between institutions of higher learning and the communities they serve is stressed as a priority. The Umthombo Youth Development Foundation (UYDF) is an educational model developed in rural KwaZulu-Natal, South Africa as a response to the scarcity of medical personnel in hospitals. Community involvement in health professional education has become a key strategy in the model, and review of the model may provide lessons for other educators towards implementing community involvement in health professional education. Objective. To review the UYDF, with emphasis on aspects of community involvement. Methods. This qualitative study used a social accountability theoretical framework. Data were collected using the Appreciative Inquiry method and participants who were involved in the UYDF model were interviewed. Themes arising around community involvement were generated inductively. Results. Community involvement in health professional education grew from a funding requirement and has strengthened over time to become an integral component of the UYDF model. Community involvement occurred mainly at the student selection process, but continued during education and after graduation. Participants suggested means by which community involvement could be strengthened. Conclusion. The UYDF successfully presents a model that facilitates community involvement in health professional education. Lessons learnt could guide other models, and the UYDF model could be strengthened by further research. Afr J Health Professions Educ 2016;8(1):50-55. DOI:10.7196/AJHPE.2016.v8i1.559

In 2010, The Lancet published recommendations of a global independent commission that aimed to establish a 21st-century vision for the education of health professionals.[1] In essence, health professional education must go beyond an emphasis on care for the individual patient and instil the importance of addressing broader social issues (social accountability). This study is based in an isolated, deep-rural area of KwaZulu-Natal (KZN), South Africa (SA) and focuses on the Umthombo Youth Development Foundation (UYDF) – a model that aims to address issues of social accountability through several strategies, including community involvement in health professional education. The World Health Organization (WHO) defines the social accountability of health professional education institutions as ‘the obligation to direct education, research and service activities towards addressing the priority health concerns of the community, region and/or nation that they have a mandate to serve’.[2] The World Development Report 2004 discussed two routes to link social accountability and education: a ‘long route’ and/or a ‘short route’.[3] In the long route citizens elect their representative politicians, who appoint or influence policy makers. The latter, in turn, form policies, taking into account the needs and preferences of citizens. The short route involves building a direct relationship between local clients (such as community members), healthcare services and health professional education institutions. Any consideration of a short route for social accountability is particularly pertinent for an SA rural context, as the building of relationships between a local community and healthcare providers was prioritised in the CommunityOrientated Primary Care (COPC) approach initiated by Sydney and Emily Kark in 1942. The Karks founded a healthcare centre in Pholela, an impover

May 2016, Vol. 8, No. 1 AJHPE

ished Zulu tribal reserve in what was then the eastern province of Natal; the two doctors expanded their medical work to include improving housing, increasing access to food, and seeking the views of community members in healthcare initiatives. Their innovative approach inspired other projects around the world. The literature notes that although community involvement and social accountability were not described as explicit goals of COPC, community involvement in healthcare is implicit in the shared understanding of social, physical and economic causes of health problems and in the design of COPC interventions.[4] Building on ideas of COPC and an implicit need for community involvement, the AIDER (assess, inquire, deliver, educate and respond) model of medical education overtly considers social accountability through community partnerships.[5] Unlike the original COPC approach, the AIDER model proposes a continuous monitoring process that explicitly incorporates education and collaboration with underserved stakeholders. Social accountability in health professional education would thus appear to require a tridirectional process of engagement between communities, healthcare providers and institutions of higher learning (IHL). There is a belief that such partnerships will create the connectedness required to accelerate advances in patient care, health professional education and research.[6] In theory, by allowing community members to directly engage with health professional training, the community can take some form of ownership of education and see the results of their efforts. A short route can strengthen what the World Bank calls ‘client power’.[3] In rural SA, taking cognizance of the literature and recommendations around the importance of community involvement in health professional service and education, the UYDF model was initiated in 1999 and

Research has developed over time as a short route, as reflected within a COPC approach. The explicit intent of the UYDF model is to consider the responsibilities of health professional education beyond a narrow focus of individual patient care, and emphasis is placed on social account ability, including community involvement.[7] In the UYDF model, the community are involved at several levels: Representatives of the community (including tribal authorities) and members of a hospital board are directly involved in decisions with regard to selection. Local patients and hospital staff have the opportunity to longitudinally interact with the students as the latter return to the hospital during vacations. Students are required to carry out a community diagnosis and intervention as part of an undergraduate ‘selectives’ module. Students return to work in the community after graduation. A review of the UYDF model is available in the literature[7] and represented diagrammatically in Fig. 1. There is an increasing amount of literature on how to measure social accountability in health professional education. There has been a call to standardise such measurements so that comparisons can be made and progress measured.[8] The WHO have adapted the AIDER model and formatted a group known as The Education for Health Equity Network (THEnet), which has developed a framework to evaluate social accountability in health professional education.[8] The framework considers many factors, including community involvement in the form of partnerships between the community, healthcare providers and health professional education institutions.[8] Although there is much literature on the need for local community involvement in health professional education, there is correspondingly less literature on how to practically ensure local community involvement. Taking cognizance of THEnet framework, this study considered practical questions with regard to the UYDF model: (i) Why was community involvement initiated?; (ii) How is community involvement achieved?; and (iii) How could community involvement be strengthened?

Study methods Study design

The study was cross-sectional, descriptive and employed qualitative methods. The theoretical

The model

School marketing Information about health science careers; grades and subjects needed; university application process; hospital open day

Hospital open days Voluntary work Selection interviews Graduates Graduate support and development

Impact: well-resourced hospitals with local staff offering comprehensive healthcare services to rural communities

Local hospital

University Tuition and academic mentoring Holiday work

Employment

Student mentoring support Academic and social mentoring support Comprehensive financial support

Graduation

Fig. 1. Representation of the strategies involved in the UYDF model.

framework was based on THEnet framework around social accountability and community involvement in health professional training.[8]

Setting

The study was conducted on persons involved in the UYDF model and therefore the setting was mainly in the Mosvold Health subdistrict in Ingwavuma in northern Zululand in rural KZN, where the UYDF model has been developed and implemented since 1999. The community served by the UYDF is diverse and primarily served by Mosvold Hospital, which according to estimates by the Department of Health (2002) comprises about 108 000 people.[9] The population is rural and poor, with adult unemployment at 60%. Only 5% of households have piped water and 3.6% have electricity.[9] Government healthcare is provided by the hospital, 10 residential clinics and three mobile clinic teams. The ravages of apartheid are still obvious in the area, where schooling is generally poor, people are trapped in cycles of poverty and are geographically isolated, and access to an IHL is extremely limited. SA higher education, including health professional education, has undergone considerable transition since the dawn of the democratic era in 1994. In post-apartheid SA, access to higher education is linked intricately to the quest for social equity, and accessibility for marginalised students such as those from Ingwavuma is crucial to the success of any attempts to achieve social justice.

Potential participants

In qualitative research, potential participants are purposely selected as ‘information-rich’ and generally relatively small numbers of participants are studied in great detail.[10] Potential participants were persons who have been involved in the UYDF model and included: • community members • representatives of a local hospital • UYDF founders, managers and mentors • graduates. Several participants from Mosvold Hospital, such as doctors, nurses, occupational therapists and physiotherapists, play the dual role of healthcare providers and educators. They are involved in the day-to-day supervision and teaching of students who attend the hospital during vacations.

Data collection method

Data were collected using a method of appreciate inquiry (AI) – an approach aimed at constructive organisational change.[11] AI explores and builds on the positive features in an organisation and is based on the premise that meaningful and fundamental change occurs through discovering and valuing the strengths, assets, vision and ideals of the individuals in an organisation. AI focuses on what is working well (appreciative) by engaging people in asking questions and sharing their perspectives (inquiry). It has been used to successfully achieve collaborative change

May 2016, Vol. 8, No. 1 AJHPE

Research in communities and organisations, including universities and medical schools.[12] In this study, based on the AI method, parti cipants were interviewed and asked to describe their experiences, involvement and wishes around the UYDF model. The interviews were tape recorded and transcribed to text.

Data analysis

Analysis involved a back-and-forth process search ing for and coding themes, patterns and words; this is described fully in the literature.[10] The process involved immersing in data, i.e. becoming very familiar with the text to the point of knowing where particular quotations occur and getting a feel for the overall meanings and themes.

Scientific rigour

Traditionally, quantitative criteria to ensure methodological scientific rigour include a consideration of validity, generalisability and reliability. However, such criteria are generally not applicable to qualitative work, and this study used different methods to ensure that the process adhered to sound scientific research principles. We relied primarily on a concept of trustworthiness, which the literature describes as an important concept, as it allows researchers to describe the virtues of qualitative terms outside of the parameters that are typically applied in quantitative research.[13] Throughout the study, trustworthiness was considered in terms of credibility, transferability, dependability, and confirmability.[13] Credibility considers how congruent the reported study findings are with reality. Methods to ensure credibility included prolonged engagement between participants and researchers; interviewing a wide range of participants; interviews at various sites; and the use of more than one person in data analysis. Transferability relates to generalisability (the extent to which findings can be applied to other contexts), which was ensured by provision of in-depth details of the methods so that the reader can relate the findings to their own positions.[13] Dependability equates to a positivist notion of reliability (another researcher using the same methods would find the same results). However, the study considered the changing nature of community involvement over time in the UYDF and therefore reliability became problematic. Dependability was achieved by fully describing processes, thereby enabling a future researcher to repeat the work, if not necessarily gain the same results. The concept of confirmability is the qualitative investigator’s

comparable concern to objectivity, and steps must be taken to ensure that the findings are the result of the experiences and ideas of the informants, rather than the characteristics and preferences of the researcher. The role of triangulation by using several people for data analysis, the interviewer not being affiliated to the UYDF, and the use of verbatim participants’ quotations are emphasised as a means to reduce any inadvertent investigator bias.

Ethical considerations

Permission to conduct the study was obtained from the Research Ethics Committee at the University of KZN (UKZN). Written permission was obtained from all key role players including the KZN Provincial Department of Health, hospital managers and relevant authorities at UKZN.

Results

Data were collected over 3 months (June - August 2014). Thirteen people chose to participate and each interview lasted on average 1.5 hours. A brief description of the participants is presented in Table 1. Three main themes arose around community involvement: (i) why it was necessary to have community involvement; (ii) how such involve ment was initially secured; and (iii) ways in which community involvement could be expanded. Each of these themes is discussed below and verbatim quotations are provided from participants to illustrate the theme.

Why was community involvement considered necessary? Participants indicated that in the UYDF model, community involvement in health professional

May 2016, Vol. 8, No. 1 AJHPE

education was initially not flagged as a specific objective and involvement grew as an organic, ad hoc process resulting from need. The need was related to the low number of doctors coming to work in the area. Reasons for the scarcity of doctors included changes to Health Professions Council of South Africa (HPCSA) registration requirements that created significant barriers for doctors from countries outside SA to work in KZN. An external potential source of funding for higher education for local students pivoted on a pre requisite of community involvement. ‘It was increasingly harder to get doctors to Mosvold Hospital because the registration requirements changed. We wanted to try and find local students because local scholars are much more likely to come back and work at your hospi tal. I went to Medical Education for South African blacks (MESAB) and they said they would go 50/50: they would pay half for education if the community paid the other half.’ (F) (MESAB was a collaborative effort by the USA and SA to support the training of black South Africans in the health professions in an effort to improve healthcare for the black population of SA. MESAB provided scholarships for black SA students at 26 SA universities and technikons, along with sundry training initiatives in commu nity health clinics.) Community involvement was also required owing to the nature of the UYDF model that involves local tribal authority leaders and hospital personnel to interview and recommend potential students for support. ‘The scheme is based around the local hospital so it is critical that we had buy-in from the local hospital.’ (M)

Table 1. Brief description of participants Description of participant

Participants, n

Pseudonym

UYDF founder: doctor

UYDF manager

UYDF mentor: physiotherapist

Member of hospital board: nurse

Graduate: doctor

GD1, GD2

Graduate: physiotherapist

Graduate who chose to opt out of work-back contract: doctor

GND

Graduate: social worker

GSW1, GSW2, GSW3

Community members whose children graduated from the UYDF

CM1, CM2

Research How was community involvement achieved?

Participants implied that community involvement was secured by approach ing two tiers of the community: (i) local tribal authorities; and (ii) the local hospital board. ‘I went to the Tribal Authority and told them about the problem of staffing and asked – do you think that every single person in Ingwavuma could give us one rand? We know that there are 100 000 people so if we get one rand from every person, we can use that as a fundraising initiative. In January I had got back about R30 000. I then approached potential funders and showed them how serious the community was about supporting their children to access tertiary education.’ (F) ‘We could ask companies to match that money raised by the community.’ (NB) ‘ At the hospital, the workers were also asked to contribute 50 rand to the scholarship. We did that. It was just a drop in the ocean but it was a start as funders matched what we had raised.’ (NB) This is a deeply impoverished community and willingness to contribute financially to the UYDF could be regarded as a reflection of willingness to become involved with initiating and investing in the future of the process. Participants illustrated how partnership between the community, local hospital and IHL was achieved through the UYDF strategy of numerous stakeholders participating in student selection: ‘Students apply to UYDF by getting their application forms from the local hospital. Then they are required to do one week voluntary work at the local hospital. Selection takes place at the local hospital and the selection committee is made up of community representatives, someone from the hospital and someone from UYDF.’ (M) ‘ The committee ask the hospital what is needed that year and for example, if the hospital needs a pharmacist, then the committee are tasked to select students to study pharmacy.’ (NB) UYDF strategies that deepen community involvement include advocacy and information sharing about a career in healthcare, which were carried out at local schools. ‘We are doing school outreach and school learners get informed about the various health science careers, including subjects and grades needed, university application process and deadlines.’ (Me) ‘ The hospital puts on an open day where learners are exposed to the various health sciences. The students do weekly voluntary work so the hospital is involved with the students from the start. Involvement at the hospital continues as students return to the hospital to work during their vacations.’ (Me)

How could community involvement be strengthened?

Participants included two community members whose children had been supported by the UYDF. Their discussions revealed that ways to strengthen community involvement arose spontaneously, as these participants currently act as advocates in their communities – sharing information and encouraging young people to consider education as a health professional. ‘I am grateful for the UYDF. If my daughter did not get the scholarship she may be having a baby by now or sitting at home doing nothing, or

selling vegetables in the market. Because she got the scholarship she is so excited … Now it is my role to encourage young people. It is my task in the community to talk about the UYDF scheme because I know that it is there and I know that young people can get it and go to university.’ (CM1) ‘ I am spreading the gospel now to others because I see how the scholarship relieved me. I was struggling to take my daughter to the university and the UYDF people came and resuscitated me.’ (CM2) ‘ I would let more people know by going out to the schools and tell them about this UYDF.’ (CM1) ‘ I would ask the community to contribute more because I have seen other programmes where people request donations. They give kids papers and the kids request donations for R2 so we could do this same pattern for the UYDF.’ (CM2) ‘ The government will get more involved when they see that we are doing something and not just folding our arms. The government will subsidise if they see that there is something we are already doing and they can come and assist us.’ (CM2) ‘ There are many churches here. We could ask for support from some.’ (CM1) These participants’ stories illustrate their gratitude to the UYDF; they spontaneously began to act as champions for a cause and advocated on behalf of the UYDF in their communities. Perhaps such champions could improve this link between the community and the UYDF if their role became formalised by strategies, including strengthening their involvement with information sharing about the UYDF in the community (e.g. in schools); involvement in fundraising activities; advocating for political support; and asking religious organisations for support and funding.

Discussion

Since 1999 there have been paradigmatic shifts in health professional education, and IHL partnering with local communities is strongly recommen ded. In 2003, the World Federation for Medical Education (WFME) called for a global accreditation of institutions offering medical education, a key component of accreditation being involvement of community stakeholders.[14] Findings from this study illustrate that, although community involvement was not an explicit intention when the UYDF was initiated in 1999, it has become integral to the ongoing growth and success of the model. The underprivileged community became deeply engaged in providing set-up funding for the UYDF and it is encouraging that they realised the value of higher education. That leaders successfully mobilised households to raise funds is heartening, as many discussions around rural areas are presented in a predominately negative way.[15] The study can begin to question assumptions around community apathy and disempowerment and illustrates the success that can be achieved by opening dialogue between the community, healthcare providers and IHL. In SA, disadvantaged rural students’ access to an IHL has historically been understood as a unidirectional process, with IHL unilaterally selecting students. This process may entrench thinking that IHL is the hegemonic domain of knowledge production and students become passive beneficiaries

May 2016, Vol. 8, No. 1 AJHPE

Research

Selection: potential students carry out voluntary work at a local hospital. The hospital and community are involved in selection and selection is responsive to hospital needs Throughout training: students return to work at the hospital during vacations After graduation: graduates work in the hospital

Fig. 2. Involvement of the community in the UYDF model.

of this knowledge, with no requirement of linkage or accountability to their local communities. Results from this study illustrate that the UYDF challenges this process. The UYDF model is seen to act as a catalyst to encourage a tridirectional accountable process between the community, health service provider and IHL. Current community involvement in the UYDF is enacted at three main levels: (i) selection; (ii) during education; and (ii) after graduation. These tiers of involvement are depicted diagrammatically (Fig. 2). Involvement by the community is displayed, as they are integrally involved in deciding which categories of students are required for a particular year (e.g. pharmacists are selected during a specific year in response to an identified hospital need). This community involvement is linked to some degree of social accountability, as graduates have been selected to fit a need when they return to their community on graduation. This process of involvement of the community in selection, as offered in the UYDF model, illustrates a COPC short-route strategy that facilitates the development of ‘client power’ in shaping health professional education. Interestingly, a community member mentioned involving politicians and thus was supporting a more long-route strategy towards community involvement. There may be increasing political will to support a model that links community involvement to health service delivery and health professional education, as community involvement in health is proposed to be an essential component of the proposed National Health Insurance programme.[16] Participants suggested several ways in which community involvement in health professional education, through the conduit of the UYDF, could be enhanced: increasing advocacy at schools; fund-raising; and lobbying politicians. Further research, taking cognizance of participants’ suggestions, may guide the UYDF strategies around strengthening community involvement. The UYDF may also gain lessons from education other than health professional education, e.g. at UKZN the Certificate of Education (CoE) focuses specifically on a link between IHL-student-community partnerships.[17] The pedagogy involved in the CoE continually and actively draws on students’ narratives around their experiences of involvement in community development. The pedagogies are based on participatory methods such as group work, role play, discussion and stimulation. Narratives described in the CoE revealed that students struggle with multiple responsibilities, and conflict within their communities and educational institutes.[17] The UYDF could advocate for IHL to similarly consider the potential value of students’ narratives. Pedagogies to encourage community involvement in service provision and IHL could be strengthened by considering a community of practice (COP). It is formed by people who engage in a process of collective learning in a shared domain of human endeavour. A COP consists of groups of people

May 2016, Vol. 8, No. 1 AJHPE

DURING EDUCATION Students return to the local hospital during vacation

SELECTION Student selection involves the community

DURING EDUCATION Strengthening community involvement is fore-fronted in pedagogies and curriculum (students and community)

The UYDF model An interphase between the community and institute of health professional education

AFTER EDUCATION Graduate works in local community

Fig. 3. A strengthened UYDF model for community involvement in health professional education.

who share a concern and passion for something they do or want to learn about. A COP could creatively involve community members becoming involved in student assessment. Additionally, the UYDF could advocate for the inclusion of community members in actively teaching students about what to expect in the field. Such a strategy has been successfully implemented in the USA.[18] Advocating for the consideration of students’ and community members’ narratives in health professional education may enable both students and communities not to regard an IHL as an ivory tower that looms over and looks down on society. An IHL can become an interactive, responsive knowledge resource that engages with students, healthcare providers and communities. Taking cognizance of these suggestions, a potentially strengthened UYDF model is represented diagrammatically (Fig. 3).

Conclusion

The UYDF model did not initially aim to foster community involvement; however, partnerships between the UYDF, community, healthcare providers and IHL developed as fundamental to success and sustainability. There was evidence that this short route of community involvement enabled the development of client power, as the community came to have direct influence on student selection to an IHL. The processes involved in developing the UYDF model illustrate that rural communities value higher education and can be successfully mobilised to take action. The UYDF model forms a responsive and accountable framework by which healthcare providers and IHL can engage with local communities. Such partnerships are becoming essential for various reasons, including a future global accreditation of the education institution. Participants provided some ideas around how to strengthen the UYDF model, and further research could consider the implementation of such suggestions. The UYDF could advocate for further strengthening of community involvement in health professional education by innovative strategies such as encouraging COP (community/healthcare provider/IHL) and having community members assess students. The challenges faced by students who come from a rural area are unique and different to the challenges faced by others, e.g. the community. Pedagogies to ensure that the voices of students and communities are heard may include narratives in which the latter can learn about the challenges faced by students/doctors in a rural, isolated community and vice versa.

Research References

1. Frenk J, Chen L, Bhutta ZA, et al. Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. Lancet 2010;376 (9756):1923-1958. [http://dx.doi.org/10.1016/S0140-6736(10)61854-5] 2. Boelen C, Heck J. Defining and Measuring Social Accountability of Medical Schools. Geneva: World Health Organization, 2014. http://whqlibdoc.who.int/hq/1995/ WHO_HRH_95.7.pdf (accessed 11 November 2014). 3. World Bank. World Development Report 2004: Making services work for poor people. http://go.worldbank. org/7EE04RBON0 (accessed 10 November 2014). 4. Geiger HJ. Community-oriented primary care: A path to community development. Am J Public Health 2002;92(11):1713-1716. [http://dx.doi.org/10.2105/AJPH.92.11.1713] 5. Sandhu G, Garcha I, Sleeth J, Yeates K, Walker GR. AIDER: A model for social accountability in medical education and practice. Med Teach 2013;35(80):1403-1408. [http://dx.doi.org/10.3109/0142159X.2013.770134] 6. Rock JA, Acuña JM, Lozano JM, et al. Impact of an academic-community partnership in medical education on community health: Evaluation of a novel student-based home visitation program. South Med J 2014;107(4):203211. [http://dx.doi.org/10.1097/SMJ.0000000000000080] 7. Ross AJ. Success of a scholarship scheme for rural students. S Afr Med J 2007;97:1087-1090. 8. Training for Health Equity Network. THEnet’s Social Accountability Evaluation Framework. Version 1. Monograph 1. 1st ed. THEnet, 2011. http://www.thenetcommunity.org/files/articles/The%20Monograph.pdf (accessed 2 March 2014). 9. Williams VH. Analysis of impact of HIV/AIDS on deaths certified at Mosvold Hospital, Ingwavuma, Northern KwaZulu-Natal from 2003 to 2006. Fam Pract 2007:49(5):13-17.

10. Terre Blance M, Durrheim K, Painter D. Research in Practice. Cape Town: University of Cape Town Press, 2006. 11. Cooperrider DL, Barrett F, Srivastva S. Social construction and appreciative inquiry: A journey in organizational theory. In: Hosking D, Dachler P, Gergen K, eds. Management and Organization: Relational Alternatives to Individualism. London: Sage Publications, 1995:157-200. 12. Quaintance J, Arnold L, Thompson G. What students learn about professionalism from faculty stories: An ‘appre ciative inquiry’ approach. Acad Med 2010;85(1):118-123. [http://dx.doi.org/10.1097/ACM.0b013e3181c42acd] 13. Lincoln YS, Guba EG. Naturalistic Inquiry. Newbury Park, CA: Sage Publications, 1995. 14. World Federation for Medical Education (WFME). Basic medical education. WFME global standards for quality improvement. WFME Office, Copenhagen. 2003. http://www.wfme.org (accessed 8 November 2014). 15. Nkambule T, Balfour R, Pillay G, Moletsane R. Rurality and rural education: Discourses underpinning rurality and rural education research in South African postgraduate education research 1994 - 2004. S Afr J Higher Educ 2011;25(2):341-357. 16. Minister of Health. NHI pilot presentation. http://www.doh.gov.za/docs/presentations/2012/nhipilot.pdf (accessed 5 November 2014). 17. Harley A, Rule P. Exploring access as dialogue in an education and development certificate programme. In: Dhunpath R, Vithal R, eds. Alternative Access to Higher Education: Underprepared Students or Underprepared Institutions. Cape Town: Pearson Education, 2012. 18. Westmoreland GR, Counsell SR, Sennour Y, et al. Improving medical student attitudes toward older patients through a ‘council of elders’ and reflective writing experience. J Am Geriatric Soc 2009;57(2):315-320. [http:// dx.doi.org/10.1111/j.1532-5415.2008.02102.x]

May 2016, Vol. 8, No. 1 AJHPE

Research Exploring knowledge, perceptions and attitudes about generic medicines among final-year health science students V Bangalee, BPharm, MPharm; N Bassa, BPharm; J Padavattan, BPharm; A R Soodyal, BPharm; F Nhlambo, BPharm; K Parhalad, BPharm; D Cooppan, BPharm Discipline of Pharmaceutical Sciences, School of Health Sciences, University of KwaZulu-Natal, Durban, South Africa Corresponding author: V Bangalee (bangalee@ukzn.ac.za)

Background. The use of generic medicines to reduce healthcare costs has become a mandated policy in South Africa. An increase in the use of generics can be achieved through improved knowledge, attitudes and perceptions of generic medicine among healthcare professionals. Objective. To explore knowledge, attitudes and perceptions among final-year health science students on generic medication. Methods. A cross-sectional survey was carried out among the final-year audiology, dental therapy, pharmacy, physiotherapy, occupational therapy, optometry, speech-language and sport science students enrolled at the University of KwaZulu-Natal. A questionnaire was used as the study tool, developed using information adapted from literature reviews. Data analysis was completed using Statistical Package for the Social Sciences (SPSS) version 21, and computed using descriptive statistics. Results. Total number of participants was 211, as follows: audiology (n=14), dental therapy (n=15), pharmacy (n=81), physiotherapy (n=41), occupational therapy (n=6), optometry (n=25), speech-language (n=6) and sport science (n=23). A total of 90.0% of students had heard of generic medicines, with 20.9% of them agreeing that generic medicines are less effective than brand-name medicines. Concerning safety, 30.4% believed that brand-name medicines are required to meet higher safety standards than generic medicines. Regarding the need for information on issues pertaining to safety and efficacy of medicines, 53.3% of participants felt that this need was not being met. Conclusion. All groups had knowledge deficits about the safety, quality and efficacy of generic medicines. The dissemination of information about generic medicines may strengthen future knowledge, attitudes and perceptions. Afr J Health Professions Educ 2016;8(1):56-58. DOI:10.7196/AJHPE.2016.v8i1.560

Drugs play a central role in ensuring the successful health of any population, and this holds equally true for South Africa (SA), which is currently grappling with a high burden of disease. Since the abolition of apartheid, the SA healthcare system has undergone drastic reform, with one of the main priorities being to ensure the availability and affordability of quality healthcare for all citizens.[1] In 2001, the SA government amended the Medicines and Related Substances Act 101 of 1965, obligating pharmacists to inform all patients of the benefits of generic medicine substitution.[2] The shift in legislation to increase the use of generic drugs was primarily instituted as a means of reducing healthcare expenditure for both patients and government. In the discussion over the use of generic medicines, prescriber and dispenser perceptions have often been neglected. Several international studies have noted the influence of healthcare professionals on the consumption of generic medicines; however, very little knowledge on the SA perspective has been previously researched.[3,4] Generic medicines are the therapeutic equivalents of proprietary, branded or innovator medicines.[5] They are of the same strength and dosage form, and possess the same safety and efficacy profiles, as the equivalent innovator medicines.[3] The primary difference is that generic medicines are much more cost effective than the innovator medicine. The sustainability of the generic medicine sector is vital, to ensure that its role in increasing accessibility and affordability of essential treatments will extend well into the future. Multiple factors contribute to generic medicine use, among them providerlevel factors such as pharmacist and prescribing-doctor beliefs or practices. Despite the several advantages offered by the use of generic medicines, the perceptions, attitudes and beliefs held by healthcare practitioners have been identified as a potential barrier to the use of generic medicines.[6] Furthermore, once practice habits have been embedded, it may prove difficult

May 2016, Vol. 8, No. 1 AJHPE

to change the prescriber and dispensing practices of healthcare providers, which highlights the importance of correcting misconceptions before the commencement of professional practice. Based on the nature of their future jobs, healthcare professionals will have an effect on their patients' medication choices. It is, therefore, essential to evaluate future practitionersâ&#x20AC;&#x2122; knowledge and perceptions of generic medicines. The findings of this study will help to address any misunderstandings that these students may have about generic medicines at this crucial stage of their professional career, and aid universities in altering curricula to improve student understanding.

Method

Study sample

A cross-sectional survey was conducted among the final-year audiology, dental therapy, pharmacy, physiotherapy, occupational therapy, optometry, speech-language and sport science students enrolled at the University of KwaZulu-Natal (UKZN), Durban, SA. Convenience sampling was adopted to approach potential respondents.

Ethical considerations

Gatekeeper permission and ethical approval for this study were obtained from UKZN Faculty of Health Sciences Ethics Committee (Ref. SHSEC 006/14). Written consent was obtained from the respondents who participated in the study. Any information disclosing respondent identity was excluded from the tool.

Data analysis

Descriptive statistics were completed using SPSS version 21 (IBM, USA).

Research Study tool

A questionnaire was developed to obtain the objectives of the study. The questionnaire was subdivided into four sections and was developed after reviewing the available published literature in the area. The first section consisted of four demographic questions about age, gender, disci pline and work experience. A 5-item Likert-type scale was used to record variations in perceptions, attitudes and knowledge, constituting the remain ing three sections. The questionnaires were personally distributed and data were collected over a 3-week period.

21 - 22 years. Of these, 14 (6.6%) were audiology students, 6 (2.8%) were speech-language, 15 (7.1%) were dental therapy, 6 (2.8%) were occupational therapy, 25 (11.8%) were optometry, 81 (38.4%) were pharmacy, 41 (19.4%) were physiotherapy and 23 (10.9%) were sport science students. Of the respondents, 68.3% had previous work experience in a hospital or community pharmacy. Majorities of 79.4% and 89.5% of students had heard of branded medicines and generic medicines, respectively. With regard to student knowledge on generic medicines (Table 2), an overall majority of students

knew that generic medicines are interchangeable (74.1%), must be in the same dosage form (58.0%) and are the therapeutic equivalents of brand-name medicines (66.3%). Only 42.8% of students were aware that generic medicines are manufactured after patent expiry of originator medicines, with a smaller percentage (19.6%) agreeing that generic medicines produce more side-effects compared with brand-name medicines. For perceptions of quality, safety and efficacy of generic medicines (Table 3), the majority of respondents did not associate generic medicines

Table 2. Knowledge of generic medicines recorded according to the Likert scale*

Results

Table 1 illustrates the demographics and general knowledge about generic medications of the respondents in the study. A total of 211 students (52 males and 155 females) completed the sur vey, with the majority of students (65.0%) aged Table 1. Respondent demographics and general knowledge about generic medications

Survey question/statement

Generic medicines are interchangeable with brand-name/ originator medicines, %

41.6

32.5

19.1

5.3

1.4

A generic medicine must be in the same dosage form (e.g. tablet, capsule) as the brand-name/originator medicine, %

37.2

20.8

23.2

14.5

3.9

Generic medicines are therapeutically equivalent to brand-name/ originator medicines, %

38.0

28.3

21.0

9.3

2.4

Generic medicines are manufactured after the patent expiry of originator/innovator, %

31.4

11.4

32.4

12.9

11.9

4.3

15.3

34.4

22.5

23.4

Demographics and knowledge

Frequency, n

Groups Audiology Speech-language Dental therapy Occupational therapy Optometry Pharmacy Physiotherapy Sport science

Generic medicines produce more side-effects than brand-name medicines, %

14 6 15 6 25 81 41 23

*1 = strongly agree; 2 = agree; 3 = neither disagree nor agree; 4 = disagree; 5 = strongly disagree.

Age (years) 18 - 20 21 - 22 23 - 25 >25

15 133 52 6

Gender Male Female

52 155

Work experience in hospital or community pharmacy/job experience Yes No

123 88

158 53

Have you ever heard of generic medicine? Yes No

Survey question/statement

Generic medicines are of inferior quality to brand-name medicines, %

4.8

17.2

23.9

25.8

28.2

Generic medicines are less effective than brand-name medicines, %

6.3

14.9

26.0

26.4

Generic medicines are less safe than original medicines, %

2.9

11.1

21.3

33.3

31.4

Generic medicines are less expensive than brand-name medicines, %

28.2

34.9

16.7

11.0

9.1

Brand-name medicines are required to meet higher safety standards than generic medicines, %

11.7

19.4

29.6

16.5

22.8

I believe that generic medicines are only meant for the poor, %

11.5

7.7

17.7

16.3

46.4

*1 = strongly agree; 2 = agree; 3 = neither disagree nor agree; 4 = disagree; 5 = strongly disagree.

Have you ever heard of branded medicine? Yes No

Table 3. Perceptions of the quality, safety and efficacy of generic medicines versus brand-name medicines recorded according to the Likert scale*

179 32

Table 4. Perceptions of current university education about generic medicines recorded according to the Likert scale* Survey question/statement

I need more information on the issues pertaining to the safety and efficacy of generic medicines, %

26.2

27.1

17.6

14.8

14.3

I find it easier to recall a medicine’s therapeutic class using generic names rather than brand names, %

17.6

18.6

41.0

12.4

10.5

I have been taught how medicines are subsidised in the Pharmaceutical Benefits Scheme, %

10.0

12.4

25.2

20.0

32.4

*1=strongly agree; 2 = agree; 3 = neither disagree nor agree; 4 = disagree; 5 = strongly disagree.

May 2016, Vol. 8, No. 1 AJHPE

Research with being of inferior quality (22.0%), being less effective (21.2%), or being less safe (14.0%) than brand-name medicines. Overall, 63.1% of students acknowledged that generic medicines are less expensive than branded medicines. Only 31.1% of students believed that brand-name medicines are required to meet higher safety standards and a smaller percentage (19.2%) felt that generic medicines are only meant for the poor. With regard to the perceptions students had of current university education relating to generic medicines (Table 4), 53.1% of respondents felt that they required more information on the safety and efficacy of generic medicines, with 36.2% finding it easier to recall a medicine’s therapeutic class using generic names rather than brand names. A small percentage thought that they had been taught how medicines are subsidised in the Pharmaceutical Benefit Scheme.

Discussion

A major responsibility for all educators of health science students is to teach future practitioners about the cost-effective use of medicines. To increase the uptake of generic medicines, it is vital that health science practitioners, in particular pharmacists, have a sound knowledge of the issues surrounding generic medicines. The current study showed that the majority of students surveyed had heard of branded and generic medicines. A similar finding was observed in a study conducted in Bangladesh, which sought to explore medical and pharmacy students’ knowledge and perceptions about generic medicines.[5] In our study, however, there was still a significant proportion of students who were unfamiliar with the correct definition of a generic medicine, which came through by incorrect responses to questions on knowledge. Despite their lower prices, generic medicines are required to meet the same safety standards as brand-name medicines. A majority of participants agreed that generic medicines cost less than brand-name medicines, which provides some promise for the use of generic medicines by these future healthcare practitioners in curtailing patient medication costs. On perceptions of current university education about generic medicines, a large percentage of students felt that they needed more information on the safety and efficacy of generic medicines, and many students were unfamiliar with the safety standards associated with the regulation of medicines in SA. Similarly, respondents to the study conducted in Bangladesh also displayed concern about the safely and quality of generic medicines.

May 2016, Vol. 8, No. 1 AJHPE

Lack of or incorrect responses to survey questions may reflect nonexposure to generic medicine issues, differences in professional curricula, or a lack of understanding. In general, mean score differences between the groups showed that pharmacy students’ knowledge was better than that of any other group; this probably stems from the limited use and pertinence of medicines in the fields of practice of other students. Future practitioners need to be exposed to these issues early in their education, so that they can be confident in counselling and treating patients whenever applicable.

Limitations

To our knowledge, the objectives of this study have not been previously researched, which limits comparison of findings with available published literature. The use of convenience sampling, together with the relatively small sample size, might affect the generalisability of the study to the larger student population in SA. Furthermore, the current study did not assess the association between curriculum content and perceptions about the use of generic medicines for each of the chosen disciplines.

Conclusion

Although there were some differences in the responses received from the different groups of future practitioners, all groups demonstrated knowledge deficits, especially on specific issues relating to the safety and effectiveness of generic medicines. These areas need to be addressed by educators, to convince students of the value of generic medicines to further strengthen policy initiatives. Acknowledgements. The authors would like to acknowledge Ms Fikile Nkwany ana for her assistance with the statistical analysis. References 1. Department of Health. National Drug Policy for South Africa. Pretoria: Government Printer, 1996. 2. Government Gazette. General regulations in terms of the Medicines and Related Substances Act, 1965, as amended. Vol. 432, No. 22235. Republic of South Africa, 2001. 3. Shrank WH, Stedman M, Ettner SL, et al. Patient, physician, pharmacy, and pharmacy benefit design factors related to generic medication use. J Gen Intern Med 2007;22(9):1298-1304. [http://dx.doi.org/10.1007/s11606007-0284-3] 4. Mott DA, Cline RR. Exploring generic drug use behavior: The role of prescribers and pharmacists in the opportunity for generic drug use and generic substitution. Med Care 2002;40(8):662-674. [http://dx.doi. org/10.1097/00005650-200208000-00006] 5. Siam MKS, Khan A, Khan TM. Medical and pharmacy students’ knowledge and perceptions about generic medicines in Bangladesh. J Pharm Health Serv Res 2013;4(1):57-61. [http://dx.doi.org/10.1111/jphs.12008] 6. Patel A, Gauld R, Norris P, Rades T. Quality of generic medicines in South Africa: Perceptions versus reality – A qualitative study. BMC Health Serv Res 2012;12(1):297. [http://dx.doi.org/10.1186/1472-6963-12-297]

Research Home-based rehabilitation: Physiotherapy student and client perspectives D Parris,1 BSc (Physio), MPhil (HSE); S C van Schalkwyk,2 PhD; D V Ernstzen,1 BSc (Physio), MPhil (Higher Ed) 1

Division of Physiotherapy, Department of Interdisciplinary Health Sciences, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, South Africa

Centre for Health Professions Education, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, South Africa

Corresponding author: D Parris (dparris@sun.ac.za)

Background. Home-based rehabilitation (HBR) in under-resourced areas in a primary healthcare (PHC) context exposes students to the real-life situations of their clients. There is a scarcity of literature on student and client experiences of HBR in the physiotherapy context. Increased knowledge of HBR could result in an enhanced experience for both student and client. This study sought to discover the perceptions of final-year physiotherapy students and their clients relating to their experiences of HBR during a PHC placement in a resource-constrained setting. Objectives. To explore the experiences and perceptions of physiotherapy students and their clients regarding HBR as part of clinical training in resource-constrained settings. To discover the barriers to and facilitators of effective HBR. Methods. An exploratory case study was performed. A qualitative phenomenological research design in the interpretivist paradigm was used. Semi structured interviews were conducted with clients (n=7) living in an under-resourced setting, who had received HBR from physiotherapy students. Paired interviews were conducted with final-year physiotherapy students (n=6) after their HBR placement. Results. Clients appreciated the students’ services; however, data revealed communication barriers and unmet expectations. Students reported strugg ling to adapt to the context, resulting in interventions not being sufficiently client-centred. They voiced a need for language competency and earlier exposure to such contexts. Conclusion. Exposure to real-life situations in under-resourced settings in HBR provides valuable situated and authentic learning opportunities for physiotherapy students. The experience can be useful in preparing graduates to address the needs of the populations they serve during community service. Afr J Health Professions Educ 2016;8(1):59-64. DOI:10.7196/AJHPE.2016.v8i1.561

South Africa (SA)’s health system places the focus on primary healthcare (PHC),[1] which deals with the health needs of a population in the communities where they live.[1] PHC policy incorporates community-based rehabilitation as part of continuity of care and includes interventions in a client’s home.[1] Physiotherapy education should produce graduates who are competent in addressing the health needs of the people within the community context.[2] Graduates need to serve as health advocates and be accountable for making informed decisions to improve healthcare.[3] To enable them to be more effective in this regard, graduates need exposure to the realities of the healthcare system, socioeconomic health determinants, and clients’ real-life situations through PHC. To train students effectively, authentic learning opportunities are necessary in the communities they will ultimately serve, particularly during their compulsory community service year.[2,4] These communities are often situated in under-resourced areas. Experiencing the real-life context of clients is vital for students so that they will ultimately provide effective interventions and develop social responsibility.[2] Students witness the roles of poverty and society in health first-hand during home-based rehabilitation (HBR). Students report feeling overwhelmed when witnessing the realities of life in poor households.[4-6] Factors such as diverse socioeconomic, racial, language and cultural backgrounds may affect healthcare interventions. Developing cultural competency, i.e. the ability to treat people from a culture different to one’s own with respect and as equals, has become critical in physiotherapy training.[7] Culturally competent and effective client-centred communication leads to improved client satisfaction, outcomes and compliance.[8]

Different skills and clinical reasoning processes are required for physio therapeutic rehabilitation in a home context compared with clinic- or hospitalbased interventions. Tasker et al.[9] report that during HBR, clinical reasoning should primarily consider the client and family needs within the home setting. Students need to be able to adjust the goal and process of the intervention to ensure efficacy and relevance to the clients’ context.[7] Listening attentively to clients can increase the students’ understanding of all factors affecting clients’ health, goals and quality of life and thereby improve client satisfaction.[8] Other skills students may develop in this context include: increased insight, coping with complexity, ability to think on one’s feet, assertiveness, building rapport, empowering others, enhanced observation skills, functioning in someone else’s space, consideration of quality of life and function as applicable to the client, knowing when to discontinue treatment, integration of services, and ability to function as a professional with limited resources.[4,9,10] The theory of situated learning, i.e. learning through active participation within a community of practice, underpins the educational experience of HBR.[11] Authentic exposure in a client’s home environment can result in experiential learning, thereby promoting transformative learning.[11] Transformative learning, i.e. learning that changes one’s view of the world, is a desired outcome of health professional education.[12] HBR leads not only to academic learning and personal development, but also to an understanding of social accountability and responsibility.[2,3] However, to gain maximum benefit from the learning opportunities available, students should be prepared effectively before exposure to HBR.[4] There is a scarcity of literature on HBR in the physiotherapy context, specifically with regard to the students’ or clients’ perceptions. This study sought to discover the perceptions of final-year physiotherapy students

May 2016, Vol. 8, No. 1 AJHPE

Research and their clients regarding their experiences of HBR during a PHC clinical training placement in resource-constrained and diverse settings. Increased knowledge of HBR could improve the preparation of students for the placement, resulting in an enhanced experience for both students and clients.

Method

Research design

An exploratory case study was conducted, focusing on the phenomenon of HBR in an educational context. The phenomenological enquiry allowed an understanding of the real-life experiences and feelings of the participants.[13] In-depth interviews generated qualitative data in an interpretivist paradigm, taking the clients’ and students’ percep tions as their reality.[14]

Research context

At Stellenbosch University (SU), Cape Town, SA, final-year physiotherapy students each spend 6 weeks at a community site learning to integrate and apply the principles of PHC and communitybased rehabilitation. Approximately 5 - 10 clients are seen in their homes per week. One of the assessed outcomes of this placement is that the students will be able to effectively evaluate and treat clients in their own homes. Before this placement, the students mostly treated clients in community health centres or hospital settings, which are more structured and better-equipped environments. Students receive 1 hour of supervision per week from their clinical supervisor; therefore, they mostly conduct HBR on their own. They have the assistance of a community member as a chaperone and translator one afternoon per week. The research was conducted at a community placement site in an under-resourced community in the Western Cape, SA. The majority of residents are isiXhosa speaking and live in informal housing with minimal resources.[15] Students provide the only physiotherapy service in the area. The clients receive HBR in their homes, as they are unable to access or afford transport to the nearest physiotherapy department.

Participants

Purposive sampling was used to invite participants who could share their experiences of HBR.[14] Final-year physiotherapy students from SU were invited to participate in the study after they had completed their rotation in the area. Clients who had received treatment in their homes by these students were approached to participate.

Data collection

Single, face-to-face, semistructured interviews with individual clients were conducted in their homes. The student interviews comprised a reflective conversation between pairs of students who had conducted HBR together, as required by SU safety guidelines.[14] The interviews were conducted in the home language (English, Afrikaans or isiXhosa) of each participant to enable more fluent answers and were recorded with their permission. Open-ended questions regarding their perceptions and experiences of HBR were used to stimulate conversation. The participants were asked about their perceptions about the barriers to and facilitators of effective HBR. Obtaining both groups’ perceptions assisted in data triangulation.[16] Students were also asked for their perspectives on the preparation for PHC and HBR. To minimise potential bias, the interviews were conducted by trained research assistants who were not associated with the SU Division of Physiotherapy.

Data analysis

The isiXhosa interviews with clients were independently translated into English and transcribed. To improve rigour, the transcriptions were checked for accuracy by an isiXhosa-speaking assistant. Member checking by clients was not feasible owing to low literacy levels. The researcher – the students’ clinical supervisor – checked the transcriptions of the students’ interviews. The data were subjected to inductive thematic analysis by the researcher.[14] Codes were manually assigned to themes identified in the data and categorised accordingly. Iterative data analysis occurred to increase the dependability of the study.[16] To ensure credibility, member checking of a transcript and data coding by a student assisted in data verifica tion.[16] The researcher acknowledged that her role as an instrument in the study may have affected student data. To minimise this, she made use of peer debriefing with the co-authors at different

points in the study to enhance the confirmability and trustworthiness of the data.[16]

Ethical considerations

Approval for the study was obtained from the SU Health Research Ethics Committee (S13/10/180) and the Western Cape Department of Health (RP032/2014). Signed informed consent was obtained from all participants. Participation was entirely voluntary and did not affect services to clients or influence student assessment. The interviews were conducted after the students’ examination papers had been marked and feedback given in order to minimise any potential bias. Confidentiality was maintained during the process, with no identifying particulars of indivi dual clients or students being kept.

Results

Participants

Seven clients, of a potential 12, were interviewed. Three clients were exclu ded owing to their inability to converse, 1 client was not traceable and 1 had died. All 6 students who had comple ted their HBR placement at the time of the study participated. The students and clients had different socioeconomic, racial, cultural and language backgrounds. The data obtained from the interviews were analysed to identify themes and categories to promote understanding of HBR. The major themes and associated categories emerging from the client and student interviews are presented separately with supporting quotations.

Client perspectives

The major themes of appreciation and clientcentredness emerged from the client interviews. The main theme related to their appreciation of the treatment received, being treated in their home and attention paid to their goals (Table 1). Categories under the theme of clientcentredness (Table 2) related to communication,

Table 1. Clients’ appreciation of students Category

Supporting quotations

Impact of treatment

‘There is a difference since; I now am able to do things independently.’ (CL2) ‘Since they came, I can do so many things that I was not able to do before.’ (CL2)

Treatment at home

‘I appreciate the fact that students come to the house for therapy; it’s expensive to hire a car. Due to the location of my house it’s too difficult for the car to get there.’ (CL1)

Treatment goals

‘They would ask what they could do to help me.’ (CL1) ‘I do want them to give me the exercises as is supposed to be.’ (CL2)

May 2016, Vol. 8, No. 1 AJHPE

Research home programmes and client expectations. Clients perceived communication from students as inadequate. Clarity and understanding were factors deemed to be important in the writtten home exercise programme the students give the client to continue with on their own. There was an expectation that the students would provide medication. Clients also expected to improve after treatment, seeing the attention by students as a source of hope and motivation. Clients wanted to know what to expect from the students, particulary with regard to regular treatment. More frequent interventions were also desired.

Table 2. Client-centredness Category

Supporting quotations

Communication

‘I wondered when they were coming back, or are they going away for good, so there was no communication.’ (CL5) ‘I was surprised that they didn’t come back again. I didn’t know what happened.’ (CL3) ‘How did they know about me?’ (CL3)

Home programme

‘The student that drew the pictures really helped me a lot.’ (CL2) ‘The student made sure to show me until I understood.’ (CL6)

Expectations of physiotherapy

‘[I thought] they would come with tablets or something.’ (CL3) ‘The experience of standing up with them makes me believe I can walk again.’ (CL7) ‘I’m happy to see them because I want to be better.’ (CL3)

Frequency of treatment

‘If people come back, to know when and how many times.’ (CL3) ‘Please come twice a week.’ (CL7)

Student perspectives

Data obtained from student interviews produced three main themes: differences, preparation for HBR, and learning. Differences noted in socioeconomic, cultural and language domains were repeatedly mentioned in the student interviews (Table 3). Although the students expe rienced culture and language as difficult areas for them, clients did not mention these issues. Differences between providing interventions in formal and informal treatment contexts were also frequently mentioned. There was a realisation that the usual physiotherapeutic interventions, which they would provide in a formal setting, were not necessarily appropriate in the home environment. The experience of seeing how clients function in their own environment was regarded as important in understanding the meaning of holistic intervention. They realised that there is more to treatment than merely the physical techniques they employ. The theme of preparation for HBR was divided into the preparation the students required from the Division of Physiotherapy and the advice they felt would be helpful to give to future students (Table 4). The categories were subdivided to facilitate understanding of the data. The students thought that earlier exposure to resourceconstrained settings would be beneficial prepara tion for HBR. They pointed out, however, that a PowerPoint presentation at the beginning of the 4th year was not helpful. All students interviewed desired more language competency in isiXhosa. Handover of clients to new students was suggested to help to prepare them for what to expect and to assist with logistics and strategies for overcoming barriers. Apart from preparation by the Division, the students had plenty of advice to give to future students, particularly with regard to setting

Table 3. Differences noted by students Category

Supporting quotations

Location

‘What could this patient’s home environment actually be like, because sometimes you can’t even imagine.’ (ST1) ‘You have to walk in between some interesting areas to get to your patient’s house.’ (ST6)

Culture

‘Completely different cultural setting, socioeconomic problems are completely different.’ (ST1) ‘It gives you a culture shock; you do not expect what you see.’ (ST6)

Informal compared with formal treatment settings

‘Just because your patient can walk a little bit wobbly on tiles [in a clinic] doesn’t mean they are going to cope at home.’ (ST6) ‘Techniques that we learn that would be good in the clinics, it literally does not work in the community.’ (ST4) ‘The success has almost got to do with more personal things than it has to do with exactly what you’re going to do.’ (ST1) ‘It’s actually more the talking, the social and psychological part that for them is more important than the physical treatment.’ (ST2)

Language

‘There was a serious communication barrier. Getting them to understand that you need them to tell you what they’re struggling with is a thing all on its own.’ (ST6) ‘Even with my translator, it’s difficult understanding them and getting my own point across.’ (ST3)

specific client-centred goals and being adaptable and organised (Table 5). The last major theme emerging from the student data related to their learning (Table 6). Underlying the situated learning experience, various other learning theories were evident in students’ comments. The need to adapt knowledge gained in the classroom and in other settings was sometimes a challenge in HBR. The students valued the learning opportunities afforded by working in pairs. The importance of learning from clients was also noted. Transformative learning experiences occurred from the experience of having to adapt to real-life situations, including in one instance dealing with the death of a client. In summary, clients were grateful for the interventions received. There were, however, concerns regarding communication and unmet

expectations. The students observed many differ ences between clients’ socioeconomic contexts, culture and language, and their own. Differences in interventions in HBR compared with a formal setting were noted. Preparation desired in the curriculum included early exposure to communities and better language competency. Assistance was required with management and strategies for dealing with problems encountered. Preparation for HBR was preferred just before entering the community – not in advance.

Discussion

This article contributes to the understanding of physiotherapy students’ and clients’ experiences of HBR in an educational context. The need for improved client-centred communication was highlighted in both client and student data.

May 2016, Vol. 8, No. 1 AJHPE

Research

Table 4. Preparation for home-based rehabilitation Category

Sub-category

Supporting quotations

Student suggestions for preparation by the Division of Physiotherapy

Early exposure

‘I don’t know if this would be viable at all, but to almost have a job-shadowing of a home visit [in 3rd year].’ (ST1) ‘Having been exposed to it before you’re very able to put the new setting and culture shock at the back of your mind and get on with why you were there.’ (ST3)

Language

‘Being able to speak Xhosa would’ve made a very big difference.’ (ST6) ‘In 1st year, even though you’re learning the words, you don’t realise why [isiXhosa is] so important. Once you see a patient you understand. More exposure to Xhosa in our 3rd year would also be best.’ (ST1)

Advance preparation

‘It’s difficult to prepare someone a hundred percent for something they’ve never seen before. A photo only says so much. The actual area of the house is completely different.’ (ST2) ‘[A lecture early in year] you forget or don’t really take it in because it is so long until then.’ (ST2)

Handover

‘So that they know more or less what they can expect ... they can be better prepared and know more than absolutely nothing.’ (ST6) ‘New students don’t need to figure out all the barriers for themselves from scratch.’ (ST1)

Strategies

‘Strategies to overcome the problem, because you see the problem but you don’t know [what to do].’ (ST1)

Table 5. Preparation for home-based rehabilitation: Advice to future students Category

Sub-category

Supporting quotations

Advice to future students

Professionalism

‘Make yourself comfortable in someone else’s house whether it’s a mansion or a shack. Be respectful of their environment.’ (ST2)

Goal setting

‘Just being goal specific.’ (ST4) ‘Put your patient’s needs first.’ (ST3) ‘Do a really in-depth subjective [evaluation] and get to know them and find out their goals etc.’ (ST5)

Adaptability

‘Don’t be so eager to try to teach patients what you are taught in class perfectly step by step; not be so technique driven in the community because that’s not going to get you anywhere, it doesn’t work.’ (ST3) ‘The plan is never set in stone so don’t forget that it should always have room to be adapted.’ (ST5)

Communication

‘Discuss why we’re only seeing you say once every two weeks.’ (ST1)

Organisational skills

‘Just organising your patients better, making sure when to see who, and making sure you have everything with you that you need.’ (ST6)

Students identified early exposure to underresourced and culturally diverse settings as a prerequisite. Clients’ appreciation of HBR suggests that it is an important aspect of healthcare, providing treatment to clients who may otherwise not be able to access physiotherapeutic rehabilitation. However, clients seemed uncertain of the students’ plans for their rehabilitation, particularly regard

ing the frequency of visits and termination of treatment, and were left wondering if the students would return. This suggests a lack of effective client-centred communication and planning. Stainsby and Bannigan[10] regard the making of decisions on frequency and cessation of interventions as a skill; it appears that students need assistance with this aspect. The need for an in-depth and relevant subjective assessment to

May 2016, Vol. 8, No. 1 AJHPE

allow for appropriate collaborative goal setting with the client cannot be underestimated. Mindful communication with clients and carers, reported by Tasker et al.,[9] is highlighted in the home environment to ensure relevant interventions. The differences in socioeconomic, racial, cultural and language backgrounds of these SU physiotherapy students compared with those of their clients may have decreased the efficacy of HBR. Communication and goal setting were most affected, as evidenced by students’ comments on initial culture shock and the difficulty in client and therapist understanding each other. Development of the students’ cultural competency and communication skills could enable the clients’ understanding of the intervention and allow their desires to be heard. Reflection with peers and supervisors to address these issues should form an integral part of the placement. Although the clients did not comment that the students could not speak their language, this may have contributed to ineffective communication and management. Language is an integral aspect of communi cation, and HBR highlights its significance in healthcare.[2,4,6] The students stated that being proficient in isiXhosa would have helped in HBR to minimise the verbal communication barrier. They reported that an introductory isiXhosa course in their 1st year seemed irrelevant to them at the time. Once they started treating clients they realised the importance of learning the language. Therefore, agreeing with Prose et al.,[8] the possibility of a more timely course, which also promotes cultural competence, should be investigated. We agree with Mbalinda et al.[6] that a translator, who could also aid cultural understanding, can be a valuable communica tion tool. Although an interpreter accompanied the participants in this study on their home visits, there was still a barrier in communication with the client. This suggests that students should be taught how to facilitate better communication with clients through an interpreter or that a lack of language competency was not the only communication barrier. Clients’ lack of knowledge regarding physio therapy may have contributed to communication barriers and unmet expectations, as seen in the common assumption that the students would provide medication. Therefore, client education is another aspect of client-centred communication. Another factor may be the rotation of different students through the placement, which may affect the consistency of treatment and progress

Research

Table 6. Learning facilitated by home-based rehabilitation Category

Supporting quotations

Authentic learning

‘In class you don’t think of those kinds of things – an uneven path or it’s steep.’ (ST1) ‘Think bigger with your treatment.’ (ST3)

Social constructive learning

‘They always talk about tools in our toolkit, things that we’ve learnt and things that we can then apply to a patient, and that was sometimes a challenge.’ (ST1) ‘It’s not outputs and techniques that we learn that would be good in the clinics. It literally does not work in the community. Use your initiative and be creative.’ (ST3) ‘Had to think out of the box a lot more. You learn to adapt – there were many life skills that you develop.’ (ST1)

Collaborative learning

‘We helped each other a lot and discussed situations.’ (ST1) ‘Your patients come up with the most interesting ways to do something.’ (ST6)

Transformative learning

‘So you had to adapt to what the patient had. And it doesn’t necessarily mean that your treatment is then poorer, it just means we had to think out of the box a lot more.’ (ST1) ‘My biggest lesson from those weeks spent in the community – you must remember where your patients are going once they leave you. It’s made me treat my patients more holistically in an acute setting because I know some of the areas they are going back to and that it’s not ideal, you have to aim your treatment that way.’ (ST3) ‘The patient passed away very unexpectedly, we knew the story, we knew she had children; we had been in her home, so it’s very different, like when you get into someone’s living space. We learnt so much out of that.’ (ST1)

towards goals. As noted in a study on education in a community, collaboration with peers in the form of more comprehensive handover is necessary to build on what previous students achieved.[6] Early exposure to the clinical environment as part of an integrated curriculum has been shown to increase student motivation and lead to deeper learning.[3,4] Students suggested that earlier exposure may assist in overcoming the initial ‘culture shock’. Experience of under-resourced environments and seeing clients in their own contexts will also help to situate physiotherapy practice from the beginning of the students’ clinical training. Providing HBR is complex, requiring the integration of many different skills.[4,9,10] The students seemed to feel that they had adequate physiotherapy skills to conduct an intervention, but required greater adaptability in the HBR situation. A level of maturity is required to enable students to cope with the sometimes challenging real-life situations they experience. Therefore, HBR is better suited to the final year of study. However, facilitated exposure at the beginning of the clinical phase could be considered. In this study, students suggested that earlier exposure would assist in preparing them for the HBR context and help to minimise the reported culture shock. Collaborative learning from accompanying final-year students on home visits could be an option. Having a background of the clients’ context may facilitate construction of

more relevant knowledge at all the levels of care to which a student is exposed. Students remarked on the differences between interventions in formal and informal settings. There is a paradigm shift in planning treatment from a purely physiotherapeutic approach to one that considers the client’s context as paramount. This realisation of the need for a more holistic approach was also noted in other studies.[2,9] Grappling with these adjustments leads to constructive and even transformative learning, as the students begin to think beyond the application of learned techniques to solving a client’s problem. Students reported using their experience of the realities clients face to influence interventions in other contexts. Students were silent on the issues of social accountability and responsibility, as well as the need for change in healthcare service delivery and their potential involvement in these areas. Although they were not specifically questioned on these aspects, it was hoped that a realisation of the place of HBR in the context of healthcare would emerge. This omission could indicate that clinical training in this context should specifically address the notion of social accountability. Students expressed a need for assistance in strategies to cope with the physical barriers they experienced in these settings. However, support is needed to assist them to recognise and deal with issues beyond the normal individualised intervention. Adopting a more reflective practice

could, as part of this placement, facilitate trans formative thinking, stimulating students to embrace the bigger picture and view their role as future agents of change.[2,3] In summary, to ensure that students are able to gain full benefit from the exposure to HBR as a learning environment, they need effective and timely preparation just prior to entering the placement and continuous support to cope with the day-to-day challenges. Communication competence and specifically isiXhosa instruction are also needed. Facilitated early expo sure to under-resourced communities should be consi dered. Effective prepara tion and support will assist the students to overcome the challenges of HBR and enhance the experience for clients. This study cannot be generalised, as it focuses on the perspectives of a small sample in a specific setting.[14] The use of research assistants may have limited the depth of probing during interviews; therefore, some comments may not have been explored sufficiently. Translation can result in some meaning being lost during interpretation. Further investigation into the client and student experience in other community settings is required to achieve more in-depth information on the learning possibilities imbedded in the HBR experience. The effects of earlier exposure to underserved areas should be investigated in future to assess how this affects students’ learning and practice. Follow-up of SU graduates to explore whether their HBR experience prepared them effectively for community service should be considered and the findings compared with those in previous similar studies.[2,4]

Conclusion

Exposure to real-life situations in underresourced settings in the form of HBR provides valuable situated and authentic learning opportunities for physiotherapy students. Clientcentredness, cultural competence, communication and adaptability are just some of the skills that students can develop and which will ultimately lead to enhanced client experiences. The HBR experience can be used to prepare graduates to address the needs of the populations they will serve during community service in SA. Acknowledgements. This research has been sup por ted by the President’s Emergency Plan for AIDS Relief (PEPFAR) through Health Resources and Services Administration under the terms of T84HA21652, Stellenbosch University Rural Medical Education Partnership Initiative.

May 2016, Vol. 8, No. 1 AJHPE

Research References 1. Western Cape Government: Health. Healthcare 2030. The road to wellness. Draft. 2013. http:www.westerncape. gov.za/health (accessed 7 May 2014). 2. Mostert-Wentzel K, Frantz J, van Rooijen AJ. A model for community physiotherapy from the perspective of newly graduated physiotherapists as a guide to curriculum revision. Afr J Health Professions Educ 2013;5(1):1925. [http://dx.doi.org/10.7196/AJHPE.203] 3. Boelen C, Woollard B. Social accountability and accreditation: A new frontier for educational institutions. Med Educ 2009;43:887-894. [http://dx.doi.org/10.1111/j.1365- 2923.2009.03413.x] 4. Ramklass S. Physiotherapists in under-resourced South African communities reflect on practice. Health Soc Care Community 2009;17(5):522-529. [http://dx.doi.org/10.1111/j.1365-2524.2009.00869.x] 5. Cameron D. Community-based education in a South African context: Was Socrates right? S Afr Fam Pract 2000;22(2):17-20. 6. Mbalinda S, Plover C, Burnham G, et al. Assessing community perspectives of the community-based education and service model at Makerere University, Uganda: A qualitative evaluation. Int Health Hum Rights 2011;11(Suppl 1):S6. 7. Chang W. Cultural competence of international humanitarian workers. Adult Educ Quart 2007;57(3):187-204. [http://dx.doi.org/10.1l77/O741713606296755]

May 2016, Vol. 8, No. 1 AJHPE

8. Prose N, Diab P, Matthews M. Experiential learning outside the comfort zone: Taking medical students to downtown Durban, South Africa. Afr J Health Professions Educ 2013;5(2):98-99. [http://dx.doi.org/10.7196/AJHPE.256] 9. Tasker D, Loftus S, Higgs, J. Head, heart and hands: Creating mindful dialogues in community-based physiotherapy. N Z J Physiother 2012;40(1):5-12. 10. Stainsby K, Bannigan K. Reviewing work-based learning opportunities in the community for physiotherapy students: An action research study. J Further and Higher Educ 2012;36(4):459-476. [http://dx.doi.org/10.1080/0309877X.2011.643769] 11. Mann K. Theoretical perspectives in medical education: Past experience and future possibilities. Med Educ 2011;45:60-68. [http://dx.doi.org/10.1111/j.1365-2923.2010.03757.x] 12. Van Schalkwyk S, Bezuidenhout J, Burch V, et al. Developing an educational research framework for evaluating rural training of health professionals: A case for innovation. Med Teach 2012;34(12):1064-1069. [http://dx.doi.org/10.3109/0 142159X.2012.719652] 13. Somekh B, Lewin C. Research Methods Social Sciences. London: Sage, 2005. 14. Silverman D, ed. Qualitative Research. London: Sage, 2011. 15. Du Plessis J, Heinecken L, Olivier D. Community needs assessment and asset mapping profile of Kyamandi. 2012. http:// admin.sun.ac.za/ci/Asset%20mapping%20report%20Kayamandi%202012%20Phase%201.pdf (accessed 22 December 2015). 16. Frambach J, van der Vleuten C, Durning S. Quality criteria in qualitative and quantitative research. Acad Med 2013;88(4):552.

Research An exploration into the awareness and perceptions of medical students of the psychosociocultural factors which influence the consultation: Implications for teaching and learning of health professionals M G Matthews,1 MB ChB, DOH, MPH; P N Diab,2 MB ChB, MFamMed School of Clinical Medicine, College of Health Sciences, University of KwaZulu-Natal, Durban, South Africa

School of Nursing and Public Health, College of Health Sciences, University of KwaZulu-Natal, Durban, South Africa

Corresponding author: M G Matthews (matthewsm@ukzn.ac.za)

Background. South African society is undergoing rapid changes, and includes people from different cultures, beliefs and social backgrounds. Research suggests that these contextual influences have an important bearing on how patients present and relate to healthcare providers. Medical students, too, have a life-world based on their own backgrounds and cultures, and may find relating to a patient with a different life-world challenging. Objectives. To explore students’ awareness and perceptions of how psychosociocultural factors in a multicultural society influence the consultation, and to suggest adaptations for teaching. Methods. Focus group discussions were conducted with final-year medical students in the Family Medicine rotation. Some of the students had viewed a video of a consultation with an isiZulu-speaking patient, and completed a self-reflection learning task. Audio recordings were transcribed and analysed thematically. Results. Exposure to patients in the clinical years had made students aware of the challenges of cultural diversity, although they felt under-prepared to deal with this. Students alluded to the influences of their own cultures, of cultural similarities as well as differences, the roles and behaviours of doctors and patients in cross-cultural consultations, the potential knowledge and experience gap that exists across cultures, and an awareness of the need for patient-centredness. Conclusion. Students should be assisted to improve their cultural competence. Recommendations are made for using various methods, including critical incidents and visual learning to provide opportunities for reflexive practice and transformative learning. Educators must be equipped to address learning objectives relating to cultural competence. Afr J Health Professions Educ 2016;8(1):65-68. DOI:10.7196/AJHPE.2016.v8i1.562

South Africa (SA), like many countries in the modern world, is a rapidly changing society that represents individuals from a multitude of different cultures, beliefs and social backgrounds. Research suggests that these contextual influences have a profound effect on how patients present to healthcare providers.[1] Such a healthcare environment is complex for young health professions students, who have a life-world based on their own backgrounds and cultures, and may find relating to a patient with a different life-world challenging. (The concept of a life-world derives from the German term lebenswelt and refers to the individual and social influences on an individual’s life that result in the subjective manner in which the world is viewed through each individual’s eyes.) As educators, we stress the concept of transformative learning and teach students that they should have a patient-centred approach to the consultation, in which they take into account the patient’s ideas, beliefs, concerns and expectations,[2] but provide little context for students to enable them to negotiate problems of this nature. As such, they may find unfamiliar situations personally challenging or difficult to manage in the clinical environment, in which the context is determined largely by the community or patient’s cultural views and behaviours that determine language, thoughts, communication, actions, customs, beliefs and values. The ability to operate effectively within this environment is referred to as cultural competence and is defined by the USA’s Centers for Disease Control and Prevention (CDC) as ‘a set of congruent behaviours, attitudes, and policies that come together in a system, agency, or

among professionals that enables effective work in cross-cultural situations’. SA’s situation is somewhat unique, as most international literature on cultural competence originates from countries in which there are ethnic minority groups from different cultures. A recent study at this medical school showed that at least a third of undergraduates come from cultural and ethnic groups[3] that differ from the dominant groups in the communities they serve. Many different methods have been suggested for sensitising students to cultural differences. The checklist method of stereotyping cultures and providing a list of characteristics that should be expected has been shown not to be very useful, especially in dynamic societies.[4] Another method of making students culturally aware is that of immersion into a host culture.[5,6] This may be effective, for example, for an emigrant learning a foreign language, but has tremendous logistical and safety implications when incorporated into an undergraduate curriculum, particularly with the large numbers necessary to increase throughput of medical students.[7] Cultural tourism has been criticised for the way in which it places societies ‘on show’, with observers viewing the culture from an outsider’s point of view. It has, however, been shown to be extremely useful in raising awareness of cultural issues.[8]

Theory

Transformative learning theory, an adult learning theory, was used as a theoretical framework for the study. The original work in this field involved Mezirow’s[9] notion of a ‘disorientating dilemma’ or life crisis, which resulted

May 2016, Vol. 8, No. 1 AJHPE

Research in perspective transformation. The theory invol ves learners transforming their perspectives by making meaning of their experiences through critically analysing underlying premises and previously held beliefs. It is useful in crosscultural contexts as, in this manner, individuals learn to change their frames of reference as they critically reflect on their assumptions and beliefs and consciously make plans that bring about new ways of defining and negotiating their worlds.[10]

Objectives

The main objective of this article was to explore medical students’ awareness and perceptions of how behavioural, social and cultural factors in SA’s multicultural society influence the consul tation and consequently their practice as future medical practitioners. The article highlights some of the frequently encountered complex cross-cultural situations that SA doctors face and, finally, recommends how teaching and learning can be adapted to address this in the current curriculum. Specific objectives are: (i) to identify and des cribe emergent themes and attempt to understand their implications for the teaching and learning of health professions students; and (ii) to make recommendations for future practice that will promote transformative learning and perspective change.

Methods

Study setting

The study comprised health professions edu cational research using a qualitative approach. It was conducted at the hospital sites where students were doing their rotations, teaching platforms that serve the Nelson R Mandela School of Medicine of the University of KwaZulu-Natal, Durban, SA.

Participants, sampling and data collection methods A group of 40 final-year medical students in their Family Medicine rotation was identified in July 2013. As part of this module, students lived and worked in groups of four at rural district hospitals where they were immersed in a host culture that was sometimes unfamiliar. Prior to their departure, students received a lecture and tutorials on communication skills. This lecture was enhanced to include elements of crosscultural learning (Group 1). A subset of this group was then shown a video that highlighted a clinical scenario in which a culturally sensitive

topic was introduced (Group 2). A further subset was shown the video and given a self-reflection questionnaire to complete (Group 3). Having had these teaching and learning oppor tunities, three groups of students were interviewed in focus group discussions (FGDs) towards the end of their module. These were mixed groups in terms of first language, ethnicity and religion, and there were no exclusion criteria. A broad interview schedule was used to gather information on the students’ experience of culture in the consultation and of cross-cultural teaching and learning. The FGDs were conducted at a pre-arranged time at the hospital sites. They were led by a research assistant from the School of Nursing and Public Health, and attended by the researchers.

Data analysis

Audio recordings of the FGDs were transcribed and analysed using inductive coding and thematic analysis.[11] The students’ responses to the selfreflection questionnaire were also included in the thematic analysis. For trustworthiness, the researchers analysed the transcripts separately, discussing and comparing emergent themes, and selected representative quotations and incidents referred to by the students. Comments were referenced as being extracted from one of the three FGDs, with no specific reference to individual students due to ethical constraints.

Ethics and consent

Ethics clearance for the study was granted by the University Humanities and Social Sciences Research Ethics Committee (HSS/0312/013). Necessary permissions were obtained from gatekeepers, and students gave individual consent. In the interviews, students were assured of anonymity and were not identified by name. They were informed that all contributions were voluntary, that the interviewers would not participate in assessment, and that there would be no negative consequences from participation in the focus groups.

Results

While students had been sensitised to crosscultural issues by the teaching exposures, across the three groups of students there were no differences in responses from those who had been exposed to the video or self-reflection questionnaire. In addition, the students’ rural experience had presented opportunities for learning experiences. Table 1 summarises the various themes and subthemes as the findings of this study.

May 2016, Vol. 8, No. 1 AJHPE

Table 1. Results of study: summary of themes and subthemes Themes

Subthemes

Interactions and relationships

With self With other students With doctors With patients

Awareness of cultural diversity

Similarities

Gaps

Knowledge

Differences Experience

Interactions and relationships

Certain students indicated an awareness of their own cultural influences that had the potential to influence their clinical interactions. Some spoke on matters of relevance in their own religions and cultures, and shared with the group the influence that these may have on their practice of medicine: ‘You have to understand religious views also, you cannot have guys and girls together. Like for us we are not allowed to go out with boys.’ (comment by a Muslim student, FGD3) ‘ ... not someone who is pulling towards the East and you are pulling West.’ (FGD1) Some participants had noted or discussed behaviours in health professionals that they considered inappropriate: ‘They were complaining about the doctors at [...] Hospital. They were telling me that they hope that I do not become a doctor like that and like, we cannot even greet!’ (FGD2) ‘ In the rural areas your stethoscope is way too powerful!’ (FGD3) ‘ Your patients will take for granted that you look the part, I mean you are black.’ (FGD1) Students had been exposed to the concept of patient-centredness, and this was an important theme, with many referring to the importance of incorporating the patient’s perspective: ‘... all that stuff to show that you actually care about the patient as a whole and that actually shows the patient that you do not take them as a disease but as a person.’ (FGD 1) ‘ ... that patient centredness, for each patient it will not work out the same, you have to work out something that is right.’ (FGD3)

Research Awareness of cultural diversity

Certain students mentioned the need for cultural sensitivity in multicultural environments, and went on to note that there were, in fact, some previously unrecognised similarities among cultures: ‘So if you are going to be judgemental and putting your religion and culture there on the table, it will not work because we are all different.’ (FGD3) ‘There are actually a lot more similarities than differences.’(FGD1)

Gaps

Some students described incidents in which behaviours demonstrated insufficient knowledge of cultural practices: ‘... I know my granny would be like, I saw this white girl and she said this ... and there is also that thing that probably she does not understand me. She does not know what ‘ukugcabha’ [the use of scarification to treat symptoms] is, so I cannot explain what that is so let’s just leave it there. She says I should take these pills. I will just take them home and that is it and I will continue what needs to be done.’ (FGD2) ‘ The patient kept saying to the consultant, he was calling her “mama,” not in a bad way, because he is respecting her. And then the consultant got offended and said “No, I am not your mom, I am your doctor!” For me, that was like OK, but the patient is trying to be respectful, not that he is saying you are old or something ... that is how we are taught, especially in blacks ... .’ (FGD1) ‘... we are not sensitised to each other’s cultures at all.’ (FGD1)

Specific anecdotes

Various participants recounted incidents that they found significant, provoking rich discussions in the groups. Some examples are included for illustrative purposes. Interesting discussions around aspects of African culture included content about isintu (traditional rituals); cultural beliefs such as thwasa (calling to be a sangoma or traditional healer); and the use of various types of muthi (traditional medicines), including herbal enemas and a therapeutic intervention called isithlambezo (traditional medicine used to induce labour). Others referred to the abovementioned ukugcaba and the importance of iziphandla (a wrist bracelet of animal skin mainly used in rural communities).[12] Some students spoke of how their lack of awareness of or misperceptions about some or all of these practices made their understanding of the patient’s perspective more difficult (various FGDs). Discussions also arose about other cultures, with the following brief narrative about Muslim culture included for illustrative purposes. This anecdote is an example of how easily cross-cultural misunderstandings may arise: ‘... during an interview seeing this Muslim patient, there was this young lady ... you could only see her eyes and there was a black registrar and she was not looking at him. Eventually he asked, “What is wrong? Why are you not looking at me? Is there something wrong with your eyes maybe?” and she said, “No, this is how we are taught to; we do not look at the men in the eyes when we are speaking to them".' (FGD 1)

Discussion

This objective of this article is to explore the awareness and perceptions of medical students of the psychosociocultural factors that influence the

consultation. It describes emergent themes and mentions incidents in multi cultural environments that were significant to students. In analysing the comments, it was noted that the responses reflected different individual levels of self-awareness and ability to cope with challenges. Some students had considered the influence of their own cultures on their behaviours and interactions with colleagues, other health professionals and patients, and perceived to varying degrees that these differences had an important influence on successful communication. The study showed that many students experienced difficulty when dealing with cross-cultural contexts, and several students recognised knowledge gaps that exacerbated the problem. The impact of introspection was demonstrated by some students who showed greater self-awareness than others and had considered the influence of their personal, cultural and religious views. Some participants realised that other people they encountered, either as colleagues or patients, also had similar innate factors or personal views that affected the consultation. Some had noted negative role-modelling and lack of cultural sensitivity in more senior health professionals they had encountered in clinical rotations. Exposure to patients from various cultures, particularly in the rural setting, had raised awareness of the challenges of cultural diversity, with several participants expressing the opinion that they felt under-prepared to deal with this. Some made reference to insufficient exposure to information about the cultures of others, and expressed a need to engage with issues related to patients’ life-worlds and cultures to improve their own knowledge. Several students narrated incidents that they considered important, and spoke of the difficulties they anticipated for their future practice as healthcare practitioners.

Implications for teaching and learning

To improve teaching and learning, it is imperative that students in the health professions are equipped to deal with cultural diversity in a culturally competent manner. Students need to gain an understanding of one another’s life-worlds. They need to develop appropriate attitudes and have the knowledge and skills to deal with challenges in multicultural professional environments. The roles and responsibilities of the educator in health professions edu cation in fostering transformative learning thus cannot be overemphasised, as learners should be assisted to become aware and critical of long-held assumptions. Perspective change can be achieved through an accumulation of transformations over a period of time, assisting students to redefine problems and improve their ability to respond to their patients’ cultural influences on health.[10] As mentioned above, many methods of teaching cultural competence have been used with varying degrees of success in other contexts. To understand another culture, one has to be self-aware and have a good appreciation of the influence of one’s own culture relative to that of others in practice – a realisation that may only occur after a certain degree of self-reflection and self-examination.[13] By reflecting upon past experiences and narrating experiences to others, rather than adopting a stereotypical approach, learners may begin to understand the complex factors that influence how a patient behaves when confronted with illness.[14] Experiences such as those described by the students in the form of ‘difficult’ consultations or challenging experiences occur throughout a professional career, including in the undergraduate phase, and may be used as critical incidents to promote self-awareness and awareness of the influence of psychosociocultural factors that affect their patients.

May 2016, Vol. 8, No. 1 AJHPE

Research Students in the FGDs were enthusiastic about sharing challenging experiences. We noted that the opportunities for sharing of the students’ narratives and discussions in the focus groups proved to be a fertile ground for new conversations between students themselves and the researchers about the challenges encountered in multicultural populations. Because of this, it is the opinion of the authors that these learning spaces in the rural attachment should be supported to provide students with opportunities for deep reflective practice and transformative learning. We propose that gaps in knowledge and experience could be filled by maximising opportunities in existing structured teaching times in communication teaching and clinical rotations within the context of routine, everyday intercultural encounters, to reflect on their cultural significance and implications in context. They could also be addressed in a structured manner in cultural seminars or when teaching a language. Opportunities exist for students to use engagement in deep reflection by journaling their experiences: how this has changed their insights and perspective, and what they have learnt. This would assist in creating the ‘change agents’ who are socially responsive and relevant to the populations they serve as healthcare professionals.[15] The authors have previously suggested the use of critical incident reflection and of video technology in the teaching and learning of communication to medical students.[14,16] As competition for teaching time already exists in the medical curriculum, it is suggested that further innovative methods be used to make this cultural learning generally available. Thus, the university’s Visual Learning Project, a repository of videos, could be used as a teaching platform for recorded scenarios illustrating appropriately selected cross-cultural consultations (with inputs from subject experts) to be critical incidents for discussion. These would allow students to improve their knowledge and reflect on challenges, while raising cultural awareness and contributing to cultural competence. Additional functionality of the video software allows learners to conduct online discussions, and educators to provide feedback on the content as necessary. While students have long been taught the importance of a ‘patientcentred’ approach (which the study shows to have been assimilated), it is suggested that educators raise awareness of the person in an individual context, thus highlighting the concept of ‘person-centredness’, to assist in developing an empathic approach.[17] This locates the person in a somewhat more complex psychosociocultural context as opposed to a pure illness context, and may help to develop culturally competent healthcare practitioners who are sensitive to their patients’ perspectives in a deeper fashion, and therefore more equipped to respond appropriately. As specific recommendations to promote cultural competence in students, the authors suggest the following: • Specifying the understanding of psychosociocultural influences in the consultation in learning objectives from early in the curriculum to raise awareness. • Developing self-awareness in students by encouraging them to analyse how their own life-worlds affect their views. • Filling knowledge and experience gaps about culturally sensitive problems. • Specifying topics and critical teaching points for educators, with the implications for practice being the value of student involvement in material selection and the use of a task-based approach.

May 2016, Vol. 8, No. 1 AJHPE

• Providing positive role-modelling and increasing the diversity in teaching staff to make it representative of the demographics of the province. • Including cultural competence as a learning outcome and assessing it specifically as a competence in clinical assessments.

Study limitations

The study was conducted using only three FDGs and self-reflection questionnaires. Only final-year students were asked to participate, as the study was done after they had been exposed to the rural attachment of the final year. While this is the case, the results showed strong emergent themes that are generative in nature, and support the necessity for a response in terms of the introduction of innovative methods for the teaching and learning of cultural competence. This study highlighted the importance of developing reflexivity and cultural awareness at undergraduate level. However, the maintenance of this awareness after qualification is beyond the scope of the study. This study also does also not highlight the behavioural change that may/may not result from experience and/or exposure.

Novelty and significance of the work

Although a great deal of research is being done internationally in cultural competence, there is a lack of literature on cultural competence in medical students in the SA context. This article provides new and important insights into local healthcare contexts and can assist in making recommendations for teaching and learning, as well as contributing to the body of knowledge internationally on facilitating multicultural competence. Acknowledgement. The study was funded by a grant from the University of KwaZulu-Natal Teaching and Learning Office. References 1. Napier A, Ancarno C, Butler B, et al. Culture and health. Lancet 2014;384(9954):1607-1639. [http://dx.doi.org/ 10.1016/S0140-6736(14)61603-2] 2. Silverman J, Kurtz S, Draper J, van Dalen J, Platt F. Skills for Communicating with Patients. Oxford, UK: Radcliffe Publishing, 2005. 3. Matthews M. Vocation-specific isiZulu language teaching and learning for medical students at the University of KwaZulu-Natal. Masters dissertation. Durban: University of KwaZulu-Natal, 2013. 4. Seibert P, Stridh-Igo P, Zimmerman C. A checklist to facilitate cultural awareness and sensitivity. J Med Ethics 2002;28(3):143-146. [http://dx.doi.org/10.1136/jme.28.3.143] 5. Anderson K, Friedemann M, Buscher A, Sansoni J, Hodnicki D. Immersion research education: Students as catalysts in international collaboration research. Int Nurs Rev 2012;59(4):502-510. [http://dx.doi.org/10.7416/ ai.2013.1926] 6. Larson K, Ott M, Miles J. International cultural immersion: En vivo reflections in cultural competence. J Cult Divers 2010;17(2):44-50. 7. World Health Organization. Transformative Education for Health Professionals. 2006. http:// whoeducationguidelines.org/content (accessed 1 August 2014). 8. Prose N, Diab P, Matthews M. Experiential learning outside the comfort zone: Taking medical students to downtown Durban. Afr J Health Professions Educ 2013;5(2):98-99. [http://dx.doi.org/10.7196/AJHPE.256] 9. Mezirow J. Transformative Dimensions of Adult Learning. San Francisco, USA: Jossey-Bass, 1991. 10. Taylor EW. Intercultural competency: A transformative learning process. Adult Educ Quarterly 1994;44(3):154-174. 11. Miles M, Huberman M. Qualitative Data: An Expanded Sourcebook. 2nd ed. Thousand Oaks, CA, USA: Sage Publications, 1994. 12. Ellis C. Communicating with the African Patient. Scottsville, SA: UKZN Press, 2004. 13. Helman C. Culture, Health and Illness. London, UK: Butterworth, 2007. 14. Diab P, Naidu T, Gaede B, Prose N. Cross-cultural medical education: Using narratives to reflect on experience. Afr J Health Professions Educ 2013;5(1):42-45. [http://dx.doi.org/10.7196/AJHPE.234] 15. Van Heerden B. Effectively addressing the health needs of South Africa’s population: The role of health professions education in the 21st century. S Afr Med J 2013;103(1):21-22. [http:/dx.doi.org/10.7196/SAMJ.6463] 16. Diab P, Matthews M, Gokool R. Medical students’ views on the use of video technology in the teaching of isiZulu communication, language skills and cultural competence. Afr J Health Professions Educ 2016;8(1):11-14. [http:// dx.doi.org/10.7196/AJHPE.2016.v8i1.402] 17. Slater L. Person‐centredness: A concept analysis. Contemporary Nurse 2006;23(1):135-144. [http://dx.doi. org/10.5172/conu.2006.23.1.135]

Research The way forward with dental student communication at the University of the Western Cape, Cape Town, South Africa R Maart,1 BChD, MPhil (Higher Education); K Mostert-Wentzel,2 MPhysT, MBA, PhD 1

Department of Prosthetics, Faculty of Dentistry, University of the Western Cape, Cape Town, South Africa

Department of Physiotherapy, Faculty of Health Sciences, University of Pretoria, South Africa

Corresponding author: R Maart (rmaart@uwc.ac.za)

Background. Dental students are extensively trained to provide dental treatment to their patients during the undergraduate programme. However, no or little time is spent on the training of basic communication skills. Embedding a communication course in the curriculum would require support of the teaching staff and clinical teachers. Objective. To explore the perceptions of the clinical teachers with regard to the teaching and learning of dental student communication. Methods. The study had a two-phase, sequential, exploratory, mixed-methods research design. The first phase explored the lecturers’ perceptions of dental student-patient communication by means of a focus group interview (n=5). Findings were used to develop the questionnaire for the second phase. The survey was distributed to 57 clinical teachers using the online SurveyMonkey system (USA). Results. Sixty-three percent rated dental student-patient communication as good. All the respondents thought communication skills should be included in the dental curriculum. Of the total, 47% strongly agreed that students wish to continue with their work and not attend to communication. Of the clinical teachers, 47% strongly agreed that they need training on how to communicate and assess communication skills. Sixty-eight percent strongly agreed that developing and teaching a communication module should be shared among faculty staff. Conclusion. Clinical teachers agreed that communication skills training and clinical assessment in the dental curriculum are important. The study raised awareness among faculty members about the importance of communication skills and ensured initial buy-in for the development of such a course. Afr J Health Professions Educ 2016;8(1):69-71. DOI:10.7196/AJHPE.2016.v8i1.571

Clinical, scientific and interpersonal skills have been recognised as important components of the dental curriculum. Good interpersonal skills enhance studentto-patient relationships.[1,2] Communication is one of the salient skills in clinicians’ relationship with patients. It is an element that is often overlooked and underemphasised, both at dental school and in continuing education.[3] Dental students at the University of the Western Cape (UWC), Cape Town, South Africa (SA) are thoroughly trained to provide dental treatment to their patients during their undergraduate programme; however, little time is spent on teaching and learning basic communication skills. At this institution the emphasis is directed towards ensuring that students are competent clinicians. However, the literature shows that the ability to communicate effectively with patients is crucial; the better communicators we are, the better clinicians we will become.[2,3] A communication course for dental students at UWC would be necessary to improve dental student communication. The inclusion of a communication course has human resource implications; the course has to be designed, taught, assessed and evaluated. The success of curricular change requires faculty buy-in and consensus. [3] Being confronted with a paradigm shift in dental education can create uncertainty and resistance among faculty.[4] Embedding a communication course in the undergraduate dental curriculum at the UWC dental school would require the support of all teaching staff and clinical teachers. Therefore, the purpose of this research was twofold: (i) to explore perceptions; and (ii) to create awareness among clinical teachers with regard to dental student communication.

Methods

Ethical clearance for the research was obtained from the Dental Faculty Ethics Committee, UWC (reference 13/4/36). Written consent was obtained from all participants. A sequential mixed-methods research design was chosen. This research was conducted at the UWC dental faculty in 2013. First, qualitative data were collected from a focus group discussion with dental clinical teachers (n=5). During this discussion, the clinical teachers were asked to give their opinions on how students communicate and empathise with patients; if and how a communication skills course should be included in the undergraduate curriculum; possible education strategies to improve dental communication between students and patients; and involvement of faculty in future communication education. Data from the interview guided the development of the questionnaire used in the second phase of the survey. Quantitative data were collected from this phase of the survey. SurveyMonkey, an online cloud-based survey tool (USA), was used to distribute the survey to all full- and part-time clinical teachers (n=57). Closedand open-ended questions were used in the survey. Data were entered on a Microsoft Office Excel 2010 spreadsheet, descriptive data analysis was applied to the closed-ended questions, and common themes were identified.

Results

Thirty-five percent of clinical teachers (n=20) completed the question naire. Thirty-seven percent of the respondents had 11 - 15 years of clinical teaching experience, 58% were female and 79% were full-time employees at UWC. Forty-two percent were between the ages of 31 and 40 years, 26% between 51 and 60 years, and only 11% <30 years.

May 2016, Vol. 8, No. 1 AJHPE

Research Sixty-three percent rated dental student commu nication with patients as good, although 47% strongly agreed that students just want to ‘get on with the work’ and that they see patients as quotas; a definite lack of empathy with patients was observed. Although only 53% strongly agreed that communication skills can be taught, 63% often commented on the student-to-patient communication during clinical supervision of students. Forty-seven percent strongly agreed that communication skills with patients should form part of the education of the clinical assessment. All of the respondents indicated that communication skills should be included in the dental curriculum in future. Sixty-four percent suggested that communication skills should be taught during the 2nd and 3rd years, while 26% suggested that these skills should be taught throughout the 5-year dental curriculum (Fig. 1). The majority of respondents (84%) agreed that video-recording of student-topatient communication and case discussions should be included as teaching methods in the communication skills course. The recommended education strategies are shown in Fig. 2. Forty-seven percent of clinical teachers agreed that they need training on how to communicate and assess communication skills effectively.

11%

1st year

26%

2nd year 3rd year

32% 32%

Other

r l

he Ot

te l

re s tu

ow -d Sh

sio us sc

di e Ca s

Le c

g la yin

-p le

re co rd i o Vi de

90 80 70 60 50 40 30 20 10 0 s

Fig. 1. Participants’ response with regard to where in the curriculum communication skills should be taught.

Fig. 2. Participants’ response (%) with regard to which teaching methods should be included in the teaching of communication skills.

Furthermore, 68% strongly agreed that developing and teaching a communication module should be shared among faculty members and not become the responsibility of one department only. The findings need to be interpreted with caution, in light of the limited participation of departments in the focus group discussion and the low response rate for the questionnaire.

Discussion

The objective of this study was to explore the perceptions of clinical teachers with regard to dental student communication and its teaching and learning. The sample included clinical teachers with regard to years of experience, clinical field of interest, gender, and age, and working mostly full time. From the findings of the survey it is clear that the clinical teachers agree that dental student-to-patient communication is important. Although the majority were of the opinion that students already demonstrate good communication skills, a lack of empathy with patients was observed. The ability to convey messages clearly to patients, listen, be observant and respond to patients’ needs, empathise, understand and carry out consultations in an organised and a professional manner are considered basic requirements of a competent dentist.[1] The clinical teachers agreed that a communication skills module should be included in the future dental curriculum. They suggested that such a course be embedded throughout the curriculum; the objective of this strategy would be to align what is taught and assessed. The inclusion of communication-related topics in a dental curriculum was also favoured in a study by Woelber et al.[5] and Cannick et al.[6] The integration ladder in curriculum planning has been accepted as an important educational strategy in medical education and can be used as an aid to planning, implementing and evaluating the medical curriculum.[7] The higher one is on the integration ladder, the more important communication and joint planning between teachers become, requiring greater participation by staff.[7] The results have suggested video-recordings and case discussions as educational strategies to teach communication in the dental curriculum. Educational strategies tend to vary in type and duration between studies, but those used appear to be similar, involving a mixture of didactic episodes and clinical scenarios.[8] If the purpose of education is to provide students with appropriate skills, the lecture-only approach in many dental schools is not sufficient. Active practice is necessary to learn communication skills.[9] Role-playing was

May 2016, Vol. 8, No. 1 AJHPE

highlighted as a possible teaching strategy, which corresponds to findings from Woelber et al.[5] and Rider and Keefer.[10] The latter suggest rolemodelling as a teaching strategy to be explored formally in the communication skills of modules and informally by all clinical teachers. The use of simulated patients who are skilled at presenting complex clinical conditions, monitoring students’ performance, and delivering specific feedback are components of a communication skills course.[11] Clinically relevant scenarios enable students to develop skills such as problem-solving, increased knowledge of referral processes, and awareness of ethical/legal issues.[11] Evidence from studies that used video-reviewing for learning and assessment supported its use, and it was well received by the majority of students. Only the student and patient need to be present for the consultation, which might minimise the feeling of ‘unreality’ of scenarios when one is simultaneously observed by peers, as reported by some students.[8] In addition to including such a course, clinical teachers have nevertheless admitted that they require training in teaching and assessing commu nication skills. Faculty development can be a powerful tool in initiating and setting the direction for curricular change.[3] Bylund et al.[12] showed that teaching and encouraging facilitators can lead to positive outcomes when training students in communication skills. A foundation for future staff participation during curriculum development and integration was established by engaging clinical teachers during the focus group interviews and participating in the questionnaire. Faculty buyin and participation in curricular change are essential in any academic context and therefore need to be prioritised. Curriculum planners often initiate and plan curriculum changes without consultation with the faculty members responsible for the implementation. This topdown approach is frequently met by faculty resistance; therefore, we recommended that faculty buy-in and participation in curriculum change are considered as viable alternative strategies. Embedding communication into the under graduate curriculum has an effect on human resources. Faculty members developing the course need to include teaching and assess ment of this aspect, as a new course adds to the work load. Human resource planning needs to form part of the curriculum planning process to ensure that the desired educational outcome is achieved.

Research Conclusion

From the results of this study it is apparent that the clinical teachers agree that effective communication skills are important in dental student training and should be included in the curriculum. The study demonstrated a way of raising awareness in the faculty about the importance of communication skills and ensured initial buy-in for the development and integration of such a communication course. After this study, embedding communication throughout the dental undergraduate curriculum, and implementation and evaluation thereof, are recommended. Acknowledgements. The authors thank Mrs Y Erasmus for her assistance with the data collection. We also acknowledge the sub-Saharan African Foundation for Advancement of International Medical Education and Research (FAIMER) Regional Institute (SAFRI) for its support.

References 1. Gonzalez M, Abu Kasim N, Naimie Z. Soft skills and dental education. Eur J Dent Educ 2013;17(2):73-82. [http:// dx.doi.org.10.1111/eje.12017] 2. Abu Kasim NH, Abu Kassim NL, Razak AA, et al. Pairing as an instructional strategy to promote soft skills amongst clinical dental students. Eur J Dent Educ 2014;18(1):51-57. [http://dx.doi.org/10.1111/eje.12058] 3. Steinert Y, Cruess S, Cruess R, et al. Faculty development for teaching and evaluating professionalism: From programme design to curriculum change. Med Educ 2005;39:27-136. 4. Spallek H, Oâ&#x20AC;&#x2122;Donnell J, Yoo Y. Preparing faculty members for significant curricular revisions in a school of dental medicine. J Dent Educ 2010;74(3):75-288. 5. Woelber J, Deimling D, Langenbach D, et al. The importance of teaching communication in dental education: A survey amongst dentists, students and patients. Eur J Dent Educ 2012;16(1):e200-e204. 6. Cannick F, Horowitz A, Garr D, et al. Use of the OSCE to evaluate brief communication skills training for dental students. J Dent Educ 2007;71(9):1203-1209. 7. Harden R. The integration ladder: A tool for curriculum planning and evaluation. Med Educ 2000;34:551-557. 8. Carey J, Madill A, Manogue M. Communications skills in dental education: A systemic research review. Eur J Dent Educ 2010;14:69-78. [http://dx.doi.org.10.1111/j.1600-0579.2009.00586.x] 9. Yoshida T, Milgrom P, Coldwell S. How do US and Canadian dental schools teach interpersonal communication skills? J Dent Educ 2002;66(11):1281-1288. 10. Rider E, Keefer C. Communication skills competences: Definitions and a teaching toolbox. Med Educ 2006;40:624-629. [http://dx.doi.org/10.1111/j.1365-2929.2006.02500.x] 11. Hannah A, Millicahamp J, Ayers K. A communication course for undergraduate dental students. J Dent Educ 2004;68(9):970-977. 12. Bylund C, Brown R, Lubrano di Ciccone B, et al. Training faculty to facilitate communication skills training: Development and evaluation of a workshop. Patient Educ Couns 2008;70:430-436. [http://dx.doi.org.10.1016/j.pec.2007.11.024]

May 2016, Vol. 8, No. 1 AJHPE

Research An online formative assessment tool to prepare students for summative assessment in physiology S Kerr,1 PhD; D Muller,1 PhD; W McKinon,1 PhD; P Mc Inerney,2 PhD School of Physiology, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa

Centre for Health Science Education, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa

Corresponding author: S Kerr (samantha.kerr@wits.ac.za)

Background. The didactic approach to teaching physiology in our university has traditionally included the delivery of lectures to large groups, illustrating concepts and referencing recommended textbooks. Importantly, at undergraduate level, our assessments demand a level of application of physiological mechanisms to recognised pathophysiological conditions. Objective. To bridge the gap between lectured material and the application of physiological concepts to pathophysiological conditions, we developed a technological tool approach that augments traditional teaching. Methods. Our e-learning initiative, eQuip, is a custom-built e-learning platform specifically created to align question types included in the program to be similar to those used in current assessments. We describe our formative e-learning system and present preliminary results after the first year of introduction, reporting on the performances and perceptions of 2nd-year physiology students. Results. Students who made use of eQuip for at least three of the teaching blocks achieved significantly better results than those who did not use the program (p=0.0032). Questionnaire feedback was positive with regard to the administration processes and usefulness of eQuip. Students reported particularly liking the ease of access to information; however, <60% of them felt that eQuip motivated them to learn. Conclusion. These results are consistent with the literature, which shows that students who made use of an online formative assessment tool performed better in summative assessment tasks. Despite the improved performance of students, the questionnaire results showed that student motives for using online learning tools indicated that they lack self-directed learning skills and seek easy access to information. Afr J Health Professions Educ 2016;8(1):72-76. DOI:10.7196/AJHPE.2016.v8i1.581

The traditional approach to teaching physiology at our university has been the delivery of a didactic lecture by an expert standing in front of the class. The lecture format usually involves the use of PowerPoint slides or chalk and a blackboard to illustrate concepts. After the lecture, students are referred to a recommended non-South African textbook.[1] Frequently, especially in large medical student classes, the lecturer may be unavailable on a one-to-one basis for follow-up tuition. Therefore, students may find themselves in a situation where misconceptions or poorly received concepts are not easily addressed and corrected. These students may subsequently ignore or discard the information and consequently perform poorly in assessments. They could therefore overlook the relevance of the knowledge, which in the context of medical disciplines, may have an effect on their careers and ability to treat their patients. Moreover, diligent students who seek to further investigate a concept, frequently do not know where to find the relevant information in their textbook, the library or even on the internet. A number of different emerging factors have made the challenges of effective teaching of healthcare professionals even more difficult in the local South African (SA) context. These include a changing school education system where disparate pedagogical approaches have been implemented over the last decade;[2] a changing patient population, with the emergence of previously neglected or unknown diseases;[3] and a lack of available resources, making clinical insight even more important as the primary tool available to healthcare workers. Consequently, long-established didactic approaches to the teaching of physiology may no longer fully meet the needs of current students or fulfil the demands for effective training of healthcare professionals. The promotion of active and self-directed learning,

May 2016, Vol. 8, No. 1 AJHPE

which may be defined as the process in which students identify their learning niches and strengths and take the initiative to fulfil their learning requirements, may be achieved through an e-learning system[1] that may prove to be a useful aid in supplementing the learning of students. A review article by Michael,[4] focusing on physiological education, highlights the evidence that active learning approaches to teaching are more effective than passive methods. The evidence suggests that active learning promotes meaningful learning and performance is improved if these methods are employed. Online learning tools, which promote active learning by students, provide a supplemental resource through which students may engage the use of assessment with feedback, which potentially allows for tutoring that may not otherwise have been available. There is evidence to suggest that e-learning tools are an effective method of providing additional teaching aids to students and also improve outcomes on summative test assessments.[5] Formative assessment provides immediate feedback to students and promotes student learning, whereas summative assessment is an evaluation of student learning.[6] e-learning systems are largely formative tools employed to assist students in summative assessment tasks.[5] According to Velan et al.,[7] the advantages of online formative assessments include â&#x20AC;&#x2DC;immediacy of feedback, flexibility in time and place of undertaking the assessment; feedback can provide links to learning resources, thereby providing motivation to study, opportunity for repetition, and interactivityâ&#x20AC;&#x2122;.[7] Therefore, in our school, we have decided to embrace the technological approach to assist in meeting the needs of current learners. The use of computerised learning methods not only appears to lend themselves to the tech-savvy nature of the new generation of learners, but also allows students

Research to learn physiological concepts at their own pace, revisit misunderstood concepts and shift their focus to where they may need the most conceptual support. In this article we assess the results of implementing our new e-learning system, designed as a formative teaching tool with the objective to provide factual knowledge and immediate results of formative assessments to students, including detailed explanations of the correct physiology after each assessment. The system has been designed to provide a student with experience in answering test questions relevant to their discipline, while explaining basic physiological concepts. Furthermore, our system is designed to provide SA students with knowledge applicable and relevant to the local population rather than information more relevant to North American or European populations, which are conventionally used. In this study, our objective was to assess the performance of students (summative assessment) after the introduction of an e-learning tool as a supplemental tool for learning and the subjective reactions of students to using such a tool.

Background Students

The School of Physiology, University of the Witwatersrand, Johannesburg, SA teaches courses in physiology and medical biochemistry as compulsory components of 2nd-year undergraduate curricula in a range of different medical disciplines. These include pharmacy, physiotherapy, occupational therapy, nursing, dentistry, medicine, health sciences and biomedical engineering. The physiology course is divided into four 8 - 10-week blocks and is delivered through a series of lectures (5 hours per week), tutorials (3 hours per week) and practical laboratory skill sessions (3 hours per week). All lectures and tutorials are taught by academic staff members and practical sessions are demonstrated by trained postgraduate students under the guidance of academic staff members. The therapeutic health science course is taught to students studying pharmacy, physiotherapy, occupational therapy, nursing and dentistry as one class (N=220). Students write a summative test at the end of each of the four teaching blocks (based on the work covered in that quarter) and a final assessment at the end of the year (based on the course content taught throughout the year). The therapeutic health science course material includes the following: Test 1: blood, body fluids, immunology, metabolism, autonomic nervous system and excitable tissue; Test 2: metabolism, central nervous system and temperature regulation; Test 3: cardiovascular system, respiration, kidney, acid-base and gastro-intestinal system; and Test 4: liver, endocrine and exercise.

eQuip

In addition to lectures, tutorials and laboratory sessions, which were already in place, an e-learning initiative was introduced as a tool to supplement studentsâ&#x20AC;&#x2122; understanding of physiology and the method of assessment. eQuip (eQuestions for Understanding Integrated Physiology) is a custom-built e-learning platform to suit the specific needs of the school (developed by the school in collaboration with coding expertise supplied by an external information technology firm (Simnat Technology Pty Ltd, Johannesburg). The framework for the platform was based on the X-type multiple-choice questions (MCQs) used in our current assessments, developed by Mitchell.[8] In our assessment protocol and eQuip program each MCQ statement has five corresponding responses, where each response must be answered as true or false. Each MCQ therefore contains five questions or statements that require an answer. In the eQuip program, once a question has been answered (true or false selected), the student is

given feedback on whether they were correct or incorrect. Additionally, an extensive explanation as to why the statement was true or false is presented to the student. Model answers have been researched and validated under the supervision of subject expert academic staff to align the model answers with current course content and objectives. Furthermore, images and explanations detailing the underlying mechanisms that explain the reasons for statements being correct or incorrect have been sourced, designed and included, enabling the students to visualise the rationale behind each statement. Selected responses also have links to approved internal library e-references or external sites (such as YouTube videos) that further clarify concepts. Therefore, eQuip was specifically designed and created to align the question types included in the program to be similar to those used in our current assessment protocols, providing experience in answering X-type MCQs and offering immediate feedback. The eQuip program draws on Kolbâ&#x20AC;&#x2122;s experiential learning cycle, as it offers students the opportunity for concrete experience and active experimentation; the immediate feedback provides them with an opportunity for reflective observation and abstract conceptualisation.[9] The home page of eQuip is divided into 16 topics, reflecting the structure of the course topics taught throughout the year. There are approximately 500 MCQs with model answers captured in the system. A unique feature of the system is that many of the questions included are based on diseases or problems prevalent in patients seen in African hospitals. Thus, at an early point in their medical career, students are being exposed to problem-solving around problems and diseases such as malaria, HIV and kwashiorkor.

Methods

Study design

At the start of the academic year all the students who registered for therapeutic health sciences disciplines were given a brief orientation session to familiarise them with eQuip and encourage them to use the platform. Participation was voluntary and no extra credit incentives were offered to encourage use of the program. Students accessed eQuip using their personal student identification username and password. No time limitations were placed on answering the questions, as it was felt that this was a learning platform and students could take as much time as required to interact with the system. Because of internet security issues, the site was only accessible from computer laboratories at the university and students could not access the platform off campus.

Questionnaire

A paper-based questionnaire to obtain feedback about eQuip was administered to the students in the final lecture of the year as part of the department-approved course evaluation. This formed a convenience sample, as not the entire class was present. The students were asked to volunteer to complete the questionnaire and could elect to complete it anonymously. It was made clear to them that there was no obligation to complete the questionnaire and they would not be disadvantaged should they choose not to participate. The questionnaire administrators and e-learning team played no role in the summative assessment process. Using a 5-point Likert scale, eQuip-specific questions were answered on ease of access, frequency of usage, relevance of material and whether students were satisfied and would recommend the program. Furthermore, based on the questionnaire developed by Seluakumaran et al.,[10] students were asked what the most useful features were.

May 2016, Vol. 8, No. 1 AJHPE

Research Outcomes

Student logins to eQuip were tracked throughout the year to determine the percentage use of the program and the number of teaching blocks each student made use of. Fisher’s exact 2 × 2 contingency tables were used to assess the relationship between students who passed and those who failed (50% cut-off) and their eQuip use per teaching block. Year-end grades (average of the results of the test at the end of each of the academic teaching blocks) were compared (using analysis of variance (ANOVA) with a Tukey’s posthoc test) according to the number of teaching blocks where the student had made use of eQuip.

Results

Participants

The mean (SD) percentage of students who used eQuip throughout the year was 67 (6.9)%. A statistically significant percentage of students who passed used eQuip (62%), compared with only 25% of those who failed (p=0.0036). A significantly greater proportion of students who

Grade average, %

†

60 50 40 30 0

used eQuip in all four teaching blocks passed compared with those not making use of the program (p=0.0006; Table 1). Students using eQuip for at least three academic blocks achieved significantly better year-end results than those who did not make use of the program (Fig. 1). Students were normally distributed in terms of grades achieved in the group using eQuip and those not using the program.

Questionnaire results

Completed questionnaires were received from 66% (n=146) of the class, and 92% of respondents stated that they had made use of eQuip at least once (Table 2). The students indicated that they made use of the e-learning tool most frequently immediately before tests. Although 146 questionnaires were collected from the class, some of the respondents elected not to answer all the statements. Accordingly, each of the statements was calculated as a percentage of the responses for that statement (Tables 2 and 3). In each case ≤20 students failed to provide an answer to any question posed. Overall, feedback from the class was positive with regard to the registration process, access, ease of use, satisfaction and relevance of material on eQuip. Students found the MCQ model answers and images particularly useful, but provided mixed reactions about the links to other sites.

Discussion 0

Number of teaching blocks students used eQuip

Fig. 1. Comparison between physiology year-end grade averages for therapeutic health science students based on the number of blocks for which the students used the e-learning tool eQuip (ANOVA, p=0.0032; Tukey’s posthoc test, three blocks used v. eQuip not used (*p<0.05), four blocks used v. eQuip not used (†p<0.01)).

Students who made use of eQuip achieved significantly better results in their summative tests than those who did not use the program, suggesting that the introduction of the online tool facilitated the learning process for students. This study is congruent with numerous studies focusing on medical education, which have also demonstrated that students who participate in formative online assessments

and use supplementary tools performed better in summative tests and assessments.[5-7,10-13] The significant differences in year-end results between the students who consistently used eQuip and those who did not may be a reflection of the higher percentage of use of the program and that students received greater benefit from the system. Kibble[12] noted that students are less likely to participate in formative assessment tasks if participation is voluntary. Although the average use of the program throughout the year was >60%, ~13% of the class reported that they had either not heard or made use of it, in the latter case primarily because access was restricted to on-campus use. Therefore, it is worthwhile noting that frequent promotion and accessibility are key features in the success of implementing an e-learning program. Historically, students have struggled in the first tests in our 2nd-year physiology course, as they are introduced to a new question format and a more challenging course than they experienced during their 1st year of study. By modelling the question type included in the e-learning program on the assessment protocol, students were familiarised with the question format when using the online system before writing the test, which may have assisted them when writing the summative assessments. Familiarisation with the question format highlights one of the strengths of eQuip, which exposes students to the question type and expectations in answering styles, thus providing them with experience with the question type. Previous literature has shown formative assessments (such as our eQuip-based questions) to be most effective if they are similar to the summative assessments that they are supporting.[5,14] Features of the eQuip system that we considered key to the successful implementation of our formative e-learning tool included the creation of a system that encouraged the learning process and provided timely and

Table 1. Students who passed or failed 2nd-year physiology according to the number of teaching blocks for which they used the e-learning program eQuip

Table 2. Summary of student feedback with regard to the use of eQuip

Number of teaching blocks students used eQuip, n

Students who passed, %

Students who failed, %

Did you make use of eQuip?

Students, %

I looked at it once

I used it just before tests

I consistently used it

I knew about it but I never used it

What is eQuip? I’ve never heard about it or used it

*p=0.0064 v. 0 (Fisher’s exact test).

May 2016, Vol. 8, No. 1 AJHPE

Research

Table 3. Summary of therapeutic health science students (physiology) feedback with regard to eQuip, provided by a questionnaire using a condensed Likert scale Students, % Student feedback

Agree

Neutral

Disagree

The registration process for eQuip was easy

eQuip was easy to access

I was happy and satisfied to use eQuip

eQuip was easy to navigate and use

The material on eQuip was relevant and appropriate

eQuip helped me to further my understanding about the topic taught in physiology

eQuip complemented traditional learning through lectures and tutorials

I recommend the School of Physiology to continue using eQuip

I don’t think I learnt much from eQuip

I like reading notes/textbooks rather than using eQuip

eQuip motivated me to learn

Model answers to each question

The images

That it allowed me to practise MCQs, but wasn’t for marks

Easy access to the model answers

Links to other sites

The most useful features of eQuip

informative feedback. Marden et al.[15] reported that formative learning tools were more useful when students were allowed multiple unsupervised and untimed attempts. Therefore, another strength of e-Quip is that it creates a non-intimidating environment for students to assess and re-assess their knowledge and practise questions. Furthermore, to facilitate student understanding and foster a student’s sense of responsibility for their learning, it was seen as beneficial that our formative assessment provides informative and supportive feedback, rather than just a grade, and does so in a timely manner to optimise learning outcomes.[16,17] We cannot definitively state that the higher results obtained by eQuip users compared with the results of those who did not use the program were because more conscientious students made use of the system and whether they would have obtained such results without the program. However, the distribution curves of the grades for students who used the system and those who did not indicate that there was an even distribution of strong and weak students. Two studies by Kibble et al.[11,13] demonstrated that students were more likely to participate in formative assessment tasks if

there were extra credit incentives. However, as noted by Kibble, increasing incentives to participate may result in a disconnect between the final grades and the amount of online participation. If incentives are offered, the motivation for using the program may not be the attainment of knowledge but rather to achieve the necessary grades. In contrast to incentive-driven engagement with a formative assessment tool, voluntary parti cipation in formative assessment tasks is more likely a reflection of self-directed learning. Our students had the freedom to access information from a reliable source at their convenience (albeit limited to on-campus access) and they could control the amount of in- and output that they obtained, essentially empowering them to take control of their learning. Self-directed learners are able to identify their own deficits and make amends. eQuip lends itself to this, as it allows students to work at their own pace and apply information. This is particularly useful in the local setting, as the majority of the class are second-language learners (not learning in their mother tongue). Conversely, it is possible that the easy access to answers on eQuip may actually facilitate a

lack of self-directed learning. The results from the questionnaire indicated that students were motivated to use eQuip, as the system provided them with easy access to model answers, which the students rated as particularly useful. It is possible that students viewed the system as a means to attaining the correct answers and not as a learning tool. Taradi et al.[18] captured this idea by stating ‘the real power of computers to improve education will only be realized when students actively use them as cognitive tools rather than passively perceive them as tutors or repositories of information’. The mixed student reaction to the additional resources supplied supports the notion that students may have used eQuip merely as an information repository. We expected that the additional resources would guide students to further their knowledge and correct misconceptions through their own learning process. However, <50% of students agreed that the links to other sites were useful. We speculate that the students may have considered it too much effort to access the additional material or that this information was superfluous to the content provided by their lecturers. Further investigation into reasons why these resources were not utilised and which resources were used is required.

Study limitations

Results from our questionnaire survey found that students complained about access to the system being limited to computer laboratories on campus and felt that they would have used the program more often if they could have accessed it from their home environment, an issue which we have subsequently addressed and rectified. Additionally, technical difficulties with access to the system were resolved when students reported such issues. As access to the system is a fundamental requirement for compliance, the provision of technical assistance (in our case, by making the contact details of the system administrator available to students) is vital to its success. The study design could not control for potential confounding variables regarding the study population. We previously stated that the higher grades obtained by eQuip users could have been because the more conscientious students made use of the system and would, regardless, have obtained higher marks; this should be considered a limitation. Moreover, the motivating factors and likelihood to commit to learning in an online environment were not

May 2016, Vol. 8, No. 1 AJHPE

Research considered in this study, which is a potential limitation and requires further research.

Conclusion

The implementation of a physiological e-learning program was related to improved summative assessment performance in a cohort of undergraduate students; however, improvements are only evident with frequent use. These results are consistent with those in the literature, which recommend the use of active online learning tools to promote educational performance. Results from a questionnaire assessing the students’ experiences suggested that the program was found to be a useful and relevant addition to our undergraduate physiology courses. Recommendations include further research into the additional resources the students are utilising and motivations behind committing to online learning tools. Moreover, encouraging students to frequently use the platform and take responsibility for their learning should further improve performance. Acknowledgements. We would like to thank Prof. Elena Libhaber, University of the Witwatersrand, for her assistance with the statistical analysis. References 1. Gorman PJ, Meier AH, Rawn C, Krummel TM. The future of medical education is no longer blood and guts, it is bits and bytes. Am J Surg 2000;180:353-356. [http://dx.doi.org/10.1016/S0002-9610(00)00514-6]

May 2016, Vol. 8, No. 1 AJHPE

2. Aldridge JM, Laugksch RC, Fraser BJ. School-level environment and outcomes-based education in South Africa. Learning Environments Research 2006;9(2):123-147. [http://dx.doi.org/10.1007/s10984-006-9009-5] 3. Etyang AO, Scott JAG. Medical causes of admissions to hospital among adults in Africa: A systematic review. Glob Health Action 2013;6(1):1-14. [http://dx.doi.org/10.3402/gha.v6i0.19090] 4. Michael J. Where’s the evidence that active learning works? Adv Physiol Educ 2006;30:159-167. [http://dx.doi. org/10.1152/advan.00053.2006] 5. Dobson JL. The use of formative online quizzes to enhance class preparation and scores on summative exams. Adv Physiol Educ 2008;32:297-302. [http://dx.doi.org/10.1152/advan.90162.2008] 6. Olson BL, McDonald JL. Influence of online formative assessment upon student learning in biomedical science courses. J Dental Educ 2004;68:56-65. 7. Velan GM, Jones P, McNeil HP, Kumar RK. Integrated online formative assessments in the biomedical sciences for medical students: Benefits for learning. BMC Med Educ 2008;8:52. [http://dx.doi.org/10.1186/1472-6920-8-52] 8. Mitchell G. Medical Physiology: Objectives and Multiple Choice Questions. 2nd ed. Durban: Butterworths, 1986. 9. Kaufman DM, Mann KV. Theory and learning in medical education: How theory can inform practice. In: Swanwick T, ed. Understanding Medical Education. Evidence, Theory and Practice. London: Wiley Blackwell, 2014. 10. Seluakumaran K, Jusof FF, Ismail R, Husain R. Integrating an open-source course management system (Moodle) into the teaching of first-year medical physiology course: A case study. Adv Physiol Educ 2011;35:369-377. [http://dx.doi.org/10.1152/advan.00008.2011] 11. Kibble J. Use of unsupervised online quizzes as formative assessment in a medical physiology course: Effects of incentives on student participation and performance. Adv Physiol Educ 2007;31:253-260. [http://dx.doi. org/10.1152/advan.00027.2007] 12. Kibble JD. Voluntary participation in online formative quizzes is a sensitive predicator of student success. Adv Physiol Educ 2011;35:95-96. [http://dx.doi.org/10.1152/advan.00053.2010] 13. Kibble JD, Johnson TR, Khalil, MK, et al. Insights gained from analysis of performance and participation in online formative assessment. Teach Learn Med 2011;23(2):125-129. [http://dx.doi.org/10.1080/10401334.2011.561687] 14. Rolfe I, McPherson J. Formative assessment: How am I doing? Lancet 1995;345:837-839. [http://dx.doi. org/10.1016/S0140-6736(95)92968-1] 15. Marden NY, Ulman LG, Wilson FS, Velan GM. Online feedback assessments in physiology: Effects on students’ learning experiences and outcomes. Adv Physiol Educ 2013;37:192-200. [http://dx.doi.org/10.1152/ advan.00092.2012] 16. Gipps V. What is the role for ICT-based assessment in universities? Studies in Higher Education 2005;30(2):71180. [http://dx.doi.org/10.1152/advan.00092.2012] 17. Hattie J, Timperley H. The power of feedback. Rev Educ Res 2007;77(1):81-112. [http://dx.doi. org/10.3102/003465430298487] 18. Taradi SK, Taradi M, Radić K, Pokrajac N. Blending problem-based learning with Web technology positively impacts students learning outcomes in acid-base physiology. Adv Physiol Educ 2005;29:35-39. [http://dx.doi. org/10.1152/advan.00026.2004]

Research The forensic autopsy as a teaching tool: Attitudes and perceptions of undergraduate medical students at the University of Pretoria, South Africa L du Toit-Prinsloo,1 MB ChB, DipForMed (SA) Path, FCForPath (SA), MMed (Path) (Forens); G Pickworth,2 BSc, BSc Hons (Psych), MEd (Psych), DPhil (Psych); G Saayman,1 MB ChB, MMed (MedForens), FCForPath (SA) 1

Department of Forensic Medicine, Faculty of Health Sciences, University of Pretoria, South Africa

Department of Education Innovation, Faculty of Health Sciences, University of Pretoria, South Africa

Corresponding author: L du Toit-Prinsloo (lorraine.dutoit@up.ac.za)

Background. Numerous articles have been published on the use of autopsies in training medical students in anatomy and different branches of pathology. Some authors have described the emotional response of students who attend such postmortem sessions. Forensic pathology is an important subdivision of pathology. In some countries undergraduate medical students are expected to attend postmortem examinations on persons who died from traumatic causes. Objective. To determine the attitudes and perceptions of 5th-year medical students with regard to forensic postmortem examinations at the University of Pretoria, South Africa. Methods. A questionnaire was voluntarily completed by medical students on the last day of the practical rotation. Results. The overall rating of the practical rotation was 82%. The strengths, weaknesses, opportunities and threats (SWOT) analysis indicated the following as strengths: record keeping, legislation review and traumatology description; as weaknesses: emotional trauma and nightmares; as oppor tunities: the attendance of autopsies; and as threats: physical dangers. Conclusion. The current study was similar to international studies with regard to students’ emotional response to attending autopsies. The autopsy remains a valuable teaching tool for undergraduate students. Emotional support is currently available for all students to assist them in overcoming their fear of attending forensic autopsy sessions. Afr J Health Professions Educ 2016;8(1):77-80. DOI:10.7196/AJHPE.2016.v8i1.589

The use of the autopsy in teaching undergraduate medical students is a well-known worldwide practice.[1,2] Autopsies are used for teaching anatomical and forensic pathology to under- and postgraduate students. Harris et al.[3] reported that general practitioners regard autopsies as ‘being of benefit to education and research’. Their study excluded coroners’ cases and was published as part of the first publication of the UK General Medical Council’s Tomorrow’s Doctors.[3] Burton[4] conducted a qualitative study on the use of the autopsy and the curriculum, indicating that the role of the former is multifactorial and can aid in teaching anatomy and clinicopathological correlations, can be a form of clinical audit and, in the forensic medicine setting, can aid in death certification. Burton included structured interviews with teachers of undergraduate medical students, including histopathologists, non-pathologists, general practitioners, sur geons and physicians. (No forensic pathologists were interviewed.) The participants in Burton’s study indicated that although it is strongly advised that students be exposed to autopsies, they did not feel that it should be compulsory. A number of articles have been written on the perceptions, views and attitudes of students after attending autopsies. In Germany, Tschernig et al.[5] reiterated that the medical student’s first encounter with corpses is usually during anatomy training. These authors indicate that even for the purpose of attending autopsies for anatomy training, students need to be prepared both psychologically and physically. Benbow,[6] from the Division of Histopathology, Department of Pathological Services, University of Manchester, UK, distributed questionnaires to 2ndand 3rd-year undergraduate medical students, reviewing personal details and experiences as well as general questions about autopsies. The study indicated that

medical students viewed the autopsies as useful, although some indicated that the only use is to determine the cause of death. The psychological effects indicated that some students could come to terms with death, while others found it ‘difficult’. Physical symptoms included the worrisome smell, palpitations and tears. Bataineh et al.,[7] from the Department of Anatomy at the Jordan Uni versity of Science and Technology, reviewed questionnaires from preclinical medical students, which indicated that 28.9% expressed a degree of fear before entering the autopsy room, 19.3% had palpitations and 50.3% were concerned about an infection risk. It would appear that there is very little literature that reviews the per ceptions of medical students who attend forensic pathology autopsies. At the University of Athens, Greece, Papadodima et al.[8] indicated that 50% of students who attended forensic autopsies were fearful and anxious, with some experiencing physical symptoms such as nausea. Sergentanis et al.[9] did a follow-up study with the objective of identifying risk factors for the psychological reactions experienced during the forensic medical practical rotation. The authors identified the following risk factors: female gender and ‘stereotypic beliefs about forensic pathologists, a more emotional frame of mind with regard to forensic dissection, more passive coping strategies, and greater fear of death’. He et al.,[10] from the Department of Pathology, Shanghai Hospital, Second Military Medical University, China, reviewed the perceptions of medical students who attended autopsies on individuals who had died from unnatural causes. The students were enrolled voluntarily and divided into groups to view the autopsies from behind a glass partition on television screens linked to a camera, or stood next to the autopsy table. They indicated the advantages of attending the autopsies, e.g. with regard to anatomy teaching: ‘the autopsy is a valuable tool to clinical practice and analysis of

May 2016, Vol. 8, No. 1 AJHPE

Research wound condition and wound rescue in future work’; a method to ‘correlate information from different sources’; and ‘the autopsy is the epitome of problem-based learning’.[10] Disadvantages included that it was physically ‘unpleasant and frightening’ and also ‘an autopsy in human terms is a horrendous thing to do … the mortuary room is a dangerous place’.[10] The authors indicated that the primary objective of the autopsy is to teach pathology; yet, most students indicated its helpfulness in teaching anatomy. The concluding remarks include that the students had a positive attitude towards autopsy (despite discomfort) and its multifactorial role. In South Africa (SA), a study by De Villiers and Ruhaya [11] from the Department of Paediatrics at the University of Limpopo (Medunsa Campus) (currently Sefako Makgatho Health Sciences University, Pretoria) reviewed questionnaires completed by final-year medical students after autopsies conducted in anatomical pathology. Twenty-two percent of students indicated that they felt that the corpses were handled disrespectfully, 58% that the first autopsy attendance was rewarding, and most (it is not exactly quantified) felt uncomfortable, with some experiencing nightmares. In forensic pathology training in SA, many of the medical schools require that undergraduate medical students attend forensic postmortem examinations. In a study by McNamee et al.,[12] from the Department of Forensic Medicine, Nelson R Mandela School of Medicine, University of KwaZulu-Natal, Durban, 10 students took part in semistructured interviews after completion of the forensic medicine practical rotation. All students experienced a degree of discomfort, 70% indicated that they had a better understanding of the mechanism of death and could better differentiate between natural and unnatural causes of death, and 80% indicated that the description of trauma aided a great deal. The current study was undertaken in the Department of Forensic Medicine, University of Pretoria, where 5th-year medical students volun tarily completed a questionnaire after completing a 2-week practical rota tion in forensic pathology. The objective was to review perceptions and attitudes of students towards the practical rotation.

Methods

At the University of Pretoria, 5th-year medical students attend a 2-week practical rotation in the Department of Forensic Medicine (8 consecutive groups). The facility at the Forensic Pathology Service, Medico-Legal Laboratory, Pretoria consists of a large dissection hall with three dissection stations, where the bodies are placed next to each other and opened; organs are eviscerated in the hall and dissected at the dissection stations. Adjacent to the large dissection hall is an autopsy amphitheatre, with a door opening into the large hall. In the autopsy theatre the students have a direct view of the dissection station. The bodies are brought into the theatre in order that an external examination can be performed, but the physical opening of the bodies and evisceration of the organs are conducted in the large dissection hall (behind the door). The students in the theatre view the organ dissection. Each student rotation consisted of ~30 students who were divided into two groups − a large group of ~20, who were seated in the theatre and observed the postmortem examinations, and a smaller group of ~10, who stood in the dissecting room. During the 2 weeks, the students were rotated between the large and small groups. Each student therefore attended ~10 postmortem examination sessions (each session lasted ~90 minutes). Of special note is that the daily cases are viewed prior to the arrival of the students and severely mutilated or decomposed bodies are not shown to the students. After the practical postmortem sessions, the students attended tutorial

May 2016, Vol. 8, No. 1 AJHPE

sessions in a lecture room at the mortuary. These lasted up to 2 hours each and were conducted as PowerPoint presentations, providing information on topical issues, such as death notification and legislation, which are not taught in formal lectures. In 2011, the year of the current study, no specific emotional support or preparation was provided to students before commencement of the rotation. At the end of the 2-week rotation, the students were asked to voluntarily complete a questionnaire. The questionnaire was divided into four parts: • Part A (quantitative): 10 general questions regarding the information provided to students, expectations and organisation. Students indicated their response on a scale of 1 (strongly disagree) - 5 (strongly agree). • Part B (quantitative): the students provided an overall rating of the practical rotation on a scale of 1 - 10. • Part C (qualitative): the so-called strengths, weaknesses, opportunities and threats (SWOT) analysis was used and the students could write their responses under each component. • Part D (qualitative): space was provided for the students to write recom mendations.

Results

In 2011, 220 students were enrolled for the 5th-year of the MB ChB degree. A total of 212 (96%) students completed the voluntary questionnaire at the end of the forensic medicine practical rotation.

Part A. General information: Quantitative

Table 1 summarises the average obtained for each of the questions.

Table 1. General information Question

Average, %

We were well informed where we had to be on the first day of the rotation

We received sufficient information on what would be expected of us in the rotation

The outcomes and objectives of the rotation, as well as the way in which we would be evaluated, were clearly communicated to us

We had sufficient exposure to forensic postmortem examinations (variety of cases) and the medicolegal investigation of death

Our time was used very effectively

The tutorial sessions were well presented and informative

The evaluation assessed what we had learned in the rotation

The consultants and registrars made positive inputs towards our training

The personal protective equipment in the mortuary was up to standard

The rotation was well organised

Part B. Overall rating of the rotation: Quantitative

All eight groups indicated a score of >8/10 as a general impression of the rotation (with an average of 82%).

Research Part C. SWOT analyses: Qualitative

Strengths Several of the students indicated the following as strengths: • the emphasis on applicable legislation pertaining to medical practitioners • the practical exposure and opportunity to attend autopsies • the importance of proper record-keeping • recognising the different aspects of death • how to deal with death (‘how to cope with dramatic deaths and injuries’) • differentiating between deaths due to natural and unnatural causes • the nature of injuries and the terminology of trauma, which were explained • how to determine the cause of death • revision of anatomy. Weaknesses Students indicated that: • they experienced the rotation as emotionally traumatising (n=64) • the rotation showed ‘too much death and dying’ and they were ‘being exposed to death in such a brutal manner’ • the smell was worrisome (n=4), and he/she experienced nightmares (n=1) • he/she could not continue the rotation after 3 days of exposure (owing to nightmares) (n=1) • there had been no debriefing (n=2) • the rotation was too short (n=4). Opportunities Students indicated that: • a wide variety of cases were seen (n=35) • the attendance of autopsies was an opportunity to learn pathology and anatomy, as normal and abnormal findings were seen (n=15) • the attendance taught them the procedure being followed and what to expect during an autopsy (n=8) • information regarding medical negligence was provided (n=7) • the rotation was a ‘good learning opportunity’, but they did not elaborate on the statement (n=5) • they learned a lot about the common causes of death in the medicolegal environment in SA (n=3) • one can ‘overcome fear of death’ (n=1) • he/she would consider forensic pathology as a specialty (n=1). Threats Students indicated that: • personal protective equipment was insufficient (n=6) • they were worried about the risk of infections, specifically tuberculosis (n=2) • the bodies or organs were handled disrespectfully (n=2) • awareness of medical negligence was highlighted (n=4).

Part D. Recommendations

Students recommended that: • attendance should not be compulsory (n=2) • there should be fewer postmortem sessions (n=2) • the practical rotation should be longer (n=5) • a debriefing session/better emotional preparation is needed prior to commencement of the practical session (n=5).

Discussion

The training of undergraduate medical students differs worldwide. In SA, the Health Professions Council of South Africa (HPCSA) is the governing body for healthcare practitioners and prescribes the clear minimum standards of training for medical students. In the core competency lists provided by the HPCSA, the exact curriculum and expectations for forensic medicine/pathology are not clearly stipulated.[13] The autopsy is a valuable tool when teaching several important branches of medicine to undergraduate students, such as anatomy and pathology.[1,2] Undergraduate students are exposed to different forms of learning with regard to the deceased. At most institutions, anatomy teaching exposes students to embalmed bodies, where they often perform the dissections. Attending postmortem examinations in pathology – anatomical as well as forensic – entails viewing a non-embalmed body (in the majority of cases). At the University of Pretoria, medical students attend most of the anatomy dissections in the 2nd year of study, anatomical pathology in the 3rd year and the forensic pathology practical rotation in the 5th year (the undergraduate degree comprises 6 years of study at our institution). The difference in the type of body seen and the expected teaching and learning outcomes makes it difficult to compare studies on the role of the autopsy as a teaching tool. Tschering et al.[5] pointed out that the first encounter with human bodies is usually in the anatomy dissection hall and they refer to the provocative question posed by Lippert in 1985: ‘How humane is human anatomy and to what extent will the behaviour of a medical student during dissection influence his/her future attitude to patients?’ The current study was smilar to other international studies with regard to students’ emotional responses to attending forensic autopsies.[8-10] It would appear that there is a degree of ambiguity in their response to the practical rotation, as the sessions were generally regarded as a positive learning expe rience, with the overall impression receiving a score of >8/10; yet, 30% of the students indicated an emotional aspect (seeing too much of death). An explanation for this discrepancy could be that most of the questions in part A relate to the administration of the rotation (time management, tutorial sessions and assessments), with one question phrased as ‘We had sufficient exposure to forensic postmortem examinations (variety of cases) and the medicolegal investigation of death’. The students scored the latter question 92%, and it could be that even those who found the rotation emotionally distressing agreed that they had ‘sufficient exposure’. A second reason could be that the SWOT analysis might not have been the best tool with which to evaluate the perceptions of students. The study also reiterated the findings of McNamee et al.[12] regarding the emotional aspect surrounding the attendance of forensic autopsies, indicating that the autopsy aided in the description of trauma and differentiated between natural and unnatural causes of death. Statistics SA[14] specified that in 2010, 543 856 people died in SA, a country with an estimated population of ~50 million. Of these deaths, 48 377 were due to unnatural causes. This indicates and emphasises the importance of training undergraduate medical students in the legislation pertaining to registering deaths correctly and how to recognise a death as being due to an unnatural cause. Furthermore, there are only 50 forensic pathologists in SA. This implies that a large number of medicolegal postmortem examinations are being conducted by general practitioners – a reality that further emphasises the importance of training undergraduate medical students in the field. At the University of Pretoria, student feedback confirmed that some students experience emotional distress relating to forensic autopsy sessions.

May 2016, Vol. 8, No. 1 AJHPE

Research This highlights the need to address both the educational goals and emotional impact for students. Emotional support has been introduced, with clinical psychologists being available to students who find it difficult to cope with the forensic autopsy sessions. Currently, a clinical psychologist addresses the entire group at the mortuary after the first postmortem session (due to other clinical engagements this was the only feasible session) and on the last Thursday of the 2-week rotation. During the first session, the clinical psychologist provides a contact number to initiate further emotional support, should the individual need arise. The feedback forms pointed out that all applicable SA legislation is made available to students, and the emphasis of the practical session is on teaching what a generalist medical practitioner should know.

Conclusion

The autopsy as part of the undergraduate training of medical students remains an invaluable tool. In a country with a high number of violent and unnatural deaths, appropriate teaching regarding legislation and management of victims (especially appropriate referral for medicolegal investigation in cases of death) cannot be neglected. Provision needs to be made to support medical students who experience the rotation as emotional. The current study only reflects the opinions of medical trainees and does not represent objective measuring of the evidence of the learning of these students.

May 2016, Vol. 8, No. 1 AJHPE

Acknowledgements. The authors would like to thank Mmes V Tredoux and H Malherbe from the Department of Education Innovation at the Faculty of Health Sciences, Univer sity of Pretoria for capturing the data onto the system. Special thanks also to Ms B English from the Faculty of Health Sciences for the language editing of the article. References 1. DeRoy AK. The autopsy as a teaching-learning tool for medical undergraduates. J Med Educ 1976;51:1016-1018. [http://dx.doi.org/10.1097/00001888-197612000-00011] 2. Hill RB, Anderson RE. The uses and value of autopsy in medical education as seen by pathology educators. Acad Med 1991;66:97-100. [http://dx.doi.org/10.1097/00001888-199102000-00011] 3. Harris A, Ismail I, Dilly S, Maxwell JD. Physicians’ attitudes to the autopsy. J R Coll Physicians Lond 1993;27:116-118. 4. Burton JL. The autopsy in modern undergraduate medical education: A qualitative study of uses and curriculum considerations. Med Educ 2003;37:1071-1081. [http://dx.doi.org/10.1046/j.1365-2923.2003.01710.x] 5. Tschernig T, Schilaud M, Pabst R. Emotional reactions of medical students to dissecting human bodies: A conceptual approach and its evaluation. Anat Rec 2000;261(1):11-13. 6. Benbow EW. Medical students’ views on necropsies. J Clin Pathol 1990;43:969-976. [http://dx.doi.org/10.1136/jcp.43.12.969] 7. Bataineh ZM, Hijazi TA, Hijleh MF. Attitudes and reactions of Jordanian medical students to the dissecting room. Surg Radiol Anat 2006;28(4):416-421. [http://dx.doi.org/10.1007/s00276-006-0101-4] 8. Papadodima SA, Sergentanis TN, Illiaki RG, Sotiropoulos KC, Spiliopoulou CA. Students who wish to specialize in forensic medicine vs. their fellow students: Motivations, attitudes and reactions during autopsy practice. Adv Health Sci Educ Theory Pract 2008;13:535-546. [http://dx.doi.org/10.1007/s10459-007-9065-3] 9. Sergentanis TN, Papadodima SA, Evaggelakos CI, Mytilinaios DG, Goutas ND, Spiliopoulou CA. Students’ physical and psychological reactions to forensic dissection: Are there risk factors? Anat Sci Educ 2010;3(6):287294. [http://dx.doi.org/10.1002/ase.182] 10. He M, Wang J, Zhu Z, et al. A survey study of military medical students to autopsy in modern medical education. Scientific Research and Essays 2011;6(32):6666-6670. [http://dx.doi.org/10.5897/SRE11.1197] 11. De Villiers FPR, Ruhaya M. Students’ opinions on autopsy and death. SA Fam Pract 2005;47(1):47-50. [http:// dx.doi.org/10.1080/20786204.2005.10873172] 12. McNamee LS, O’Brien FY, Botha JH. Student perceptions of medico-legal autopsy demonstrations in a studentcentred curriculum. Med Educ 2009;43:66-73. [http://dx.doi.org/10.1111/j.1365-2923.2008.03248.x] 13. Health Professions Council of South Africa. www.hpcsa.co.za (accessed 14 December 2015). 14. Statistics South Africa. Mortality and causes of death in South Africa, 2010: Findings from death notification. http://www.statssa.gov.za/publications/P03093/P030932010.pdf (accessed 16 March 2014).

Research Preliminary study: Predictors for success in an important premedical subject at a South African medical school N J Allers, MSc, PhD, BEd; L Hay, MSc, PhD; R C Janse van Rensburg, BSc, BA Hons Department of Human Physiology, Sefako Makgatho Health Sciences University, Pretoria, South Africa Corresponding author: N J Allers (nico.allers14@gmail.com)

Background. Admission to Sefako Makgatho Health Sciences University, Pretoria, South Africa is determined by the student’s Matriculation (Matric) results, while the writing of the National Benchmark Test (NBT) is not compulsory. An average student pass rate of 80.1% in physiology (MB ChB II) in the past 13 years compelled this institution to investigate which criteria in the selection process could be predictors for success in physiology, a fundamental discipline for medical studies. Objectives. To compile a profile of very successful physiology students and those who failed the subject, and to identify predictors for success in physiology. Methods. A quantitative study, using Matric and available NBT results, was conducted among the MB ChB II group (N=228) of 2011. Two target groups were identified: highly successful students in physiology (n=37) and those who failed (n=46). Statistical analysis of the results was performed on SAS 9.2, Microsoft Windows (SAS Institute Inc., USA) (p-values ≤0.05 are significant) using the t-test, Fisher’s exact test and the simple linear regression test. Results. These indicated that a good Matric symbol (≥5) in English and life science and a good performance in the quantitative literacy domain of the NBT were statistically significant predictors for success in physiology. Conclusion. This preliminary study indicates the need for our institution to rethink the effectiveness of the selection criteria, redesign the selection process of students for the MB ChB course, and consider making use of the NBT. Afr J Health Professions Educ 2016;8(1):81-83. DOI:10.7196/AJHPE.2016.v8i1.647

Background

The criteria and processes to select students for the MB ChB course have been the subject of much controversy and debate over the past 20 years, as these represent the main mechanisms by which racial and gender imbalances in student profiles can be rectified.[1-4] Consequently, all medical schools have in recent years evaluated and adapted their selection criteria and processes.[5,6] Most South African (SA) medical schools use academic and non-academic criteria to select students, the former accounting for 70 - 80% of admission requirements.[5] Some authors suggest that previous academic performances alone are not a fair reflection of some other very important characteristics required to be successful at medical school.[5] Academic criteria emphasise an overall Matriculation (Matric) pass rate (the M-score) and subject choices; recently, the National Benchmark Test (NBT) was also introduced.[5,7,8] The National Senior Certificate (NSC) is the school-leaving certificate in SA and replaced the Senior Certificate in 2008. Pupils study at least six subjects from six different groups, including two compulsory official SA languages – a first and second language – and four selected subjects. Requirements for Matric pass rates vary, as do those for subject choices. Most medical schools require Matric physical science and mathematics, and in some cases life science, while some also have language requirements.[5] The NBT was a 3-year project commissioned by Higher Education SA, designed as an instrument to assess the NSC system, and calibrated against the previous Matric system. A second and equally important goal of the NBT was to help to assess the validity of the NCS results as a predictor of success in university studies. The NBT provides information about the competence of students with regard to three core domains of knowledge and skills, i.e. academic literacy (AL), quantitative literacy (QL) and mathematics (MA). The tests are written during the year before university entry or at the beginning of the year of entry. Research indicated that many students lack

quantitative concepts and techniques that medical courses appear to assume students have.[6] Although all medical students have studied MA at school, it does not necessarily mean that they are prepared for the QL demands of the curriculum. School MA does not include applying the mathematical techniques in real contexts and the statistical content of the school MA curriculum is very limited.[7]

Problem setting

During recent years, more applicants at Sefako Makgatho Health Sciences University, Pretoria, SA have come from diverse educational, linguistic, cultural and socio economic backgrounds. At this university, admission to medical studies is primarily based on the academic abilities of the student and determined by the student’s Matric results. Lately, the students have also written the NBT, but it is not currently used in the selection process. An average student pass rate of 80.1% in physiology (MB ChB II) in the past 13 years compelled the institution to investigate which factors are predictors of success or failure in this subject. Failure to pass physiology often prevents students from continuing their medical studies. The information obtained by means of this study can be used to identify factors that place students at risk of failure.

Objectives

The objectives of the study were to analyse existing data (Matric and NBT results) of the 2011 MB ChB II group; to compile profiles of successful students and those who failed physiology; and to identify predictors for success in physiology.

Methods

Sampling of specific target groups

Although the study included data from all 2011 2nd-year medical students (N=228), two specific target groups were identified for comparison, i.e.

May 2016, Vol. 8, No. 1 AJHPE

Research highly successful students in physiology, with a final mark of ≥65% (n=36; group 1), and those who failed physiology, with a final mark of <50% (n=45; group 2). The performance of these two groups was compared with regard to the four Matric subjects that were used for their selection, i.e. MA, English, life science and physical science, and also for the NBT results.

Time schedule

Data were analysed after the final physiology results became available.

Data analysis

Statistical analysis of the quantitative results was done by the Statistical Consultation Service of the University of Limpopo, Medunsa Campus. All statistical procedures were performed on SAS 9.2, Microsoft Windows (SAS Institute Inc., USA) (p-values ≤0.05 are significant) using the t-test, Fisher’s exact test and the linear regression test. The last test was used to determine the extent to which there is a linear relationship between the dependent variable (final mark in physiology) and ≥1 independent variables.

Results

Scores for the subjects that were used in the selection process (MA, physical science, life science and English) were taken in consideration. The grading systems for the subjects were between level 5 and 7 – level 5 was the cut-off point. For level 5 a student has to obtain 60 - 69% in a subject, which was regarded as an aboveaverage achievement. The Matric and NBT results for the two groups are presented in Table 1.

Discussion Matric results

As students were selected on the basis of their performance in only four of their Matric subjects, one has to assume that they performed relatively well in these subjects. The overall combined mark also plays a role in the selection; therefore, it does not necessarily mean that they achieved high marks in all four of the subjects. Our study objective was to establish whether performing better in one or more of these subjects could be to the student’s advantage while studying physiology. Higher education institutions in SA mainly use English as the language of preference. At our institution, English is not the mother tongue of the majority of students. The language issue

might be more unique to SA, as minority groups from disadvantaged backgrounds in countries such as the USA and UK are more likely to have been exposed to the language of tuition. At our institution the majority of students come from disadvantaged backgrounds, with little exposure to English. The significance of language is highlighted by the results, which indicated that the successful group had significantly higher marks for English in Matric than their counterparts who failed physiology. A similar tendency was seen for students with English as a home language, but it was not statistically significant, probably due to the low number of students in this category. The other subject in which the successful group showed significantly better results, was life science. The results were supported by the linear regression test. Linear regression analysis for the entire group of students (N=228) showed that of the four Matric subjects, life science was the only statistically significant predictor of the final mark in physiology (p=0.0006). This result was not completely unexpected, as the school syllabus for life science provides a good foundation for physiology. Similar observations were made in both the USA and UK.[9,10] In these countries, chemistry also correlated well with a good performance in medical studies. In SA, however, chemistry is part of physical science,

which includes physics. A surprising finding in our study was that there was no difference between the performances of the two groups for MA and physical science. Therefore, these subjects were found not to be good predictors of success. Similar observations with regard to MA have been made elsewhere in the world.[9] These findings are significant and somewhat ironic as, for many years (before 2008), our institution selected students solely on the basis of their performance in these two subjects, and they were not required to have life science (formerly biology) as a Matric subject.

NBT results

Even though our results and those of others suggest that MA was not a good predictor of success, it is interesting that when applying mathematical skills in problem-solving, as tested by the QL of the NBT, the successful group performed significantly better. This is supported by both the t-test (Table 1) and linear regression analysis. The latter showed that of AL, MA and QL, the last was the only statistically significant predictor of the final mark in physiology (p=0.001). This correlates with the pilot tests of the NBT Project in 2009, which indicated that only onequarter of all students tested were classified as ‘proficient’ in QL.[11] The performance of the

Table 1. Matric and NBT results Matric: students with a ≥ level 5 gradation* in the four essential Matric subjects (Fisher’s exact test) Group 1: students highly successful in physiology (n=36)

Group 2: students who failed physiology (n=45)

Matric subject

n (%)

p-value

English (home language)

10 (80.0)

6 (66.7)

0.60

English (first additional language)

26 (84.6)

39 (61.5)

0.05*

Life science

36 (91.7)

45 (62.2)

0.004*

Physical science

36 (86.1)

45 (86.7)

1.00

Mathematics

36 (91.7)

45 (95.6)

0.65

NBT: average percentage obtained by students in the three NBT domains (t-test)

NBT domain

Group 1: students highly successful in physiology (n=29)†

Group 2: students who failed physiology (n=21)†

Average %

p-value

Academic literacy

53.97

48.24

0.15

Quantitative literacy

52.28

37.05

0.0006*

Mathematics

47.35

43.05

0.22

*p≤0.05 statistically significant. † Students who wrote the NBT, where the results were available.

May 2016, Vol. 8, No. 1 AJHPE

Research group that failed tended to be weaker for both AL and MA, but this is not supported by the statistical analysis. Our results therefore suggest that the NBT could have some value in predicting the success of candidates in their 2nd year of study.

of the contexts studied in the physiology curriculum. Ideally, this could be done by ensuring that the lecturers in physiology are fully aware of the quantitative demands made by their curricula, the possible difficulties that students could experience, and knowing how to address these difficulties.

Conclusion

References

The results suggest that a good performance in Matric English, the QL of the NBT, and excellent results in life science increase a student’s chances of success in physiology in MB ChB II. This study will be repeated for the following two consecutive years to ensure reliability.

Recommendations

Institutions should investigate different parameters to be used in the selection process of medical students, of which the Matric performance in life science should be one of the factors. This preliminary study indicates the need to rethink the effectiveness of our selection criteria, redesign our selection process for MB ChB, and consider using the NBT. It will be desirable to seek to integrate the QL provision into the students’ experience

1. Mitchell KJ. Traditional predictors of performance in medical school. Acad Med 1990;65(3):149-158. 2. De Silva NR, Pathmeswaran A, de Silva N, et al. Admission to medical schools in Sri Lanka: Predictive validity of selection criteria. Ceylon Med J 2006;51(1):17-21. 3. Ramos SM, Croen L, Haddow S. Predictors of preclinical and clinical performance of minority medical students. J Natl Med Assoc 1986;78(7): 601-607. 4. Reede JY. Predictors of success in medicine. Clin Orthop Relat Res 1999;362:72-77. 5. Lehmann U, Andrews G, Sanders D. Change and Innovation at South African Medical Schools. Durban: Health Systems Trust, 2000. 6. Frith V. Quantitative literacy interventions at University of Cape Town: Effects of separation from academic disciplines. Numeracy 2012;5(1):Article 3. [http://dx.doi.org/10.5038/1936-4660.5.1.3] 7. Foxcroft CD, Stumpf R. Matric: What is to be Done? CHET: Pretoria, 2005. 8. Du Plessis L, Gerber D. Academic preparedness of students – an exploratory study. Journal for Transdisciplinary Research in Southern Africa 2012;8(1):81-94. 9. Montague W, Odds FC. Academic selection criteria and subsequent performance. Med Educ 1990;24(2):151-157. 10. Jones RF, Thomae-Forgues M. Validity of the MCAT in predicting performance in the first two years of medical school. J Med Educ 1984;59(6):455-464. 11. Yeld, N. The National Benchmark Tests Project: Addressing student educational needs in the tertiary education system. In: Hofmeyer J, ed. Recession and Recovery. Cape Town: Institute for Justice and Reconciliation, 2009:76-83.

May 2016, Vol. 8, No. 1 AJHPE

CPD Questionnaire May 2016 True (A) or false (B): The development of a reflective vascular training portfolio: Using a countryspecific infrastructure 1. Training portfolios cannot be used to provide direction and standardisation of educational activities.

Mapping undergraduate exit-level assessment in a medical programme: A blueprint for clinical competence? 12. Multiple-choice questions were the least common format of written assess ments, and short answers the most common.

Randomised controlled trials in educational research: Ontological and epistemological limitations 2. The positivist perspectives of reality involve an attempt to understand pheno mena from an individual’s perspective, and consequently recognise that in certain contexts it is not possible to determine causality.

The Umthombo Youth Development Foundation (UYDF), South Africa: Lessons towards community involvement in health professional education 13. Community involvement was an explicit intention when the UYDF was initiated in 1999.

Standard setting and quality of assessment: A conceptual approach 3. The effect of assessment outcomes extends to political, economic, social and policy domains. Medical students’ views on the use of video technology in the teaching of isiZulu communciation, language skills and cultural competence 4. Teaching communication skills through videos has generally been found to be unsuccessful. Effect of curriculum changes to enhance generic skills proficiency of 1st-year medical students 5. As a result of the interventions introduced, students’ information technology skills, presentation skills and organisational skills practice were enhanced. Perceived stressors of oral hygiene students in the dental environment 6. Emotional exhaustion rather than burnout has been reported by dental students and professionals. Balancing the educational choices in the decision-making of a dean of medicine: Fission or fusion? 7. Leadership was understood to involve the stepping away from the individual patient relationship. Relationship between student preparedness, learning experiences and agency: Perspectives from a South African university 8. Students who have a better learning experience take more responsibility for their own learning. Perceptions of undergraduate dental students at Makerere College of Health Sciences, Kampala, Uganda towards patient record keeping 9. Medical record-keeping clinical skills are a core part of the training of a dental student and should be given the platform they deserve in the curriculum.

Exploring knowledge, perceptions and attitudes about generic medicines among final-year health science students 14. The beliefs of pharmacists and prescribing doctors are examples of factors that contribute to generic medicine use. Home-based rehabilitation: Physiotherapy student and client perspectives 15. Although an interpreter accompanied students on their home visits, the students still felt a barrier in communication with the client, suggesting that a lack of language competency was not the only communication barrier. An exploration into the awareness and perceptions of medical students of the psychosociocultural factors which influence the consultation: Implications for teaching and learning of health professionals 16. Exposure to patients from various cultures, particularly in the rural setting, had raised awareness of the challenges of cultural diversity, with several participants observing behaviours that demonstrated insufficient knowledge of cultural practices. The way forward with dental student communication at the University of the Western Cape, Cape Town, South Africa 17. Clinical teachers agree that dental student-to-patient communication is important and that they have the skills to teach and assess communication skills. An online formative assessment tool to prepare students for summative assessment in physiology 18. Despite <50% of students accessing the additional resources available in feedback on the answer to a multiple-choice question, those who made use of the testing programme achieved significantly better results in their summative tests than those who did not use the programme.

Fieldwork practice for learning: Lessons from occupational therapy students and their supervisors 10. Clinical reasoning improves over time and as students gain experience in the clinical field.

The forensic autopsy as a teaching tool: Attitudes and perceptions of undergraduate medical students at the University of Pretoria, South Africa 19. He et al. (2011) from the Department of Pathology, Shanghai Hospital, Second Military Medical University, China, indicated that the primary objective of the autopsy is to teach anatomy; yet, most students indicated its helpfulness in teaching pathology.

On being agents of change: A qualitative study of elective experiences of medical students at the Faculty of Health Sciences, University of Cape Town, South Africa 11. Students agreed that having extended responsibilities for patient care prevents against erosion of empathy.

Preliminary study: Predictors for success in an important premedical subject at a South African medical school 20. Good performance in Matric English, the quantitative literacy of the National Benchmark Test, and excellent results in life science, increase a student’s chances of success in physiology in MB ChB II.

A maximum of 3 CEUs will be awarded per correctly completed test. The CPD programme for AJHPE is administered by Medical Practice Consulting. CPD questionnaires must be completed online at www.mpconsulting.co.za After submission you can check the answers and print your certificate. Questions may be answered up to 6 months after publication of each issue. Accreditation number: MDB015/175/02/2016

May 2016, Vol. 8, No. 1 AJHPE

AJHPE

African Journal of Health Professions Education | May 2016, Vol. 8, No. 1, Suppl 1

GUEST EDITORIAL

86 Going rural – protracted immersion or toe-wetting: Does it matter? O Nkomazana

RESEARCH

87 Implementation and outcome evaluation of the Medical Education Partnership Initiative biostatistical reasoning workshops for faculty and postgraduate students at the University of KwaZulu-Natal, Durban, South Africa M Muzigaba, M L Thompson, B Sartorius, G Matthews, N Nadesan-Reddy, S Pillay, U Lalloo 92 Predictors of site choice and eventual learning experiences in a decentralised training programme designed to prepare medical students for careers in underserved areas in South Africa M Muzigaba, K Naidoo, A Ross, N Nadesan-Reddy, S Pillay 99 Exploring the relationship between demographic factors, performance and fortitude in a group of diverse 1st-year medical students S Hamid, V S Singaram 104 Motivated strategies for learning and their association with academic performance of a diverse group of 1st-year medical students S Hamid, V S Singaram 108 Assessing the effect of an online HIV/AIDS course on 1st-year pharmacy students’ knowledge F Suleman 113 Feedback as a means to improve clinical competencies: Consultants’ perceptions of the quality of feedback given to registrars C I Bagwandeen, V S Singaram 117 Feedback as a means to improve clinical competencies: Registrars’ perceptions of the quality of feedback provided by consultants in an academic hospital setting C I Bagwandeen, V S Singaram 121 Third-year medical students’ and clinical teachers’ perceptions of formative assessment feedback in the simulated clinical setting R M Abraham, V S Singaram

EDITORIAL BOARD

EDITOR-IN-CHIEF Vanessa Burch University of Cape Town SUPPLEMENT EDITOR Oathokwa Nkomazana University of Botswana INTERNATIONAL ADVISORS Deborah Murdoch-Eaton Sheffield University, UK Michelle McLean Bond University, QL, Australia SENIOR DEPUTY EDITORS Juanita Bezuidenhout Stellenbosch University Jose Frantz University of the Western Cape DEPUTY EDITORS Jacqueline van Wyk University of KwaZulu-Natal Julia Blitz Stellenbosch University Michael Rowe University of the Western Cape Elizabeth Wolvaardt University of Pretoria ASSOCIATE EDITORS Francois Cilliers University of Cape Town Lionel Green-Thompson University of the Witwatersrand Dianne Manning University of Pretoria Sindiswe Mthembu University of the Western Cape Ntombifikile Mtshali University of KwaZulu-Natal Anthea Rhoda University of the Western Cape Ben van Heerden Stellenbosch University Marietjie van Rooyen University of Pretoria Gert van Zyl University of the Free State

HMPG

HMPG BOARD OF DIRECTORS Prof. M Lukhele (Chair), Dr M R Abbas, Dr M J Grootboom, Mrs H Kikaya, Prof. E L Mazwai, Dr M Mbokota, Dr G Wolvaardt ISSN 2078-5127

Guest editorial Going rural – protracted immersion or toe-wetting: Does it matter? Universal access to healthcare mandates that all people worldwide have access to comprehensive healthcare services, without suffering financial hardship.[1] However, unless the severe shortages and inequitable distribution of healthcare workers, especially in many low- and middle-income countries, are addressed,[2,3] universal access will, similar to ‘Health for all by the year 2000’, go down in history as a desirable but unattainable goal. The dearth of healthcare workers follows an ‘inverse care law’,[4] with the direst shortages in areas of greatest need, mostly rural areas.[2,3] In a bid to address the challenge in sub-Saharan Africa, many new medical schools with larger class sizes have sprung up in the past 20 - 30 years.[5,6] There is strong evidence, mostly emanating from the USA, Australia, the Philippines, Thailand and Canada, linking rural-based training of healthcare workers with increased retention.[7-10] Consequently, there has been an increase in curricular innovations to incorporate or strengthen ruraland community-based training in sub-Saharan Africa. Recent investments by the US President’s Emergency Plan For AIDS Relief (PEPFAR) to support medical and nursing institutions in sub-Saharan Africa, through the Medical Education Partnership Initiative (MEPI) and the Nursing Education Partnership Initiative (NEPI), respectively, have added further impetus to curricular innovations aimed at promoting retention of graduates in rural areas and primary care.[11-13] The funding has supported infrastructure development in the form of teaching spaces, hostel accommodation and internet access at rural training sites of a significant number of schools.[14,15] Although most of these community-based training programmes have the same primary goal, i.e. increasing the number of healthcare workers in underserved areas, the programmes vary greatly in duration and frequency of exposure. Rural contact ranges from single blocks of 6 - 8 weeks, through multiple exposures of 4 - 8 weeks, to a 1-year attachment at the Ukwanda Rural Clinical School.[11,13] In this issue of AJHPE, Muzigaba et al.[16] describe a pilot community-based programme lasting 10 days. This variability begs the question: is there a threshold length of exposure that will give the desired outcome, i.e. an increased number of healthcare workers in rural areas? Is the wetting of toes just as effective as protracted immersion? This is a critical question, considering the substantial cost of setting up and running these programmes and the urgent need to increase the number of healthcare workers in rural areas. MEPI and NEPI grants have fostered strong north-south and south-south partnerships, creating opportunities for joint learning and relevant research to optimise the programmes. Muzigaba et al.[16] introduce a very important concern, i.e. that students of rural origin were more likely to choose primary care exposure in urban centres. This raises two key questions: (i) What proportion of students should participate in rural-based training to achieve a meaningful effect on rural retention? (ii) As rural origin and training in rural areas are independent predictors of rural retention, is the University of KwaZuluNatal's programme limiting its potential effect by allowing students to elect not to have rural exposure? This elective v. required rural contact characterises a number of programmes.[11-13] Stellenbosch University (SU) stands out as a trailblazer on the African continent with regard to longitudinal clinical clerkship. Nevertheless, only a small, self-selected proportion of medical students rotate through the Ukwanda Rural Clinical School.[13] Therefore, institutions that may want to emulate SU, need to answer the question: What proportion of students need to participate to realise the desired effect? This is fundamental with regard to cost-effectiveness of

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

interventions in the face of pressing needs and resource constraints. The parallel rural community curriculum of Flinders University, Adelaide, Australia, requires 40% of students to complete 1 year of training in six regions, spanning 3 500 km.[17] However, the Northern Ontario School of Medicine in Canada obliges all its medical students to complete a 1-year comprehensive community clerkship in 12 large communities across the vast region.[17] Although evaluation has demonstrated comparable health systems and educational value of the Australian and Canadian programmes, comparative analysis of their cost-effectiveness will be very instructional. Rural-based training of healthcare workers, especially doctors, is receiving increasing attention in sub-Saharan Africa. This has been largely influenced by evidence from resource-rich countries and south-east Asia, with the belief that it will similarly increase the number of healthcare workers in rural areas. Experience of the Walter Sisulu University, Mthatha has also significantly heightened this expectation of value accrual.[18] Nevertheless, the association between rural exposure and rural retention is unlikely to be simple, as context and other known confounders, such as rural recruitment of learners, are likely to affect the correlation.[8] It is therefore imperative that highquality, methodologically rigorous longitudinal studies be conducted to inform the innovations in rural-based training in sub-Saharan Africa.

Oathokwa Nkomazana

Guest editor Faculty of Medicine, University of Botswana, Gaborone, Botswana nkomazanao@mopipi.ub.bw 1. World Health Organization. Health Systems Financing: The Path to Universal Coverage. Geneva: WHO, 2010. 2. Nkomazana O, Peersman W, Willcox M, Mash R, Phaladze N. Human resources for health in Botswana: The results of in-country database and reports analysis. Afr J Prim Health Care Fam Med 2014;6(1):e1-e8. [http:// dx.doi.org/10.4102/phcfm.v6i1.716] 3. World Health Organization. Working Together for Health. Geneva: WHO, 2006. 4. Hart JT. The inverse care law. Lancet 1971;1(7696):405-412. 5. Derbew M, Animut N, Talib ZM, Mehtsun S, Hamburger EK. Ethiopian medical schools’ rapid scale-up to support the government’s goal of universal coverage. Acad Med 2014;89(8 Suppl):S40-S44. [http://dx.doi. org/10.1097/ACM.0000000000000326] 6. Monekosso GL. A brief history of medical education in sub-Saharan Africa. Acad Med 2014;89(8 Suppl):S11-S15. [http://dx.doi.org/10.1097/ACM.0000000000000355] 7. Wibulpolprasert S, Pengpaibon P. Integrated strategies to tackle the inequitable distribution of doctors in Thailand: Four decades of experience. Hum Resource Health 2003;1(1):12. 8. Ranmuthugala GI, Humphreys J, Solarsh B, et al. Where is the evidence that rural exposure increases uptake of rural medical practice? Aust J Rural Health 2007;15(5):285-288. 9. Dolea C, Stormont L, Braichet J-M. Evaluated strategies to increase attraction and retention of health workers in remote and rural areas. Bull World Health Organ 2010;88(5):379-385. [http://dx.doi.org/10.2471/BLT.09.070607] 10. Rabinowitz HK, Diamond JJ, Markham FW, et al. Medical school programs to increase the rural physician supply: A systematic review and projected impact of widespread replication. Acad Med 2008;83(3):234-243. [http://dx.doi.org/10.1097/ACM.0b013e318163789b] 11. Mariam DH, Sagay AS, Arubaku W, et al. Community-based education programs in Africa: Faculty experience within the Medical Education Partnership Initiative (MEPI) Network. Acad Med 2014;89(8 Suppl):S50-S54. [http://dx.doi.org/10.1097/ACM.0000000000000330] 12. Child MJ, Kiarie JN, Allen SM, et al. Expanding clinical medical training opportunities at the University of Nairobi: Adapting a regional medical education model From the WWAMI program at the University of Washington. Acad Med 2014;89:S35-S39. [http://dx.doi.org/0.1097/ACM.0000000000000350] 13. Van Schalkwyk SC, Bezuidenhout J, Conradie HH, et al. ‘Going rural’: Driving change through a rural medical education innovation. Rural Remote Health 2014;14:2493. 14. Ndhlovu CE, Nathoo K, Borok M, et al. Innovations to enhance the quality of health professions education at the University of Zimbabwe College of Health Sciences – NECTAR Program. Acad Med 2014;89(8 Suppl):S88-S92. [http://dx.doi.org/10.1097/ACM.0000000000000336] 15. Vovides Y, Chale SB, Gadhula R, et al. A systems approach to implementation of eLearning in medical education: Five MEPI schools’ journeys. Acad Med 2014;89(8 Suppl):S102-S106. [http://dx.doi.org/10.1097/ ACM.0000000000000347] 16. Muzigaba M, Naidoo K, Ross A, Nadesan-Reddy N, Pillay S. Predictors of site choice and eventual learning experiences in a decentralised training programme designed to prepare medical students for careers in underserved areas in South Africa. Afr J Health Professions Educ 2016;8(1 Suppl 1):xx-xx. [http://dx.doi.org/10. 7196/AJHPE.v8i1.741] 17. Couper I, Worley PS, Strasser R. Rural longitudinal integrated clerkships: Lessons from two programs on different continents. Rural Remote Health 2011;11(2):1665. 18. Iputo JE. Training doctors from and for rural South African communities. MEDICC Review 2008;10(4):25-29.

Afr J Health Professions Educ 2016;8(1 Suppl 1):86. DOI:10.7196/AJHPE.2016.v8i1.797

Research Implementation and outcome evaluation of the Medical Education Partnership Initiative biostatistical reasoning workshops for faculty and postgraduate students at the University of KwaZulu-Natal, Durban, South Africa M Muzigaba,1 PhD, MPH, MPhil, BSc; M L Thompson,2 PhD, BSc Hons; B Sartorius,3 BSc, BSc Hons, MSc, EPIET, PhD; G Matthews,4 PhD (Mathematical Statistics); N Nadesan-Reddy,3 MB ChB, FCPHM, MMed (Public Health Medicine); S Pillay,1 MB ChB, DOM, Dip HIV Man (SA); U Lalloo,5 MB ChB, FRCP, DOH, PhD Medical Education Partnership Initiative, School of Clinical Medicine, College of Health Sciences, University of KwaZulu-Natal, Durban, South Africa

1 2

Department of Biostatistics, University of Washington, Seattle, USA Discipline of Public Health Medicine, School of Nursing and Public Health, College of Health Sciences, University of KwaZulu-Natal, Durban, South Africa

School of Mathematics, Statistics and Computer Science, College of Agriculture, Engineering and Science, University of KwaZulu-Natal, Durban, South Africa

4 5

Faculty of Health Sciences, Durban University of Technology, South Africa

Corresponding author: M Muzigaba (mochemoseo@gmail.com)

Background. There is a shortage of biostatistics expertise at the University of KwaZulu-Natal (UKZN), Durban, South Africa and in the African region. This constrains the ability to carry out high-quality health research in the region. Objectives. To quantitatively and qualitatively evaluate a programme designed to improve the conceptual and critical understanding of bioÂstatistical concepts of UKZN health researchers. Methods. A 40-hour workshop in biostatistical reasoning was conducted annually between 2012 and 2015. The workshops were structured around interpretation and critical assessment of nine articles from the medical literature, with a mix of in-class sessions and small group discussions. Quantitative evaluation of the knowledge gained from the workshops was carried out using a pre- and post-workshop quiz, and qualitative evaluation of the workshop process was done using a mid-workshop questionnaire and focus group discussions. Results. For each year that the workshop was conducted, post-workshop quiz scores were significantly higher than pre-workshop scores. When quiz assessments from all 4 years of training were combined, the pretest median score was 55% (interquartile range (IQR) 40 - 62%) and the posttest median score was 68% (IQR 62 - 76%), with p<0.0001 for the overall comparison of pre- v. post-scores. There was a general consensus among participants that the workshop improved their reasoning skills in biostatistics. Participants also recognised the value of the workshop in building biostatical capacity at UKZN. Conclusion. The workshops were well received and improved the critical and conceptual understanding of the participants. This education mode offers the opportunity for health researchers to advance their knowledge in settings where there are few professional biostatistician collaborators. Afr J Health Professions Educ 2016;8(1 Suppl 1):87-91. DOI:10.7196/AJHPE.2016.v8i1.740

The University of KwaZulu-Natal (UKZN), Durban, South Africa (SA) was formed in January 2004 as a result of a merger of the former univerÂ sities of Natal and Durban-Westville.[1] It has ~44 000 students, which makes it the largest residential university in SA; 26% of the students are in postgraduate programmes.[2] The university sees its research enterprise as fundamental to the initiative of significantly increasing its output of doctoral graduates. Biostatistics is the application of statistics to questions about human health.[3] Biostatistical considerations inform the design of medical research studies, their analysis and the interpretation of the conclusions. It is an inherently collaborative discipline that is essential in advancing and integrating biomedical, genomic and clinical research. In the KwaZuluNatal region of SA there is a wide and growing range of medical research activities, the majority of which have a biostatistical component. There is, however, a serious shortage of biostatisticians in SA and in the broader African region.[4] The shortage of expertise manifests itself not only in concrete problems

such as difficulty in recruiting suitable biostatisticians for medical research collaboration, but also in less tangible ways affecting quality of research.[5] In an effort to build research capacity in the College of Health Sciences (CHS) at UKZN, a research methodology project (REMETH) was developed by UKZN in partnership with the Medical Education Partnership Initiative (MEPI). The goal of REMETH was to improve the research methodology skills of faculties from the schools of medicine, nursing, pharmacy and pharmacology. The MEPI Biostatistics Initiative was conceived in 2011 to support the development of the discipline of biostatistics at UKZN and to strengthen biostatistics skills among researchers and postgraduate students in the CHS. The initiative arose over several years of stakeholder engagement and is a collaborative effort between the Department of Biostatistics at the University of Washington, Seattle, USA and the UKZN Discipline of Statistics (in the College of Agriculture, Engineering and Science), as well as Disciplines in the CHS, including Public Health Medicine and Occupational and Environmental Health.

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

Research In this article, we describe one component of that strategy, a workshop in biostatistical reasoning, and we present an assessment of the workshop over the 4 years of its implementation. As part of the project description, we present the context and rationale of the project, describe the project structure and delivery mechanism, and outline the implementation process. The outcomes evaluation assessed the actual and perceived knowledge gain in biostatistical reasoning among workshop attendees. The process evaluation focused on the perceptions of the workshop attendees regarding the implementation and value of the workshops.

Project description Context and rationale

The majority of PhD candidates in the UKZN CHS have not had any formal biostatistics training during their under- and postgraduate years. They are reliant on the few biostatisticians in the CHS, who, in addition to being few in number, have many competing demands on their time. This places these candidates at a disadvantage when conducting their research, particularly in critically reading the relevant literature, developing their study design, assessing its validity and generalisability and developing an appropriate statistical analysis plan. While recognising the need for a greater number of professional biostatisticians at UKZN, it was concluded by the Biostatistics Initiative that one approach to advancing research capacity in a setting with limited biostatistics expertise would be to offer workshops in biostatistical reasoning to CHS researchers themselves to improve their understanding of biostatistics concepts. Complementary approaches, also undertaken as part of the initiative, include the development of online asynchronous biostatistics modules, which offer the opportunity for hands-on acquisition of biostatistics analysis skills, biostatistics software tutorials, protocol development workshops and one-on-one consultations with REMETH candidates.

Workshop structure and delivery

As part of the MEPI Biostatistics Initiative, a 40-hour biostatistical reasoning workshop was offered annually to UKZN CHS researchers from 2012 to 2015. This workshop was intended to provide a broad overview of biostatistics methods relevant to the health sciences, emphasising interpretation and concepts rather than computation or mathematical details. Topics covered include data description, study design, sampling variability, statistical inference and regression (linear, logistic, Poisson and Cox). The workshop was built around nine articles from the medical literature. Consequently, the material development was not linear, but addressed topics as they arose in each of the articles, with each article advancing the complexity of the concepts covered. This may be considered a variant of case-based learning, which has been shown to overcome many of the limitations of a traditional lecture-based mode of instruction.[6] After the material necessary to understand a particular article had been covered in class, participants broke into small groups to review the article, with a list of questions aimed at guiding the discussion. Each group then reported back to the class as a whole and there was further discussion. The in-class sessions were designed to be participatory, with ample opportunity for participants to raise questions or discussion points. Upon completion of the workshop, participants should have been able to recognise relevant study design features and explain how they affect interpretation of results, interpret key data displays and statistical results commonly found in medical research reports, and judge whether the conclusions drawn from a study are justified.

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

The workshop learning objectives are outlined below: • Interpret and critique graphical displays of data (e.g. box plots, scatter plots, Kaplan-Meier curves). • Interpret and critique numerical summaries of data. • Translate scientific questions into measurable outcomes and associated statistical goals. • Explain the difference between observational and experimental studies. • Identify and describe the key features of different study designs (e.g. randomised trials, cohort, case-control and cross-sectional studies). • Explain the concept of bias and how a given study design does or does not control for types of bias. • Identify sources of random variation for a given study. • Explain how sample size, variability and effect size interact to determine the power of a study. • Explain the concepts of confounding and effect modification. • Explain the distinction between association and causation. • Explain the key elements of statistical hypothesis testing. • Identify common statistical tests that might be applied to specific research questions. • Explain and interpret p-values and confidence intervals and their implications for the research question under consideration. • Explain the distinction between statistical significance and practical significance. • Identify questions that can be addressed with regression models and interpret regression coefficients in different settings (linear, logistic, Cox proportional hazards). • Identify common abuses of statistical methods in the literature.

Workshop implementation

The workshop was held for the first time in 2012, over 14 successive 3-hour afternoon sessions. It was found that this scheduling made it difficult for clinicians among the participants to attend regularly, as they often had clinics and patients to attend to. Irregular attendance then possibly contributed towards difficulty in grasping all the material. From 2013, the workshop was offered over 8 full days, which improved attendance, but with disadvantages in terms of participant ability to absorb and process the material. The in-class sessions were a few hours’ duration each, with breaks between sessions. After requests from 2012 participants, supplementary exercises, which participants could work on in their own time, were also provided, with further questions relating to interpretation of biostatistics concepts that had been covered in the workshop in-class sessions. The 2012 workshop was conducted by a faculty member (MLT) from the Department of Biostatistics at the University of Washington in Seattle, USA. With a view to sustainability, from 2013 there was increasing participation in facilitating the small group discussions from the three biostatisticians in CHS and two members of the UKZN Discipline of Statistics, and in 2014 and 2015 they participated as co-instructors.

Project evaluation

An evaluation was conducted to assess whether participants’ understanding of biostatistical concepts improved following the training. This also explored participant expectations of the workshop before attending and the extent to which these expectations had been met throughout the workshop. Of additional interest were the participants’ perceptions of the strengths and limitations of the workshop, the perceived effectiveness of the workshop in increasing know

Research

Methods

Evaluation components

The objective of the evaluation was to assess the knowledge gained as well as workshop process and included both quantitative and qualita tive components: (i) quantitative pre- and postworkshop assessments; (ii) a brief qualitative midworkshop questionnaire; and (iii) focus group discussions (FGDs).

Population and sampling

The target population comprised current and future health researchers at UKZN. For the quantitative evaluation of post- v. preworkshop knowledge gain, and the mid-workshop quali tative evaluation, the sample comprised all 2012 - 2015 workshop participants. These participants were regarded as representative of current and future UKZN health researchers who require skills in biostatistical reasoning. Participants for the FGDs were randomly selected from the 2014 and 2015 workshop participants. Each FGD comprised seven to eight participants and, in total, three FGDs were conducted.

Data collection

The same knowledge assessment quiz was used with all available participants in each annual cohort, before and after the workshop, to quantify participants’ change in knowledge following the training. Box 1 provides an example of one of the questions used for pre- and post-test assessments. The anonymous mid-workshop questionnaire enquired about the pace of the workshop and participants were asked to describe in one sentence something that they really liked about the workshop and to make a constructive suggestion to improve it. FGDs were conducted by a single interviewer (MM), during which information about the objectives of the FGD and the overall study was provided. Each participant was asked to provide informed consent to participate and to be recorded using a tape recorder. A structured discussion guide was used to facilitate the FGDs. Saturation was allowed to be reached during the three FGDs conducted.

Data analysis

Quantitative data were cleaned and analysed using Microsoft Excel and Stata version 13 (Statacorp LP,

USA). The distribution of pre- and post-scores by cohort was graphically assessed using box plots, and the change in knowledge based on the preand post-test scores was assessed for each year of training using the Wilcoxon signed-rank test. The raw audio data from FGDs were first carefully reviewed and then transcribed verbatim into Microsoft Word in English, the language of the interview. The audio transcriber (MM) was familiar with the theoretical perspectives of the study and was able to ensure that these were reflected in the approach to transcriptions, which was interpretative, to ensure that the views and representations of the participants in the FGDs were fully conveyed.[7] The Framework Analysis Technique[8] was then used to analyse the transcribed data. This technique was chosen as it has been shown to preserve the integrity of individual responses throughout the analytical process, thereby providing a platform for reconsidering and reworking of ideas where more clarity is needed.

To ensure trustworthiness of the qualitative data, respondent validation (cross-checking interim findings) was conducted by means of reflection to ensure that information reported by participants had been accurately understood. The data were collected and transcribed by one interviewer (MM), thereby minimising inter-investigator bias. The same interviewer also carried out the coding and analysis to ensure internal consistency. Furthermore, a peer-review process was undertaken whereby a fellow senior researcher reviewed the steps taken to analyse and interpret data as a way of improving the inter-rater reliability of the study findings.[9]

Results

Quantitative findings

The number of workshop participants in each year was 20, 17, 22 and 19 for 2012 - 2015, respec-

Researchers obtained birth weights for a random sample of 1 500 infants. The mean birth weight was 3 250 g and the standard deviation was 550 g. The 95% confidence interval (CI) for the mean was 3 221 - 3 278 g. Which one of the following statements best describes the information given by the 95% CI? (i) 95% of babies in the sample had a birth weight between 3 221 and 3 278 g (ii) There was a probability of 0.95 that the sample mean birth weight would be between 3 221 and 3 278 g (iii) There was a probability of 0.95 that the 95% CI will contain the mean birth weight for infants in this population (iv) 95% of babies in this population had a birth weight between 3 221 and 3 278 g. Box 1. Sample quiz question. 2012 (n=18)

2013 (n=13)

2014 (n=21)

2015 (n=14)

Wilcoxon signed-rank test, p=0.016

Wilcoxon signed-rank test, p=0.007

Wilcoxon signed-rank test, p=0.001

80 70 60 Quiz score, %

ledge and skills in biostatistics, and the value of the workshop in increasing biostatistics capacity at faculty and postgraduate levels. The evaluation was approved by the UKZN Biomedical Research Ethics Committee (Ethics Ref. No. BE035/15).

50 40 30 20 10 0

Pretest

Post-test

Fig. 1. Box plots of the participants’ quiz scores before and after the workshop.

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

Research tively. In total, 78 postgraduate students and faculty members were trained over this period, the majority of whom were from the School of Clinical Medicine (33%), with the remainder from the School of Nursing and Public Health (27%), Laboratory Medicine and Medical Sciences (13%) and other disciplines in Health Sciences (13%). The pre- and post-workshop assessment achieved a response rate of 84.6% (n=66). The 15.4% non-respondents included participants who were not available at either time of testing as well as those who only took part in one test and not the other. Fig. 1 shows, for each year, box plots of the participants’ quiz scores (%) before and after the workshop and whether there was a statistically significant change in score, pre- v. post-workshop. The median post-test scores were higher than the median pretest scores across all 4 years of training; these differences were statistically significant, based on the Wilcoxon signed-rank test. When data from all 4 years of training were combined, the results were as follows: pretest median score 55%, interquartile range (IQR) 40 - 62%; post-test median score 68%, IQR 62 - 76%; and p<0.0001, for the pre- v. post-workshop comparison.

Qualitative findings

Feedback on the mid-workshop questionnaire regarding the workshop process was generally positive. With very few exceptions, participants found the pace ‘about right’. Comments regarding aspects of the workshop that participants really liked included ‘good technique re. learning concepts rather than calculations’, and ‘the fact that it emphasises intuitive understanding of concepts’. Comments regarding possible improvement often included requests for consideration of the biostatistics regarding participants’ own research studies.

Prior expectations v. actual satisfaction

Participants attended with many expectations about the workshop and it was encouraging to learn from the FGD analysis that the majority were satisfied with the material delivered: ‘Generally the pre-course expectations were quite high … but I felt the course has met those expectations in terms of practically understanding biostatistics. It has taken our interpretation of medical literature to another level.’ ‘ We are not exposed to basic [statistics] stats on a regular basis. Our training too hasn’t equipped us with this knowledge. Our knowledge is limited by what we read. Coming here, my expectation was to attain the skills to do equations and how to do stats and I have achieved more than what I have expected.’ ‘ I have come here to get to know how to do statistical testing for my research purposes. But what we got was more than that, which increased my understanding … .’

Perceptions about course delivery

Participants were also particularly impressed with the way the course was delivered and the approach used by the workshop facilitator: ‘When I first did stats 30 years ago it was manually done. I had various attempts to gain this knowledge and this is the first time we had a person like [name of facilitator] who made it easy to understand the concepts … the effectiveness of this method of teaching made us understand the concepts. That is what we found as beneficial … .’

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

‘ So she has a nice way of breaking it down to useable chunks, so that you can get a good grasp. She takes you from where you are and your level and builds on that potential.’ ‘ The way she brings the message home she is a very good teacher. We had people here like he said … people who are intelligent don’t know how to transfer knowledge to other people [and] that becomes a problem. You ask her a question and she will be able to bring it down to the level of your understanding. That makes it wonderful.’

Perceived improvement in the subject matter

The workshop also boosted the participants’ ability to engage with the literature. The general sense from those who took part in FGDs was that the biostatistical aspect of the literature had been a no-go area and they were grateful for the opportunity to learn how to critically interpret the statistics reported in biomedical and public health literature: ‘It has broadened our understanding because I didn’t have a background in statistics, for example when reading articles I would just rely on the discussion to understand but now I have a clue on how to interpret it.’ ‘ I often skip over the stats-based concepts and proceed to the discussion, not knowing whether the study was reliable and valid. This has given me the tools to understand study design, the terminology and the concepts that we use.’ ‘ … personally I also feel the same way, my knowledge has gone up by 50 to 60 percent … .’

Value of the workshop

Participants also recommended that the workshop continues as it holds the promise of narrowing the skills gap in biostatistics in the school: ‘Personally I think it’s a course that should continue, I think it actually is increasing our appetites in terms of how we look at papers, and the advice it gives to students is incredible … .’ ‘ I think most of us are teachers and supervisors so when we are supervising we can also consider this as a master trainer kind of course, where this information is just not going to remain among the participants. We are supervising other students at different levels, so obviously the level and intensity of supervision will also improve as opposed to sending the student to the biostatistician to be assisted.’ ‘ … we are just Masters students, it is a good course and maybe more of the students should have an opportunity to be a part of this, because we don’t know anything about statistics so we rely on the supervisor who will refer us to someone else.’ Nevertheless, some respondents felt that the workshop was too compressed and could benefit from a more extended timetable to accommodate busy candidates: ‘I think this is a valuable course. However, I think [in] its current form, where it is compressed over 2 weeks, we have difficulty in keeping the commitment of being here the whole day over the 2 weeks. It’s going to be difficult. Also this is intensive information which you need some time to process and to practise. They have provided all the building blocks,

Research they provide us with articles to read, they ask us questions to ensure we understand the concepts being taught, and they also provide us with supplementary exercises to go over. All of these resources are very helpful but at the same time it is very tiring. I think it needs to be changed from the current format and perhaps done over a longer period of time.’

Discussion

This article was written to share a possible model for improving biostatistical reasoning capacity among health researchers in academic institutions with a shortage of biostatistical expertise, and to demonstrate some of the lessons learned and the outcomes realised during a 4-year implementation period. There are unique needs for health research in the African region, but the ability to carry out this research is limited by the shortage of biostatistical expertise.[4] Consequently, Africa is often a source of data collection, but studies are designed and analysed elsewhere.[4] The UKZN Biostatistics Initiative envisaged a range of strategies to address this problem, one of which was a workshop in biostatistical reasoning for UKZN health researchers. The intention of the workshop was to improve the ability of participants to critically interpret the biostatistical components of the literature in their field and for them to be better able to assess the reports and proposals that they are responsible for reviewing. It was also hoped that the knowledge gained by individual participants would be shared with others within their own disciplines. A case-discussion approach to teaching biostatistics has been found to be successful with medical students in terms of both learning and student evaluation,[10] which is consistent with our experience with these workshops. The workshops were well received and there was both quantitative and qualitative evidence of participants achieving some level of learning. The evaluation of pre- and post-test scores demonstrated quantitative improvement in understanding, and in the FGDs the participants indicated that they felt that they had indeed gained knowledge. The opportunity for small group discussions and in-class participation was considered by both instructors and participants to be an important component towards this success. This type of learning platform has been shown to lead to positive perceptions about knowledge gain.[11] Some education researchers have advanced a view that to maximise learning outcomes, there is a need to consider group composition, to ensure homogeneity of expertise and seniority.[12] In this project, there was considerable heterogeneity in the seniority and experience of participants. The senior faculty among the participants often raised insightful points for discussion in class, arising from their own experience in their area of specialisation. The challenge for the facilitators was then to address these questions in a way that made them relevant for the broader workshop audience. Another challenge for the facilitators was the diverse backgrounds of the participants, ranging from junior researchers, working on Master’s-level projects, to heads of departments. However, it was felt that this diversity allowed the opportunity for improvement in biostatistical understanding throughout the spectrum of CHS researchers and hence possibly longer-term effect. These pedagogic challenges are most readily met by biostatisticians with a depth of experience in the field, which, of course, presents a ‘Catch-22’ challenge in a setting where this expertise is uncommon and, when present, often newly acquired. Leading a workshop such as this is very different from teaching a more conventional biostatistics

course and, as has been observed in other comparable settings, the choice of instructors is crucial to its success.[13] It is noteworthy that the biostatistical understanding of participants in the 2012 - 2015 workshops was typically poor to modest before the workshops. This is not surprising, as the dearth of biostatistical expertise in the subSaharan region has been widely recognised,[4,14] but it confirms the need for educational initiatives such as this. While the workshops increased quiz scores by an average of 13% overall, the understanding of many participants, post workshop, was still modest. Scheduling the workshop over 8 full days improved participation, but was not ideal from a knowledge-processing perspective. This constraint is reflected in some of the comments from the FGDs. This study does not assess to what extent the knowledge of biostatistical concepts was retained in the longer term. Refresher sessions might be one way of maintaining and improving knowledge. It is further hoped that the knowledge gained from the workshops would better enable participants to engage with the hands-on online material which has also been developed as part of the Biostatistics Initiative.

Conclusion

While the workshops were successful, there were limitations. The full-day condensed format, while improving attendance, was not ideal for learning. The concepts covered were increasingly complex and a gestation time to process the ideas would have been preferable. There is clearly a need for UKZN health researchers to also develop some analytic biostatistics skills, given the local shortage of professional biostatisticians for collaboration. Acknowledgement. This work was made possible by grant No. 5R24TW008863 from the US President’s Emergency Plan for AIDS Relief (PEPFAR), and the National Institutes of Health, US Department of Health and Human Services. Its contents are solely the responsibility of the UKZN MEPI programme and do not necessarily represent the official views of the US government.

References 1. University of KwaZulu-Natal. The History. University of KwaZulu-Natal, 2016. http://www.ukzn.ac.za/aboutukzn/history (accessed 29 January 2016). 2. University of KwaZulu-Natal. The UKZN Institutional Intelligence Report. University of KwaZulu-Natal, 2016. https://ii.ukzn.ac.za (accessed 11 February 2016). 3. Winner L. Introduction to Biostatistics. Department of Statistics, University of Florida, 2004. http://www.stat.ufl. edu/~winner/sta6934/st4170_int.pdf (accessed 6 February2016). 4. Gezmu M, DeGruttola V, Dixon D, et al. Strengthening biostatistics resources in sub-Saharan Africa: Research collaborations through US partnerships. Stat Med 2011;30(7):695-708. [http://dx.doi.org/10.1002/sim.4144] 5. Mandala WL, Cowan FM, Lalloo DG. Southern Africa consortium for research excellence (SACORE): Successes and challenges. Lancet Glob Health 2014;2(12):e691-e692. [http://dx.doi.org/10.1016/s2214-109x(14)70321-3] 6. Bolt B. Encouraging cognitive growth through case discussions. J Teach Phys Educ 1998;18(1):90-102. 7. Chandler CIR, Reynolds J. ACT Consortium Guidance: Qualitative Research Protocol Template with Example Tools and SOPS. London: London School of Hygiene and Tropical Medicine, 2013:21-26. 8. Bryman A, Burgess R. Qualitative data analysis for applied policy research. In: Analyzing Qualitative Data. New York: Routledge, 1994:173-194. 9. Daly J, McDonald I, Willis E. Why don’t you ask them? A qualitative research framework for investigating the diagnosis of cardiac normality. In: Daly J, McDonald I, Willis E, eds. Researching Health Care: Designs, Dilemmas, and Disciplines. London: Routledge, 1992:189206. 10. Marantz P, Burton W, Steiner-Grossman P. Using the case‐discussion method to teach epidemiology and biostatistics. Acad Med 2003;78(4):365-371. [http://dx.doi.org/10.1097/00001888-200304000-00008] 11. Flosason TO. Evaluating the impact of small-group discussion on learning in an organizational psychology class utilizing a classroom response system. Dissertations 2011; Paper 406. http://scholarworks.wmich.edu/ dissertations/406 (accessed 28 March 2016). 12. Bennett J, Lubben F, Hogarth S, Campbell B. A systematic review of the use of small-group discussions in science teaching with students aged 11-18, and their effects on students’ understanding in science or attitude to science. In: Research Evidence in Education Library. London: EPPI Centre, University of London, 2004. 13. Ambrosius WT, Manatunga AK. Intensive short courses in biostatistics for fellows and physicians. Stat Med 2002;21:2739-2756. [http://dx.doi.org/10.1002/sim.1212] 14. Fegan G, Moulsdale M, Todd J. The potential of internet-based technologies for sharing data of public health importance. Bull World Health Organ 2011;89(2):82. [http://dx.doi.org/10.2471/blt.11.085910]

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

Research Predictors of site choice and eventual learning experiences in a decentralised training programme designed to prepare medical students for careers in underserved areas in South Africa M Muzigaba,1 PhD, MPH, MPhil, BSc; K Naidoo,2 MB ChB, MCFP, MFamMed; A Ross,2 MB ChB, DCH, MMed (Family Medicine), FCFP (SA); N Nadesan-Reddy,3 MB ChB, FCPHM, MMed (Public Health Medicine); S Pillay,1 MB ChB, DOM, Dip HIV Man (SA) Medical Education Partnership Initiative, School of Clinical Medicine, College of Health Sciences, University of KwaZulu-Natal, Durban, South Africa

Department of Family Medicine, School of Clinical Medicine, College of Health Sciences, University of KwaZulu-Natal, Durban, South Africa

Discipline of Public Health Medicine, School of Nursing and Public Health, College of Health Sciences, University of KwaZulu-Natal, Durban, South Africa

Corresponding author: M Muzigaba (mochemoseo@gmail.com)

Background. There is a dire need for medical schools in South Africa to train medical doctors who have the capacity and willingness to work in primary healthcare facilities, particularly in rural areas. Objectives. To assess the effect of students’ gender, race, place of birth and place of high school completion on their choice of training site location and to assess the extent to which the training programme enhanced students’ learning experiences relevant to primary care across training sites. Methods. A survey design involving six cohorts of 4th-year undergraduate medical students (N=187) who were part of the 2013 Family Medicine rotation at the Nelson R Mandela School of Medicine. Self-administered questionnaires were completed by students at the end of each rotation. Data analyses involved descriptive computations and inferential statistical tests, including non-parametric tests for group comparison and generalised polynomial logistic regression. Results. Students believed that their knowledge and skills relevant to primary care increased after the rotation (p<0.0001). There were statistically significant differences between rural and urban sites on certain measures of perceived programme effectiveness. Male students were less likely to choose urban sites. Black students were less likely to choose rural sites compared with their white and Indian counterparts, as were students who attended rural high schools (odds ratio (OR) 9.3; p<0.001). Students from a rural upbringing were also less likely to choose rural sites (OR 14; p<0.001). Conclusion. Based on the findings, an objective approach for student allocation that considers students’ background and individual-level characteristics is recommended to maximise learning experiences. Afr J Health Professions Educ 2016;8(1 Suppl 1):92-98. DOI:10.7196/AJHPE.2016.v8i1.741

There is a maldistribution of the health workforce in favour of urban areas, with fewer medical doctors practising in geographical areas where the need is greatest in South Africa (SA).[1] Research commissioned by the SA Department of Labour in 2008 showed that the health workforce shortages are not only geographically defined.[2] In addition to rural and urban disparities, there are recruitment and retention difficulties in the public sector compared with the private sector. Over the past decade, Canada, Australia and the USA have also reported disparities between metropolitan and non-metropolitan areas, despite an increase in the number of medical graduates per capita.[3,4] The shortage of doctors in rural areas has been linked to poor treatment outcomes for diseases such as HIV/AIDS and other complicated illnesses, when compared with urban areas in SA.[2] The incorporation of primary care or family medicine in the undergraduate medical curriculum and decentralising training outside of large academic complexes are two of 10 evidence-based undergraduate interventions demonstrated to influence medical graduates’ decisions to work in rural areas.[5] This ‘distributed’ approach has become more popular in medical universities with a drive to promote rural health. Students spend some time in decentralised training sites with a view to enhancing their experiential learning and increase their chances of eventual career paths and retention in these areas.[6,7] There is also some evidence to show that students from rural origins are more likely to pursue their medical career in rural settings after graduation, compared with those from non-rural origins.[8]

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

To maximise the outcomes of such intervention, particularly those related to experiential learning and desire for a future career in rural settings, it is important to understand what factors influence the choice for training site location, as this may dictate whether an objective approach for student allocation may be necessary. In this study, we evaluated a decentralised training programme which was piloted at the University of KwaZulu-Natal (UKZN) in 2013 on 4th-year undergraduate medical students as part of their Family Medicine block. The primary hypothesis was that students’ choice for site location was determined by their gender, race, place where they were raised and where they completed high school. Secondly, we tested the hypothesis that learning experiences differed among students depending on the ‘rurality’ of the site where they completed their rotation. This formative evaluation was partly conducted to share lessons learned with the medical education community and to generate information to inform the refinement of this pilot intervention.

The UKZN Decentralised Training Programme

The UKZN Decentralised Training Programme was piloted in 2013 in SA through a collaboration between UKZN, the Medical Education Partnership Initiative (MEPI) and the KwaZulu-Natal (KZN) Department of Health. As part of the programme implementation plan, students were allowed to self-select in groups of four to six. The resultant groups of students then chose one of six primary healthcare facilities across KZN, where they were attached for 12 conse

Research cutive days. Two of the facilities were rural, two peri-urban and two urban. For this programme, an urban area was defined as a geographical location where there is a high population density and economic functioning, such as the City of Durban. A rural area was defined as a geographical location with very low population density and little to no economic activity. A peri-urban area was considered to be an area between consolidated urban and rural regions. The students received one week of teaching and skills training prior to the clinical attachment. In the short term, the programme sought to enhance student learning experiences at each of the training sites. The mediumterm outcome following directly from this would have been an increased propensity or willingness to return to such areas after undergraduate training.

the need to maximise the response rate and to obtain accurate and relevant information for the survey. Questions were constructed using short and simple sentences asking one piece of information at a time. With the study being exclusively quantitative, only closed questions were used. This allowed for specific information on dependent and independent variables of interest to be gathered. The independent variables of interest were categorical in nature and included studentsâ&#x20AC;&#x2122; gender, race, geographical areas that best describe where they completed high school and were born, and geographical location of the site where they completed their rotation. The questionnaire also contained questions that were developed against specific learning objectives set out in the primary care/family medicine curriculum. These constituted the dependent variables and were formulated using a 5-point Likert scale ranging from 1 (no knowledge or skills) to 5 (no need for supervision). The scale was used to rate each learning objective before and after the rotation, as a subjective measure of the change in knowledge/skills on that learning objective. A number of experiential learning variables on the evaluation of clinical placement programmes which are commonly cited in the literature[6,7] were also included in the questionnaire. Students used a 5-point Likert scale to rate their experiences during the rotation period, based on these variables. To maximise accuracy of the information gathered, negative questions were avoided as far as possible and questions

Methods Design

This study involved cross-sectional surveys with six successive cohorts of 4th-year medical students who benefited from the MEPI-UKZN Decentralised Training Programme in 2013. Of a total of 187 4th-year students who completed the rotation, 183 consented to participate, giving a response rate of 97%. At the end of each rotation, students were given a structured quantitative questionnaire to complete. The questionnaire was developed mindful of

Table 1. Distribution of students (n) across cohorts by their demographic characteristics and type of DTC location they selected Variable

Cohort 1

Cohort 2

Cohort 3

Cohort 4

Cohort 5

Cohort 6

Total

Male

Female

109

Total

181

White

Black

Indian

Coloured

Total

183

Rural

102

Urban

Peri-urban

Total

178

Rural

Urban

109

Peri-urban

Total

183

Rural

Urban

105

Peri-urban

Total

183

Pearsonâ&#x20AC;&#x2122;s Ď&#x2021;2

p-value

14.28

0.501

11.47

0.721

15.74

0.112

9.34

0.500

5.14

0.881

Sex

Ethnicity

Type of DTC location

Area where student completed high school

Area where student was born

DTC = decentralised training centre.

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

Research that were likely to induce bias in responses were minimised. Imprecise questions were avoided to ensure that there were no differences in how respondents understood the questions.

Statistical analysis

The survey data were first entered into Microsoft Excel 2010, then imported into Stata version 13.0 (StataCorp, USA)[9] for cleaning and analyses. Descriptive analyses involved computation of summary statistics and graphical presentations based on students’ background characteristics and other survey responses. Pearson’s χ2 test with measures of associations was used to compare student cohorts by gender, race and the decentralised training centres (DTCs) where they completed their rotation. The data were not normally distributed and therefore non-parametric tests were used for between- and within-group comparisons based on participants’ responses. To compare the ‘before’ and ‘after’ perceived knowledge and skills scores, Wilcoxon’s signed-rank test was used. The Kruskal-Wallis test was used to compare different areas of attachment by the perceived quality of the programme delivery and perceptions about the programme in general. To assess the relationship between students’ background characteristics and their choice of location of the DTC where they completed their rotation, a generalised polynomial logistic regression model was used. A p-value of <0.05 was considered statistically significant.

Ethics approval

The study was approved by the UKZN Biomedical Research Ethics Committee (Ref. No. BE046/13) before the study commenced.

Results

Of the 183 students surveyed in all six cohorts, 180 satisfactorily completed the questionnaire − a completion rate of 98.3%. Sixty percent of these students were female. Regarding racial distribution, 79 (43%) were Indian, 79 (43%) were black, 20 (11.5%) were white and 4 (2.5%) were coloured. Sixty percent of the students completed their clinical rotation in rural DTCs while the rest were attached to peri-urban or urban DTCs in almost equal proportions. More black students (44.30%) had a rural upbringing than an urban or peri-urban upbringing (29.11% and 26.58%, respectively). Students who attended rural high schools and those who attended urban high schools were equal in proportion (39.24%), with the rest (21.52%) attending high school in peri-urban areas.

There were no statistically significant differences between cohorts in terms of sex, race, and chosen DTC, or in geographical area of high school completion and upbringing (Table 1). Similarly, the differences between cohorts in terms of choice for site location were not statistically significant. Students felt that their knowledge of and skills in

a variety of subject matters increased following exposure to the clinical rotation. All perceived changes were statistically significant (p<0.001). On a Likert scale of 1 - 5, the median ratings increased from 3 (some knowledge/skills) to 4 (good knowledge/skills but need supervision/ support). The most notable perceived change

Table 2. Students’ ratings of their perceived change in knowledge/skills (Wilcoxon signed-rank test) Statistic Knowledge and skills variable

Median

z-value

p-value*

−9.87

<0.001

−10.59

<0.001

−10.24

<0.001

−10.33

<0.001

−10.97

<0.001

−10.44

<0.001

−10.79

<0.001

−10.52

<0.001

−8.43

<0.001

−10.75

<0.001

−10.50

<0.001

Effective communication skills in carrying out a patientcentred interview Before

179 3.00

After

183 4.00

Understanding context of the patient Before

179 3.00

After

183 4.00

Describing the indication and risks with common investigations and procedures Before

177 3.00

After

180 4.00

Clinical problem-solving skills Before

177 3.00

After

181 4.00

Formulate a three-stage assessment and management plan Before

177 3.00

After

181 4.00

Manage undifferentiated problems Before

177 2.00

After

181 4.00

Manage common chronic illnesses Before

178 3.00

After

181 4.00

Clinical record keeping Before

179 3.00

After

183 4.00

Ethical issues in clinical practice such as confidentiality, consent and patient autonomy Before

179 3.00

After

182 4.00

Health promotion and disease prevention Before

179 3.00

After

184 4.00

Liaising with other members of the healthcare team Before

176 3.00

After

181 4.00

*Asymptotic significance (two-tailed).

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

Research in knowledge/skills was observed in the management of undifferentiated problems where the median rating increased from 2 (vague knowledge/skills) to 4 (good knowledge/skills but need supervision/support) (Table 2). Further post-hoc analyses conducted to com pare rural, urban and peri-urban sites revealed that there were no site-specific differences in all 11 items for perceived change. Statistically significant differences between rural, urban and peri-urban DTCs were only observed in relation to students’ ratings of their relationship with the clinic staff (p<0.001), their access to materials during the rotation (p<0.05) and the clinical skills/knowledge they gained (p<0.05). Shown in Table 3, students who were in urban DTCs reported, for the most part, better quality of programme delivery than those in rural and peri-urban DTCs. Ratings of other measures such as the relationship with their supervisors, quality of supervision and

teaching and the overall content of the training were again better in urban DTCs than in other DTC locations, but these differences were not statistically significant (Table 3). Despite the DTC-specific differences reported here, the median perception ratings were encouraging overall. They ranged between 3 (good) and 5 (excellent) across the board on a Likert scale of 1 (poor) to 5 (excellent). The perceived effect of the decentralised training programme was, for the most part, also rated higher by students who completed their rotation in urban DTCs (Table 4). However, these differences were only statistically significant for four subjective indicators of the programme’s effectiveness. For example, although students in urban DTCs reported to be much more motivated to continue with their medical career following their experience at the urban DTCs, this was not statistically different from that reported by students in rural and peri-urban DTCs (p>0.05).

Table 3. Students’ rating of the quality of programme by DTC locations (Kruskal-Wallis test) Statistic Variable

Mean rank

Median

p-value

4.17

3.00

0.153

5.88

3.00

0.042

2.03

3.00

0.391

Overall content of the training Rural

101

91.08

Urban

92.97

Peri-urban

73.00

Clinical skills/knowledge gained Rural

102

90.79

Urban

95.88

Peri-urban

70.59

Rural

102

86.25

Urban

98.00

Peri-urban

84.09

Rural

102

89.93

Urban

103.25

Peri-urban

66.87

Quality of supervision and teaching

Discussion

Relationship with clinic staff 12.31

4.00

0.000

7.88

3.00

0.021

2.82

4.00

0.241

Access to necessary materials Rural

100

84.53

Urban

105.19

Peri-urban

75.44

Relationship with the rotation supervisor Rural

102

84.92

Urban

99.04

Peri-urban

86.84

df = degrees of freedom.

The same result was found for the perceived effect of the programme in easing students’ transition to the clinical environment. However, it was encouraging to learn that on a scale of 1 - 5, the median ratings ranged between 4 and 5 across all indicators of programme effectiveness. Table 5 shows results from both the univariable and multivariable analy ses of four predictors of site choice. At univariable level, most odds ratio (OR) estimates were statistically significant. The model predicted that male students were less likely to choose urban DTCs than their female classmates. With regard to ethnicity, black students were more likely to choose urban DTCs than their white and Indian classmates. The students who attended and completed their high school in rural areas were nine times less likely to select a rural DTC for their rotation compared with those who completed high school in urban and peri-urban areas. The same direction of predictions was also observed for students who were born and raised in rural areas. These students were 14 times less likely to choose the DTCs located in rural areas. Although the direction of ORs remained similar across all the four predictors after multivariable analyses, the ORs became smaller. Some predictors also became statistically insignificant. For example, only sex and race remained statistically significant predictors of site choice, with males more likely to choose rural DTCs than their female counterparts, and black students less likely to choose rural DTCs than their white and Indian classmates. Students’ place of birth and high school completion were not statistically significant predictors of site choice after model adjustment.

This pilot study was conducted with a view to documenting some of the factors that influ ence undergraduate medical students’ choice of site location within the context of the UKZN decentralised training programme. The objective of the study was also to find out whether the site location determined students’ perceptions about the quality of the programme, as well as their perceived learning experience. The purpose of the study was to generate information for future use by the programme developers to craft an intervention that is more context-sensitive and likely to be well received by the actual end users. The programme structure and mode of delivery were, in principle, well aligned with

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

Research

Table 4. Students’ rating of the quality of programme effectiveness by DTC locations (Kruskal-Wallis test) Statistic n

Variable

Mean rank

Median

p-value

3.88

5.00

0.212

2.55

5.00

0.382

6.75

5.00

0.031

10.37

5.00

<0.001

9.05

4.00

0.025

3.65

4.00

0.213

9.48

4.00

0.022

Will ease my transition to the clinical environment Rural

101

90.50

Urban

93.47

Peri-urban

74.69

Rural

101

87.78

Urban

97.01

Peri-urban

80.84

Motivated me to continue with the medical career

Orientated me towards the social context of practice Rural

101

92.57

Urban

89.83

Peri-urban

69.76

Rural

101

91.02

Urban

100.26

Peri-urban

67.77

Rural

101

87.87

Urban

104.26

Peri-urban

72.30

Made me more confident to approach patients

Made me more aware of myself and others on the team

Strengthened my theoretical knowledge Rural

101

90.13

Urban

94.14

Peri-urban

75.01

Rural

100

88.97

Urban

100.82

Peri-urban

68.46

Introduced me to the organisation of the healthcare system and role of various professionals within it

novel strategies in medical education,[5-7] which are geared towards enhancing experiential learn ing among undergraduate medical students during their early years of medical training. Indeed, there is ongoing discourse around how training in a variety of clinical teaching facilities correlates with eventual practice locations.[10] Some proponents of this approach continue to argue that it results in students entering permanent practice in a location similar to the one in which they trained.[11] This study adds a

slightly different but complementary dimension to what already exists in the literature. As the study participants were only recruited in their 4th year of undergraduate medical training, this study was not able examine the effect of the decentralised training programme on eventual practice location. A follow-up study to examine their career paths and practice location after graduation and whether these are influenced by the individual-level characteristics assessed in this study, would be useful. However,

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

our findings begin to show some intricacies of allowing students to self-select in a model that involves one programme and multiple training sites which vary in resources, infrastructure and location. We advance a view that medical educationists with an interest in the distributed approach ought to be more objective in how they allocate students to training sites away from the traditional medical schools to place the ‘right’ candidate in the ‘right’ training location. The study was able to show that students with rural upbringing and rural high school education were less likely than those from urban areas to select rural training sites as part of their family medicine rotation, when these two predictors were considered independently. When combined with sex and race in a multivariable model, the two predictors produced similar results, but with smaller ORs. Despite the lack of statistical significance in the multivariable model, the direction of predictions reported here should not be overlooked. The univariable analyses tell a story that ought to be explored further in much larger studies within the SA context. A much larger study in SA may use a mixed-effects or multilevel modelling approach to assess the potential effects of additional upstream factors, such as the programme type and name of academic institution, on site choice. This would address the possible influences of curriculum differences within institutions, as well as the kind of students trained at each institution. The same approach can also be used within the context of an international study, by taking into account the country in which the programme was implemented. A large-scale study is therefore recommended, as it is more likely to be able to answer the question around the generalisability of the findings reported here. There is ongoing debate elsewhere that if rural students are selected to enter medical training programmes and are provided with both under- and postgraduate training opportunities in rural areas, they are more likely than those selected from urban areas to return to such areas for medical practice.[12] This has been referred to as the ‘pipeline approach’ to rural physician resources.[13,14] The disinclination of students with a rural background to select rural sites as part of their decentralised training begs the question as to whether a targeted approach, with sites selected for students based on student characteristics, may more objectively ensure that the pipeline approach works more effectively. A qualitative study would be valuable to further unpack the

Research

Table 5. Relationship between studentsâ&#x20AC;&#x2122; background characteristics and their choice of DTCs using a polynomial logistic regression model Univariable analysis Predictors

95% CI

p-value

Location of the DTC

Rural

Male v. female

Urban

0.29

0.12 - 0.72

0.0076

Male v. female

Peri-urban

1.57

0.73 - 3.41

0.2506

Multivariable analysis 95% CI

p-value

0.166

0.05 - 0.53

0.002

1.337

0.54 - 3.28

0.525

Sex *

Race Rural

White v. black

Urban

0.04

0.01 - 0.33

0.0027

* 0.047

0.01 - 0.41

0.006

White v. black

Peri-urban

0.22

0.05 - 0.87

0.0315

0.154

0.03 - 0.78

0.024

Indian v. black

Urban

0.03

0.01 - 0.12

<0.0001

0.047

0.01 - 0.19

<0.0001

Indian v. black

Peri-urban

0.22

0.10 - 0.51

0.0005

0.213

0.07 - 0.62

0.004

Area where student completed high school Rural

Rural v. urban

Urban

9.40

3.47 - 25.45

<0.0001

* 3.326

0.60 - 8.30

0.167

Rural v. urban

Peri-urban

5.03

1.81 - 13.94

0.0019

2.734

0.53 - 3.99

0.227

Peri-urban v. urban

Urban

2.80

1.07 - 7.23

0.0358

2.862

0.59 - 13.78

0.189

Peri-urban v. urban

Peri-urban

1.30

0.45 - 3.72

0.6269

2.798

0.62 - 12.45

0.176

Rural

Rural v. urban

Urban

14.01

5.08 - 38.67

<0.0001

1.872

0.35 - 10.01

0.463

Rural v. urban

Peri-urban

4.20

1.60 - 10.96

0.0035

0.797

0.15 - 3.98

0.782

Peri-urban v. urban

Urban

5.40

1.97 - 14.79

0.0010

1.373

0.27 - 6.84

0.699

Peri-urban v. urban

Peri-urban

0.85

0.26 - 2.82

0.7955

0.243

0.04 - 1.42

0.117

Area where student was born *

*Reference value.

underlying constructs of this observation. It may be possible that black students who, within the context of this study, were mostly raised in rural settings, may indeed have chosen to complete their rotation in urban settings but have the desire to go back to rural areas once they have completed their medical training. Therefore, one aspect to explore would be whether there is a short-term desire to experience learning in a more urban setting but a long-term career plan in a rural setting, and the reasons for these choices. Despite the above findings, it was encouraging to learn that the largest proportion of students, regardless of their sex, ethnicity and background characteristics, selected rural DTCs. This pattern was observed across all six cohorts of students. Although not specifically asked, this may be an indication that these students anticipated better learning opportunities in rural health facilities, which are presumed to be more understaffed and student-friendly than facilities in urban areas. Our findings also revealed that students in urban areas reported better learning experiences than those in rural areas. This was particularly true with regard to access to necessary hospital resources, their interaction with the clinical team and the clinical skills and knowledge gained. Similarly, compared with rural DTCs, students in urban DTCs felt more confident in approaching patients and became more conscious of the organisation of the healthcare system and the role of various professionals within it. However, students from rural DTCs were better orientated towards the social context of practice, meeting the objective of the programme. The last finding

mirrors what has been shown elsewhere.[15] These are important issues to consider if rural DTCs are to contribute to the long-term staffing of rural facilities. Training institutions therefore need to ensure that resources are equitably provided in both rural and urban training sites. The differences in the studentsâ&#x20AC;&#x2122; learning experiences across the DTCs could be attributed to logistical and operational issues that were encountered during the course of the programme. The programme was not implemented wholly as originally intended owing to delays in the mobilisation of resources, particularly in rural DTCs. This highlights the complexities of the distributed approach in which students are allocated to multiple settings with varying characteristics. The situated learning theory postulates that a training location provides the context within which a student develops his or her professional identity.[16,17] Programmes that utilise a distributed design must ensure that adequate resources, both human and material, are available to enhance the personal learning of students in rural areas. This would more likely encourage these students to return to similar settings for future practice. Universities and other medical training institutions may not achieve this without supportive policies at macro level, particularly from government structures and other relevant civil society organisations. Finally, our study revealed that the studentsâ&#x20AC;&#x2122; knowledge and skills increased across all sites following their exposure to the DTCs. This was based on a number of learning elements posited in the programme. Despite

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

Research the use of a subjective measure of knowledge and skill gain, this was an encouraging finding which indicates promising effects of the programme in enhancing learning among the students.

Conclusion

Students had positive perceptions and experiences about the primary care curriculum and the decentralised training programme, even though these varied depending on the geographical location of the site. The choice for the location of the DTC was dependant on gender, race, and place where the student grew up and completed high school. Although this study can be considered a pilot, our findings suggest that students should be allocated to sites based on their individual characteristics to maximise their potential for experiential learning and the likelihood of eventual career paths in areas where they are needed the most. Nevertheless, large-scale studies conducted within a much broader context may be required to substantiate the findings reported here. Acknowledgements. This work was made possible by grant No. 5R24TW008863 from the US President’s Emergency Plan for AIDS Relief (PEPFAR), and the National Institutes of Health, US Department of Health and Human Services. Its contents are solely the responsibility of the UKZN MEPI programme and do not necessarily represent the official views of the US government.

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

References 1. De Vries E, Irlam J, Couper I, Kornik S. Career plans of final-year medical students in South Africa. S Afr Med J 2010;100(4):227-228. [http://dx.doi.org/10.7196/samj.3856] 2. Research Consortium. The Shortage of Medical Doctors in South Africa. Scarce and Critical Skills Research Project. South Africa: Research Consortium, 2008. http://www.labour.gov.za/DOL/downloads/documents/ researchdocuments/Medical%20Doctors DoL Report.pdf (accessed August 2014). 3. Rural Health Workforce Australia. Will More Medical Places Result in More Rural GPs? Melbourne: RHWA, 2008. 4. United States General Accounting Office. Physician Workforce: Physician Supply Increased in Metropolitan and Nonmetropolitan Areas but Geographic Disparities Persisted. Report No. GAO-04-124. Washington: USGAO, 2003. 5. Hsueh W, Wilkinson T, Bills J. What evidence-based undergraduate interventions promote rural health? N Z Med J 2004;117(1204):U1117. 6. Zink T, Center B, Finstad D, et al. Efforts to graduate more primary care physicians and physicians who will practice in rural areas: Examining outcomes from the University of Minnesota – Duluth and the Rural Physician Associate Program. Acad Med 2010;85(4):599-604. [http://dx.doi.org/10.1097/ACM.0b013e3181d2b537] 7. Woloschuk W, Tarrant M. Does a rural educational experience influence students’ likelihood of rural practice? Impact of student background and gender. Med Educ 2002;36(3):241-247. [http://dx.doi.org/10.1046/j.13652923.2002.01143.x] 8. Wilson NW, Couper ID, de Vries E, Reid S, Fish T, Marais BJ. A critical review of interventions to redress the inequitable distribution of healthcare professionals to rural and remote areas. Rural Remote Health 2009;9(2):1060-1081. 9. StataCorp. Stata Statistical Software: Release 13. College Station, TX : StataCorp LP, 2013. 10. Slade S. On the move: A retrospective, longitudinal view of physician migration patterns in Canada. Paper presentation, 6th Annual Physician Workforce Research Conference, Alexandria, Virginia, USA, 6 - 7 May 2010. 11. Strasser R, Hogenbirk JC, Lewenberg M, Story M, Kevat A. Starting rural, staying rural: How can we strengthen the pathway from rural upbringing to rural practice? Aust J Rural Health 2010;18(6):242-248. [http://dx.doi. org/10.1111/j.1440-1584.2010.01167.x] 12. Norris TE. Education for rural practice: A saga of pipelines and plumbers. J Rural Health 2000;16(3):208-212. [http://dx.doi.org/10.1111/j.1748-0361.2000.tb00458.x] 13. Rabinowitz HK, Paynter NP. The role of the medical school in rural graduate medical education: Pipeline or control valve? J Rural Health 2000;16(3):249-253. [http://dx.doi.org/10.1111/j.1748-0361.2000.tb00468.x] 14. Henry JA, Edwards BJ, Crotty B. Why do medical graduates choose rural careers? Rural Remote Health 2009;9(1):1083. www.rrh.org.au (accessed 6 November 2013). 15. Dornan T, Bundy C. What can experience add to early medical education? Consensus survey. BMJ 2004;329(7470):834. [http://dx.doi.org/10.1136/bmj.329.7470.83] 16. Gray JD, Steeves LC, Blackburn JW. The Dalhousie University experience of training residents in many small communities. Acad Med 1994;69(10):847-851. [http://dx.doi.org/10.1097/00001888-199410000-00016] 17. Norris TE, Norris SB. The effect of a rural preceptorship during residency on practice site selection and interest in rural practice. J Fam Pract 1988;27(5):541-544.

Research Exploring the relationship between demographic factors, performance and fortitude in a group of diverse 1st-year medical students S Hamid, BSocSc, BSocSc Hons, MA, PhD; V S Singaram, BMedSc, MMedSc, PhD Clinical and Professional Practice, School of Clinical Medicine, College of Health Sciences, University of KwaZulu-Natal, Durban, South Africa Corresponding author: S Hamid (shaista.saib@gmail.com)

Background. The majority of 1st-year students are ill-equipped for university life. This heightens stress levels, which are accentuated by a lack of resilience and impact negatively on academic performance and personal wellbeing. Objectives. To explore, within the paradigm of positive psychology, the relationship between the self, family and support constructs of fortitude, and academic performance of 1st-year medical students. Method. First-year medical students completed a fortitude questionnaire and their academic performances in two academic modules were collated. Mann-Whitney and Kruskal-Wallis tests were employed for statistical analysis of the variables. Pearson correlation coefficients were calculated to assess the relationship between academic performance and fortitude subscales, as well as the fortitude composite score. Results. The student population was multicultural, multilingual and had different educational and residential backgrounds. The fortitude instrument was found to be reliable and correlated significantly with student academic performance. Male students had significantly higher fortitude scores than female students. Students who had attended state/government schools had significantly lower fortitude than those who had attended private and ex-Model C schools. Students with prior degrees had higher fortitude than matriculants. Conclusion. The significant, albeit moderate, positive correlation between fortitude and academic performance highlights the need for further exploration of wellbeing and holistic development of medical students. Support programmes are recommended to bridge the gap related to gender and educational background. Low and fair levels of fortitude indicate a need for corrective measures. These could include consulting relevant support networks such as student counsellors, mentors and academic development personnel. Afr J Health Profession Educ 2016;8(1 Suppl 1):99-103. DOI:10.7196/AJHPE.2016.v8i1.748

The majority of 1st-year students are ill-equipped for university life and academia. This under-preparedness is accentuated by literacy issues, socioeconomic factors and lack of resilience.[1] These issues heighten stress levels, which affect academic performance and personal wellbeing.[2] Although psychological morbidity may be high among university students, it is often neglected in educational settings. Emotions are the primary forces driving motivation, and poor control and understanding of one’s own or others’ emotions may result in flawed social interactions.[3] For health practitioners, this finding is crucial, as it may not only potentially compromise academic functioning and their personal wellbeing, but may also, in the long term, affect patient care. It has been reported that there is a paucity of research on the coping and adjustment of medical professionals, particularly in the South African (SA) context.[4] This view is reiterated by O’Rourke et al.[5] and Greysen et al.[6] with regard to medical students. The former authors note that although medical students’ distress is acknowledged, more research into causes and intervention strategies is needed. The latter asserts that the dramatic changes in sub-Saharan Africa over the past few decades necessitate a greater diversity of research into the perspectives and experiences of and within medical education. Barriers to student success that compound adjustment issues affect throughput and retention rates at SA universities, accounting to a large degree for high dropout rates.[7] Despite these findings, it appears that tertiary institutions still place primary emphasis on intellectual/cognitive factors, ultimately overlooking the possibility of non-cognitive factors as significant role players in student success. The study of positive psychology concerns itself with all aspects of positive living, and the fostering of inter- and intrapersonal resources for personal

development and happiness. Positive psychology adopts a strengths-based perspective (i.e. factors which contribute to things going right, such as ‘mental health’ as opposed to ‘mental illness’) in understanding and enhancing wellbeing. Extending the salutogenic perspective, Strümpher,[8] a pioneer in the field of psychofortology, theorised that fortitude resulted in psychological coping, emotional stability and stress tolerance. Following Strümpher’s theory, Pretorius[9] expanded on the concept of fortitude within the SA context. He hypothesised that there are three main constructs that contribute to fortitude. They are an evaluative awareness of the self (self-appraisal), an evaluative awareness of family (family appraisal) and an evaluative awareness of social support (support appraisal). Pretorius proposed that fortitude was derived and shaped from the construction of self and world and that these constructions were shaped from our appraisal of ourselves and our perceived sense of family and social support.[9] He further described that a person with low fortitude would be more prone to self-doubt, impaired perception of personal competency in coping with stressors and a disengagement from active coping efforts. However, a person with high fortitude would be more confident and would adopt more problem-focused styles of coping. There is limited research in the field of fortitude focusing on medical students. A previous study, which researched salutogenic factors among SA community-service doctors, recommended that it would be beneficial to investigate fortitude, as opposed to salutogenic factors, as fortitude is more holistic and all-encompassing.[10] Equipping students to recognise their inner strengths and resources may assist them in coping with stress and adversity and could result in empowered, confident and well-adjusted individuals. The objective of this article is to explore the relationships between the three

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

Research constructs of fortitude,[9] academic performance and demographic factors in a diverse group of 1st-year medical students.

Methods

Ethical clearance and gatekeeper approval were obtained from the Human and Social Sciences Ethics Committee at the University of KwaZuluNatal (UKZN), Durban, SA (HSS/0119/013D). A quantitative correlational research design was adopted using convenience sampling. The study was conducted at the Nelson R Mandela School of Medicine at UKZN. All 1st-year medical students in the 2013 cohort were invited to complete the questionnaire adopted for use in the study. A total of 200 questionnaires were distributed, and 165 students consented to participate. Students’ end-of-semester academic results from two academic modules, Becoming a Profes sional (36-credit bearing) and Basic Science (96-credit bearing), were obtained from the Faculty of Medicine. These are the only two academic modules that all 1st-year students participate in – the other three modules (8-credit bearing), English, isiZulu and Computer Skills, are based on an entrance examination.

Instrument

The research instrument consisted of 34 questions, to which the students responded using either a nominal or an ordinal scale. The questionnaire was divided into a demographic and fortitude component. Demographic data were collected on gender, race, age, type of school (urban v. rural), sources of funding, and attendance at peer-mentoring and study skills sessions. The Fortitude Questionnaire (FORQ) was used to assess the fortitude component, adopted from Pretorius and Heyns[11] with permission from the authors. It comprises 20 items aimed at measuring the theoretical construct of fortitude. It has three subscales described below: • The self-appraisal scale comprises seven items related to the global appraisal of the self, as well as more specific appraisals such as problemsolving efficacy and mastery or competence (e.g. ‘I take a positive attitude towards myself ’). • The family-appraisal scale has seven items regarding the evaluative awareness of the family environment, such as support from family, level of conflict, cohesiveness in the family and family values (e.g. ‘there is plenty of attention for everyone in my family’).

100

• The support-appraisal scale is an evaluative awareness of the support from others. These six items also included beliefs about the efficacy of using such support resources (e.g. ‘I am very satisfied with the comfort and support I get from others’).

fortitude subscales, as well as the fortitude composite score. Reliability of the instrument was assessed using Cronbach’s α.

Results

The sample was predominantly female (62%). According to racial background, the sample consisted of black (77%), Indian (13%), white (4%) and coloured (7%) students. Ages ranged between 18 and 21 years (78%). Seventy-five percent of participants had attended a state/government school (i.e. poorly resourced), 7% had attended a private school and 18% had attended an ex-Model C school (i.e. well resourced). Forty-five percent of participants completed their schooling in a rural area. The majority lived in a university residence (61%), 29% lived with family or friends and 10% lived in a private residence. Twenty-two percent of the students had attended some form of study skills session and 82% had attended peer-mentoring sessions. Cronbach’s α coefficients were 0.77 for selfappraisal, 0.88 for family-appraisal and 0.82 for support-appraisal scales. This shows a high internal consistency of the items for all three constructs. A comparison of the medians of the fortitude subscales showed that self-appraisal (20.79), with a range of 8 - 28, had the highest median, while support appraisal (16.73), with a range of 2 - 24, had the lowest (Table 1). The overall median

Data analysis

The data were analysed using Stata/IC 13.0 (StataCorp LP, USA).[12] Descriptive analyses involved computation of summary statistics using frequencies and graphs, based on survey responses. Continuous data were tested for normality using the one-sample KolmogorovSmirnov test. The results of this test revealed that the p-values were statistically significant (p=0.000), implying that distribution of the data was not normal. Hence, for inferential statistical analyses, the non-parametric MannWhitney U-test was used to test whether the medians of the continuous variables between two independent groups were statistically different at p<0.05, and the Kruskal-Wallis test was employed to test the statistical significance of the differences between three or more groups. Where possible, box plots were constructed to show the difference in medians and interquartile ranges of continuous variables across groups. Pearson correlation coefficients were calculated to assess the relationship between academic performance on the two subjects and

Table 1. Descriptive statistics for fortitude and its subscales Scale

Median (SD)

Items per subscale, n

Minimum scored

Maximum scored

IEP (a - b)

Self-appraisal

20.79 (4.23)

(8.20)

Family appraisal

19.50 (6.50)

(0.28)

Support appraisal

16.73 (4.26)

(2.24)

Fortitude

57.01 (11.44)

(20.80)

SD = standard deviation; IEP = interval endpoints for each score.

Table 2. Correlations between fortitude and academic performance Self-appraisal score Module completed

Becoming a Professional

158

0.23

Basic Science

152

0.09

Average of the two assessment scores

135

0.16

*Pearson’s product moment correlation coefficient. † Correlation at p<0.01 (two-tailed). ‡ Correlation at p<0.05 (two-tailed). § Correlation at p<0.001 (two-tailed).

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

†

‡

Familyappraisal score

Supportappraisal score

Fortitude composite score

158

0.20

‡

158

0.25

†

158

0.28§

152

0.17‡

152

0.22†

152

0.20‡

135

0.21

135

0.29

135

0.29§

Research

Discussion

Three main constructs that contribute to fortitude are the evaluative awareness of the self

Support appraisal

Friends often have good advice to give 6.0

38.0

45.4 28.3

25.1

28.2 19.0

47.0

30.0

26.3

31.0 37.0

26.0

35.0

33.0

35.0

I rely on my family for emotional support 8.0 20.3

34.0

38.1

There is plenty of time and attention for 3.0 30.0 everyone in our family

3.0

38.1

Learning about new and different things is 3.0 17.0 very important in our family

38.0

At times I think I am no good at all 9.0 12.0

32.0

When making a decision, I weigh the consequences of 10.0 each alternative and compare them with each other

I trust my ability to solve new and difficult problems 1.3

28.0

I have no trouble making up my mind 7.0

48.0 45.0 41.0

41.0 44.3

31.3

I take a positive attitude towards myself 4.0 18.4 I always feel pretty sure of myself 5.0

41.3

45.0

On the whole I am satisfied with myself 5.0 13.4

32.0

Applies very strongly

30.3 28.0

Members of my family are good at helping me 4.0 24.3 solve problems I have a deep sharing relationship with a number of members of my family 7.0

30.0

32.2

53.3

16.0

Does not apply

36.2

39.4

I am very satisfied with the comfort and 3.0 22.0 support that I get from others

In my family we tell each other about our 14.0 personal problems Family support

41.0

I know that someone will always be around if 4.0 20.3 I need assistance I am very satisfied with the help and support that I get from those that I can count on 1.014.0

Activities in our family are pretty carefully planned 9.2

17.0

40.0

My friends give me the moral support I need 3.0 26.4

In general, there are more than five people that I could really count on to be dependable when I need help

Self-appraisal

for the composite score of fortitude was 57.01 (Table 1). Fig. 1 shows the distribution of fortitude scores across the 20 different measures posited in the study questionnaire. As illustrated, the majority of the students felt strongly that these statements applied to them. This implies that, with regard to the family-appraisal subscale, students believed that they had strong levels of family support; the self-appraisal subscale revealed that they felt positive about themselves, and the support-appraisal subscale indicated that they were satisfied with the support they received from others. No statistical differences were found between the race groups for all three subscales and the overall fortitude scores. Median scores for the family-appraisal subscale, the support-appraisal subscale and the fortitude composite score were not statistically different between male and female students (p>0.05; Fig. 2). However, male students had a significantly higher median score than their female counterparts on the self-appraisal subscale (p<0.0001). The Kruskal-Wallis test revealed statistically significant differences in the composite fortitude scores of students who had attended different types of schools (p<0.05). Those students who had attended state/government schools had the lowest composite median score (Fig. 3). Fig. 4 illustrates that students with other higher educational experience, such as the completion of another degree, prior to doing medicine, had significantly higher median scores across all three fortitude subscales compared with those without such experience. These differences were statistically significant (p<0.05 for the family-appraisal, supportappraisal and self-appraisal subscales; p<0.001 for the fortitude composite score). The results shown in Table 2 indicate that there was a weak but statistically significant correlation between the self, family and support subscales and performance in the Becoming a Professional and Basic Science academic modules. When the three subscales were combined, the resultant fortitude composite score was significantly but moderately correlated with the academic scores of the Becoming a Professional module. Furthermore, the composite score of fortitude was also significantly moderately correlated with the average score for both academic modules.

43.0

27.0 19.3 41.0

37.0 45.0

19.1

10 20 30 40 50 60 70 80 90 100 %

Fig. 1. The distribution of fortitude scores across the 20 different measures of fortitude: analysis disaggregated by fortitude subscale.

(self-appraisal), an evaluative awareness of family (family appraisal) and an evaluative awareness of social support (support appraisal).[11] High inter nal consistency of the items in all three constructs supports the reliability of the instrument used in this study. We found that the majority of the participants had high levels of fortitude in all three constructs. This implies that this cohort of medical students had positive perceptions or appraisals of themselves, their families and their social supports. The self-appraisal scale presented with the highest mean scores overall for this group of students. This may be due to the finding that, compared with other university students, medical students were ‘known to be highly motivated students’.[13] Also, being selected from a massive pool of applicants, these students are constantly reminded that they are the ‘cream of the crop’.

Women generally present with greater psycho logical distress[14] and hence may have lower fortitude scores, as we have found in this study. When comparing mean scores, males had higher fortitude and significantly different scores for self-appraisal than females. Although Rahim[15] did not find any significant relationship between fortitude and gender, Roothman et al.[16] found that men presented with higher fortitude levels than females. The finding in this study reflects that male students have a higher sense of self, i.e. they are more certain of themselves, have more positive attitudes about themselves, have less trouble making up their minds and trust their abilities to solve problems. Dhaniram[10] also found, in her research of stress levels among SA community-service doctors, that female students and female physicians showed higher stress levels. These findings highlight the need

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

101

Raw score

Research

10 0

0 Male

Female

Self-appraisal subscale Support-appraisal subscale

Family-appraisal subscale Composite score for fortitude

Fig. 2. Distribution of the three fortitude subscale scores by gender.

Second degree Self-appraisal subscale Support-appraisal subscale

First degree Family-appraisal subscale Composite score for fortitude

Fig. 4. Distribution of the three fortitude subscale scores by whether student acquired a previous degree.

80 70

Raw score

60 50 40 30 20 10 0 Private school

State/government school

Self-appraisal subscale Support-appraisal subscale

Ex-Model C school

Family-appraisal subscale Composite score for fortitude

Fig. 3. Distribution of the three fortitude subscale scores by type of school where student matriculated.

for wellbeing programmes that target females, although not to the exclusion of male students. The underlying reasons for this finding should also be explored further using interviews or focus groups. Students who had acquired other educational experiences or degrees prior to studying medicine had significantly higher median scores across all three fortitude subscales compared with those who had not. These findings may be attributed to the increased experience and level of maturity of these students, compared with those pursuing their first degree directly after high school.[10] Students who had attended state/government schools had significantly lower fortitude than those who had attended private or ex-Model C schools. The type of school attended has been found to influence academic scores.[17] These findings may be attributed to the fact that private and ex-Model C schools have greater infrastructure, resources and much lower teacher-to-student ratios than state schools. They also highlight the need for support programmes to bridge this gap in higher education. This study contributes to the body of knowledge regarding the association between non-cognitive predictors, such as student psychological wellness on academic performance.[18-20] A moderately statistically significant

102

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

relationship was found between fortitude and academic performance. This finding highlights the need to investigate confounding variables that may influence academic performance. Hence, further research in this area, preferably with a larger sample, is needed. Furthermore, the weak but statistically significant findings of this study suggest an assoÂ ciation between psychological wellbeing and academic performance.[13] Developing fortitude in medical students and fostering an environment of positive social and academic support could have positive implications for academic success. These findings support the need for further exploration of self-development and wellness programmes at medical schools. Such programmes could serve as buffers against medical school stressors and could contribute to enhancing and sustaining fortitude. As the study was limited to 1st-year students only, sampling across all years of medical study is recommended for future studies to investigate the effect of year of study on fortitude. It is also recommended that longitudinal studies be used to assess the fortitude of students as they progress through their years of study. Further validation of the instrument is recommended in other health science settings. This study was dependent on self-reported information and perceptions from the participant. However, the fortitude instrument used was found to be reliable. It is suggested that further study includes other health science students, such as those from nursing and pharmacy, to gain a more extensive view of the levels of fortitude. This would enable comparison of curriculum and academic environment influences in health science education.

Conclusion

Male students had significantly higher fortitude scores than females. Students who had attended state/government schools had significantly lower fortitude than those who had attended private or ex-Model C schools. Students with prior degrees had higher fortitude than matriculants. Low and fair levels of fortitude are indicative of a need for corrective measures. These could include consulting the relevant support networks, such as student counsellors, mentors and academic development personnel. The significant, albeit moderate, correlation between fortitude and academic performance highlights the need for further investigation of the fortitude instrument.

Research Acknowledgements. This publication was made possible by grant No. R24TW008863 from the Office of the US Global AIDS Coordinator and the US Department of Health and Human Services, National Institutes of Health. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the US government. The medical students are acknowledged for their participation, and Dr M Muzigaba for statistical guidance and analysis. References 1. Nel BP. Academic advising as intervention for enhancing the academic success of ‘at-risk students’ at a comprehensive university in South Africa. Mediterr J Soc Sci 2014;5(27):732-739. [http://dx.doi.org/10.5901/ mjss.2014.v5n27p732] 2. Pillay AL, Ngcobo HSB. Sources of stress and support among rural-based first-year university students: An exploratory study. S Afr J Psychol 2010;40(3):234-240. [http://dx.doi.org/10.1177/008124631004000302] 3. Ahammed S, Abdullah AS, Hassane SH. The role of emotional intelligence in the academic success of United Arab Emirates university students. Int Educ 2011;41(1):7-21. 4. Brown-Baatjies O, Fouché P, Watson M, Povey JL. The biopsychosocial coping and adjustment of female medical professionals. S Afr J Psychol 2006;36(1):126-143. [http://dx.doi.org/10.1177/008124630603600108] 5. O’Rourke M, Hammond S, O’Flynn S, Boylan G. The medical student stress profile: A tool for stress audit in medical training. Med Educ 2010;44(10):1027-1037. [http://dx.doi.org/10.1111/j.1365-2923.2010.03734] 6. Greysen SR, Dovlo D, Olapade-Olaopa EO, Jacobs M, Sewankambo N, Mullan F. Medical education in subSaharan Africa: A literature review. Med Educ 2011;45(10):973-986. [http://dx.doi.org/10.1111/j.13652923.2011.04039.x] 7. Kelly-Laubscher RF, van der Merwe M. An intervention to improve academic literacies in a first year university biology course. Crit Stud Teach Learn 2014;2(2):1-23. [http://dx.doi.org/10.14426/cristal.v2i2.23]

8. Strümpher DJW. The origins of health and stress. From ‘salutogenesis’ to ‘fortigenesis’. S Afr J Psychol 1995;25(2):81-89. [http://dx.doi.org/10.1177/008124639502500203] 9. Loots T, Ebersohn L, Ferreira R, Eloff I. Teachers addressing HIV and AIDS-related challenges resourcefully. Southern Afr Rev Educ 2012;18(1):56-84. 10. Dhaniram N. Stress, burnout and salutogenic functioning amongst community service doctors in KwaZuluNatal. MA thesis. Pretoria: University of South Africa, 2003. http://hdl.handle.net/10500/1526 (accessed 19 April 2016). 11. Pretorius TB, Heyns PM. Fortitude as stress-resistance: Development and validation of the Fortitude Questionnaire (FORQ). 2005. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.464.1299&rep=rep1& type=pdf (accessed 15 June 2012). 12. StataCorp. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP, 2013. 13. Jafari N, Loghmani A, Montazeri A. Mental health of medical students in different levels of training. Int J Prev Med 2012;3(Suppl 1):s107-s112. 14. Akhlaq BA, Arouj K. Study on the self-esteem and strength of motivation of medical students. Int J Bus Humanit Technol 2014;4(5):58-63. 15. Rahim MZ. Investigating the relationship between fortitude and academic achievement in students from historically disadvantaged backgrounds. MPsych thesis. Cape Town: University of the Western Cape, 2007. http:// hdl.handle.net/11394/2762 (accessed 19 April 2016). 16. Roothman B, Kirsten DK, Wissing MP. Gender differences in psychological well-being. S Afr J Psychol 2003;33(4):212-218. 17. Sommerville T, Singaram VS. The whole is greater than the sum: A longitudinal study of demographic influences on medical student assessment scores. Alternation 2015;17:28-53. 18. Singaram VS, van der Vleuten CP, Muijtjens AM, Dolmans DH. Relationships between language background, secondary school scores, tutorial group processes and students’ academic achievement in PBL: Testing a causal model. Interdiscip J Probl Based Learn 2012;6(1):153-164. [http://dx.doi.org/10.7771/1541-5015.1316] 19. Sommer M, Dumont K. Psychosocial factors predicting academic performance of students at a historically disadvantaged university. S Afr J Psychol 2011;41(3):386-395. [http://dx.doi.org/10.1177/008124631104100312] 20. Van Lingen JM, Douman DL, Wannenburg I. A cross-sectional exploration of the relationship between undergraduate nursing student wellness and academic outcomes at a South African higher education institution. S Afr J Psychol 2011;41(3):396-408. [http://dx.doi.org/10.1177/008124631104100313]

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

103

Research Motivated strategies for learning and their association with academic performance of a diverse group of 1st-year medical students S Hamid, BSocSc, BSocSc Hons, MA, PhD; V S Singaram, BMedSc, MMedSc, PhD Clinical and Professional Practice, School of Clinical Medicine, College of Health Sciences, University of KwaZulu-Natal, Durban, South Africa Corresponding author: S Hamid (shaista.saib@gmail.com)

Background. Most instruments, including the well-known Motivated Strategies for Learning Questionnaire (MSLQ), have been designed in western homogeneous settings. Use of the MSLQ in health professions education is limited. Objective. To assess the MSLQ and its association with the academic performance of a heterogeneous group of 1st-year medical students. Methods. Eighty-three percent of 1st-year medical students consented to participate in this quantitative study. The MSLQ consisted of a motivation strategies component with six subscales, while the learning strategies component had nine subscales. Demographic and academic achievement information of the students was also collected. Stata version 13 (StataCorp LP, USA) was used for the statistical analyses of all data. Results. Female students displayed significantly higher motivational scores. Students with prior educational experience and those who attended peermentoring sessions had significantly higher learning strategy scores. Significant but moderate relationships were found between academic performance and the motivation strategies subsumed within the categories ‘task value’ and ‘self-efficacy for learning performance’. In terms of the ‘learning strategy component’, ‘critical thinking’, and ‘time and study environment’, the composite score was significantly but poorly correlated to academic performance. Conclusion. Overall, limited correlations were found between the MSLQ scores and academic performance. Further investigation of the use of the MSLQ and its association with academic achievement is recommended, with greater focus on specific learning events than on course outcomes. This study highlights the importance of evaluating an instrument in a specific context before accepting the findings of others with regard to the use of the instrument and its correlation with academic performance. Afr J Health Professions Educ 2016;8(1 Suppl 1):104-107. DOI:10.7196/AJHPE.2016.v8i1.757

Poor learning strategies are among the factors responsible for the high failure rate of 1st-year students.[1] Non-cognitive factors must be taken into account to facilitate academic success.[2] Motivation is a prominent factor, which is linked to positive academic outcomes and associated with psychological wellbeing.[3] Motivation is one of the central constructs in understanding academic performance and influencing learning strategies. Students with higher motivation levels are more attentive and engaged in their learning than those with lower levels of motivation.[4] The former may be viewed as self-regulated students with a higher degree of independent engagement in their learning processes. Self-regulation has been defined as the ‘mindful capacity to plan, guide and monitor one’s behaviour flexibly according to changing circumstances and is considered as vital for autonomous and adaptive functioning’.[5] Therefore, self-regulated learners tend to be cognitively, emotionally and behaviourally involved in their learning processes.[6] Among medical students, high motivation was linked to high academic performance in both the preclinical and clinical years and to health-related extracurricular activities.[7] Although these results are supported by other studies,[8-10] contradictory findings, related to lack of association between academic performance and motivation, have also been published.[11] Lack of motivation or amotivation has also been found to be one of the important barriers to learner achievement and performance.[12] Therefore, factors that enhance motivation need to be investigated. Motivation is understood to be triggered by intrinsic or extrinsic factors. Intrinsic motivation refers to a person’s actions being influenced by an internal state – a self-determined form of motivation.[13] External motivation, in contrast, is influenced by external sources, e.g. an anticipated reward. Both intrinsic and extrinsic forms of motivation have been found to be positively associated with

104

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

adjustment to university.[13] Consequently, students who are well adjusted experienced a sense of belonging within the university, did not feel overwhelmed by the amount of work, and performed well academically.[7] Much research on motivation has been done in general education, but in medical education such research has been limited.[7] While there are many instruments to measure motivation and learning strategies, most of these, including the well-known Motivated Strategies for Learning Questionnaire (MSLQ),[14] have been designed in settings in western countries. Use of the MSLQ in health professions education is also limited. Hence, the objective of this study was to assess the MSLQ and its association with the academic performance of a diverse group of 1st-year medical students.

Methods

All 1st-year medical students at the Nelson R Mandela School of Medicine (NRMSM), University of KwaZulu-Natal (UKZN), Durban, South Africa were invited to participate. A total of 200 questionnaires were distributed; 165 students (83%) consented to completing the questionnaire. The summative end of semester academic results of the only two compulsory academic modules involving all 1st-year students (Becoming a Professional and Basic Science) were obtained from the Faculty of Medicine. Ethics approval and gatekeeper permission were obtained from UKZN’s Human Social Sciences Research Ethics Committee (HSS/0119/013D).

Instrument

The research instrument contained 95 items, with nominal and ordinal levels of measurement. It comprised two sections, i.e. a demographic section and the MSLQ.[7] The demographic section consisted of 14 items related to gender, age, type of school (urban v. rural), attendance of peer-mentoring

Research sessions, attendance of study skills sessions and degree choice. The MSLQ had 81 items.[14] This validated scale assesses motivation and self-regulated learning strategies, as illustrated in Table 1.

Most of the sections have a reliability score that is close to or exceeds the recommended value of 0.7. This indicates an overall degree of acceptable, consistent scoring of items within each construct.

Data analysis

Analyses of relationship between demographic characteristics, academic performance and motivation

Stata version 13 (StataCorp LP, USA) was used for analysis of all the data.[15] Reliability was measured using Cronbach’s α, which determines the internal consistency or average correlation of items in a survey instrument to gauge its internal validity. Continuous variables were first inspected using the ShapiroWilk and Shapiro-Francia tests for normality to determine which statistical tests were appropriate for the data. The data that were found to be normally distributed were analysed using parametric tests. Non-parametric tests were employed for data that were not normally distributed. For normally distributed data, the two independent samples t-test was used to compare mean composite scores for two independent groups. A one-way analysis of variance (ANOVA) was used for categorical independent variables (three or more categories) and a normally distributed interval-dependent variable (composite scores of learning strategy) to test for differences in the means of the dependent variable broken down by the levels of the independent variable. Otherwise, non-parametric equivalent tests were used. With regard to scoring of the MSLQ, students rated themselves on a 7-point Likert scale from 1 (not at all true of me) to 7 (very true of me). Scales were constructed by taking the mean of the items that comprise that scale, e.g. intrinsic goal orientation (IGO) has four items. An individual’s score for IGO was computed by adding the four items and dividing the total by the number of items to obtain an average score.

Results

Table 1 depicts the reliability and descriptive statistics obtained for the MSLQ. The reliability statistics for the MSLQ displayed fair to good internal validity. Table 1. Reliability and descriptive statistics for the MSLQ (N=165) MSLQ scales

As illustrated in Table 2, statistically significant differences were found between male and female students in the composite score for motivation (p=0.03). Based on the rank sum (7 814) and expected rank sum (8 466) scores, female students had much higher scores than males. No other statistically significant relationships were found between student characteristics and motivation. Task value, self-efficacy for learning performance and test anxiety (inversely) correlated significantly with both modules (Table 3). IGO and control of learning beliefs correlated significantly but poorly with the Becoming a Professional module. The composite score for motivation and other subscales had limited correlation with the academic performance in both modules. Table 2. Comparison between composite score for motivation and demographics: Mann-Whitney U-test (N=165)* Test statistics Student characteristics

Rank sum

Expected rank sum

z-score

p-value

Male

5 881

5 229

2.190

0.03

Female

103

7 814

8 466

Urban

7 481

7 636

−0.509

0.61

Rural

6 214

6 059

First

142

11 710

11 786

0.358

0.72

Second

1 985

1 909

Yes

159

13 210.5

13 117.5

0.891

0.37

319.5

412.5

Yes

1 944.5

1 732.5

1.045

0.3

143

11 585.5

11 797.5

Yes

2 959

2 970

−0.044

0.97

128

10 571

10 560

1.464

0.14

Gender

Geographical area of high school

Medical degree first or second choice

Mean (SD)

Cronbach’s α

Intrinsic goal orientation (4 items )

5.01 (1.07)

0.60

Extrinsic goal orientation (4 items)

5.75 (1.04)

0.62

Task value (6 items)

5.71 (0.97)

0.80

Control of learning beliefs (4 items)

5.44 (1.03)

0.51

S elf-efficacy for learning and performance (8 items)

5.22 (1.04)

0.88

Test anxiety (5 items)

4.42 (1.32)

0.68

Rehearsal (4 items)

5.01 (1.22)

0.64

Elaboration (6 items)

5.12 (1.14)

0.80

Organisation (4 items)

5.16 (1.28)

0.71

Peer learning (3 items)

3.98 (1.61)

0.72

Critical thinking (5 items)

4.26 (1.30)

0.77

Metacognitive self-regulation (12 items)

4.67 (0.97)

0.77

Time and study environment (8 items)

4.53 (0.96)

0.55

Effort regulation (4 items)

4.97 (1.27)

0.58

Yes

135

11 551

11 205

Help-seeking (4 items)

3.87 (1.27)

0.56

2 144

2 490

Motivation strategies

Learning strategies

SD = standard deviation.

Satisfied with the current degree choice

Student has a previous degree

Attended study skills sessions

Attended peermentoring sessions

*The N-score will vary, depending on the number of responses received for each factor.

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

105

Research Correlational analyses of learning strategies Independent sample t-test results in Table 4 depict that having obtained a previous degree and attending peer-mentoring sessions were found to be statistically significantly correlated with the learning strategies adopted. No significant associations were found between learning stra tegies and other demographic variables or student characteristics. The time and study environment subscale was moderately significantly correlated with both academic modules (Table 5). The composite score for the learning strategies and the rest of the eight subscales significantly correlated poorly with academic performance in both academic modules.

Discussion

This study explored the motivated strategies for learning and their association with the academic performances of a diverse group of 1st-year medical students. The MSLQ instru ment was found to be reliable, as there was an overall degree of acceptable, consistent scoring of items within the different categories. Statistically significant differences were found between gender and the composite score for motivation. Similar to Sikhwari’s[16] results, it was found that females had higher scores then males. These studies revealed that females generally engage more with academic activities than males and are consequently higher achievers academically. By comparison, men are reported to place less value on engaging with academic activities. Although urban students had higher scores than their rural counterparts, the differences, such as those between the other characteristics and demographic factors, were not significant. Significant moderate relationships were found between academic performance and the motivation strategies subsumed within task value and self-efficacy for learning performance. Task value refers to students’ perceptions of how important they believe the subject matter is. It is also associated with higher engagement in learning. These students may be more likely to put in greater effort if they appraise academic content as meaningful and relevant. Increased effort and engagement with the subject matter could contribute positively to academic performance. Self-efficacy for learning performance relates to the students’ sense of confidence in their ability to achieve their goals. The influence of self-efficacy on motivation is often ignored

106

Table 3. Correlations between six subscales of motivation and academic performance with regard to two modules in 1st year of medical school Becoming a Professional (n=158)

Basic Science (n=152)

Subscales

Intrinsic goal orientation (average subscore)

0.1864

Extrinsic goal orientation (average subscore)

−0.0016

−0.07

Task value (average subscore)

0.2533†

0.19† 0.13

0.13

†

Control of learning beliefs (average subscore)

0.1777

Self-efficacy for learning performance (average subscore)

0.3672†

Test anxiety (average subscore)

−0.3379

Composite score for motivation (combining the above average subscores)‡

0.1382

†

0.27§ −0.21‡

0.09

*Pearson’s product moment correlation coefficient. † Correlation at p<0.05 (two-tailed). ‡ Correlation at p<0.01 (two-tailed). § Correlation at p<0.001 (two-tailed).

Table 4. Comparison of students’ characteristics by their mean composite scores for learning strategy – independent sample t-test (N=165)* Test statistics Student characteristics

Mean (SD)

95% CI

p-value 0.11

Gender Male

4.85 (0.76)

4.66 - 5.05

Female

101

4.65 (0.81)

4.49 - 4.81

Geographical area where high school was completed Urban

4.67 (0.83)

4.49 - 4.84

Rural

4.81 (0.75)

4.63 - 4.98

First

141

4.75 (0.76)

4.62 - 5.07

Second

4.63 (0.99)

4.21

0.25

Medical degree first or second choice 5.07

Satisfied with the current degree choice Yes

158

4.75 (0.78)

4.62 - 4.88

4.18 (0.98)

2.96 - 5.39

Yes

5.05 (0.94)

4.63 - 5.46

142

4.73 (0.79)

4.61 - 4.81

Yes

4.76 (0.73)

4.51 - 5.01

127

0.79 (0.79)

4.61 - 4.85

0.11

Previous degree 0.05

Attended study skills sessions 0.82

Attended mentoring sessions Yes

135

4.80 (0.82)

4.66 - 4.94

4.41 (0.79)

4.61 - 4.85

0.01

CI = confidence interval. *The N-score will vary, depending on the number of responses received for each factor.

in research; yet students’ beliefs in their own ability are important and merit attention.[17] Self-efficacy and academic performance are interlinked and can be mutually beneficial, as was

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

found previously.[17] Conversely, unlike results of other studies,[18] intrinsic and extrinsic goal orientation correlated poorly with one academic performance in this study.

Research

Table 5. Correlations between nine subscales of the learning strategy scale and academic performance with regard to two modules in 1st year of medical school (n=157) Becoming a Professional

Basic Science

Subscales

Rehearsal (average subscore)

0.0397

−0.01

Elaboration (average subscore)

0.1407

0.09

Organisation (average subscore)

0.0766

0.01

Peer learning (average subscore)

0.0753

0.01

Critical thinking (average subscore)

0.1862

0.16†

Metacognitive self-regulation (average subscore)

0.2082

0.09

Time and study environment (average subscore)

0.3041§

0.25‡

† ‡

Effort regulation (average subscore)

0.2098

0.10

Help seeking (average subscore)

0.0658

0.01

Composite score for learning strategy (combining the above average subscores)

0.2125

0.11

‡

*Pearson’s product moment correlation coefficient. † Correlation at p<0.05 (two-tailed). ‡ Correlation at p<0.01 (two-tailed). § Correlation at p<0.001 (two-tailed).

In our study, test anxiety was found to have a significantly inverse relationship to academic performance. Opateye[8] also found a significant negative relationship between test anxiety and academic performance. The results of this article suggest that students with high task value and high self-efficacy may present with lower test anxiety, as they may be more likely to feel better equipped to deal with the examinations and would probably judge themselves as prepared for the task at hand. Performing well academically further reinforces these feelings and motivation, which may become a cyclical process of continued engagement and motivation. This is supported by previous studies on self-efficacy and its relationship with test anxiety.[9,10] However, our study found that the majority of the six subscales of motivation and academic performance were poorly correlated, although some significant associations were noted. In the ‘learning strategies’ section, stu dents who had prior higher education qualifications obtained higher scores than those who entered medical school without post-school qualifications. Students with existing qualifications are referred to as mature students and their higher scores may be due to their increased tertiary experience. Students who attended mentoring sessions also had significantly higher scores for learning strategies. This is a positive finding, as attending mentoring sessions is currently compulsory at NRMSM. This could act as a source of continuous external motivation and engagement with the learning process in a supportive context. This finding highlights the importance of peer-mentorship programmes for developing the student on a personal and academic level, as reported in other studies.[14] With regard to the learning strategies, only the ‘time and study environment’ subscale showed some significant moderate correlation with academic performance. The significant though weak correlation between critical thinking and academic performance is contrary to findings in other studies, as critical thinking is expected to be positively associated with academic

performance. This factor is indicative of deeper engagement with the academic content compared with rote learning.[18] Overall, this study found limited correlations between the majority of the nine subscales of the learning strategy component of the MSLQ and academic performance. A limitation of the study may be the correlation of once-off self-reported scores to end-of-year summative results. Future studies should perhaps be more focused by correlating the MSLQ to specific learning events. Additionally, several measurements throughout the year may address the possible bias attributed to self-reporting in studies investigating course or curriculum achievement outcomes.

Conclusion

Female students reported more positively on motivation strategies than males. First-year medical students with prior educational experience and those who attended the peer-mentoring sessions reported more positively on learning strategies that they adopted. This study found limited though significant correlations between the MSLQ self-reported scores and academic performance at UKZN. Overall, the study highlights the importance of evaluating an instrument in a specific context before accepting the findings of others with regard to the use of the instrument and its correlation with academic performance. These findings warrant further investigation of the use of the MSLQ in health professions education. Acknowledgements. This publication was made possible by grant No. R24TW008863 from the Office of the US Global AIDS Coordinator and the US Department of Health and Human Services, National Institutes of Health (NIH OAR and NIH ORWH). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the US government. We thank the medical students for their participation in the study and Dr M Muzi gaba for statistical guidance and analysis. References

1. Maree CM, van Rensburg GH. Reflective learning in higher education: Application to clinical nursing. Afr J Phys Health Educ Recreation Dance 2013;19:44-55. [http://dx.doi.org/10.hdl.handle.net/2263/32417] 2. Munteanu A, Costea J, Palos R. Relationships between academic achievement and personality dynamics during adolescence. S Afr J Psychol 2011;41(4):552-561. [http://dx.doi.org/10.1177/008124631104100413] 3. Pajares P. Toward a positive psychology of academic motivation. J Educ Res 2001;95(1):27-35. [http://dx.doi. org/10.1080/00220670109598780] 4. Schunk DH, Meece JL, Pintrich PR. Motivation in Education: Theory, Research and Applications. 4th ed. Boston: Pearson, 2014. 5. Wissing MP, ed. Well-Being Research in South Africa. New York: Springer, 2013. 6. Artino AR, Hemmer PA, Durning SJ. Using self-regulated learning theory to understand the beliefs, emotions, and behaviours of struggling medical students Acad Med 2011;86:S35-S38. [http://dx.doi.org/10.1097/ ACM.0b013e31822a603d] 7. Kusurkar RA, ten Cate THJ, van Asperen M, Croiset M. Motivation as an independent and a dependent variable in medical education: A review of the literature. Med Teach 2011;33:e242-e262. [http://dx.doi.org/10.3109/014 2159X.2011.558539] 8. Opateye JA. The relationship between emotional intelligence, test anxiety, stress, academic success and attitudes of high school students towards electrochemistry. IFE Psychologia 2014;22(1):239-249. 9. Richardson M, Abraham C, Bond R. Psychological correlates of university students’ academic performance: A systematic review and meta-analysis. Psychol Bull 2012;138(2):353-387. [http://dx.doi.org/10.1037/a0026838] 10. Bertrams A, Englert C, Dickhauser O, Baumeister RF. Role of self-control strength in the relation between anxiety and cognitive performance. Emotion 2013;13(4):668-680. [http://dx.doi.org/10.1037/a0031921] 11. Petersen I, Louw L, Dumont K. Adjustment to university and academic performance among disadvantaged students in South Africa. Educ Psychol 2009;29(1):99-115. [http://dx.doi.org/10.1080/01443410802521066] 12. Goodman S, Jaffer T, Keresztesi M, et al. An investigation of the relationship between students’ motivation and academic performance as mediated by effort. S Afr J Psychol 2011;41(3):373-385. [http://dx.doi. org/10.1177/008124631104100311] 13. Sommer M, Dumont K. Psychosocial factors predicting academic performance of students at a historically disadvantaged university. S Afr J Psychol 2011;41(3):386-395. [http://dx.doi.org/10.1177/008124631104100312]  14. Pintrich PR, Smith DAF, Garcia T, McKeachie WJ. A Manual for the Use of the Motivated Strategies for Learning Questionnaire (MSLQ). Ann Arbor, Michigan: National Centre for Research to Improve Post-Secondary Teaching and Learning, 1991. 15. StataCorp. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP, 2013. 16. Sikhwari TD. A study of the relationship between motivation, self-concept and academic achievement of students at a university in Limpopo Province, South Africa. Int J Edu Sci 2014;6(1):19-25. 17. Javanmard A, Hoshmandja M, Ahmadzade L. Investigating the relationship between self-efficacy, cognitive and metacognitive strategies, and academic self-handicapping with academic achievement in male high school students in the tribes of Fars Province. J Life Sci Biomed 2013;3(1):27-34. 18. Karbalaei A. Critical thinking and academic achievement. Medellin–Colombia 2012;17(2):121-128.

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

107

Research Assessing the effect of an online HIV/AIDS course on 1st-year pharmacy students’ knowledge F Suleman, BPharm, MPharm, PhD Discipline of Pharmaceutical Sciences, School of Health Sciences, University of KwaZulu-Natal, Durban, South Africa Corresponding author: F Suleman (sulemanf@ukzn.ac.za)

Background. More international HIV/AIDS intervention initiatives targeting young adults are needed to help reach targets set by the Joint United Nations Programme on HIV and AIDS (UNAIDS). Objectives. To determine the effect of an online HIV/AIDS course on 1st-year pharmacy students’ knowledge of HIV prevention and transmission and of the science of HIV infection. Methods. Online anonymous surveys, the Pre- and Post-Confirmation of Acceptance for Studies, were administered by means of an online survey tool, SurveyMonkey (USA), to 1st-year pharmacy students, from 2009 to 2013. These surveys were administered online during the first and last weeks of class, respectively. Results. With regard to HIV prevention and transmission, student knowledge of the manner in which HIV is transmitted improved during the course. Overall, students were more confident about their knowledge of HIV prevention. They also indicated that as a direct result of taking the AIDS Online International course they were more reluctant to have unsafe sex and more confident about how to protect themselves against HIV/AIDS. Their scientific knowledge also improved. Conclusion. The implementation of an online HIV/AIDS course has proved to be an effective method of HIV/AIDS education, and may also be a viable HIV intervention initiative. Afr J Health Professions Educ 2016;8(1 Suppl 1):108-112. DOI:10.7196/AJHPE.2016.v8i1.750

One of the targets of the Joint United Nations Programme on HIV and AIDS (UNAIDS) 2016 - 2021 plan is the following: ‘90% of young people are empowered with the skills, knowledge and capability to protect themselves from HIV’.[1] Therefore, the development of effective HIV prevention programmes is a top public health and policy priority.[2] However, in spite of calls to increase awareness of the high levels of HIV transmission in young women,[3] particularly in southern Africa,[4] there is little scientific consensus about how best to prevent HIV infection among the youth. Comprehensive sexuality education is considered an important means of addressing adolescent risk behaviours,[5] although little evidence supports its direct effect on biological measures of prevention success, particularly of HIV and other sexually transmitted infections (STIs).[5] In sub-Saharan Africa, experience with youth HIV prevention programmes is limited, with evidence regarding effectiveness still emerging. Prior reviews and studies of youth intervention programmes in both developed and developing countries[2,5,6] suggest an important role for education inter ventions to increase young people’s knowledge of sexuality, reproductive health and HIV prevention, with a majority of interventions leading to reductions in reported risk behaviours. As young people are a target for knowledge and skills development, it is imperative that more HIV/AIDS prevention and education initiatives are developed that target the nation’s youth. This research addressed the effect of an online HIV/AIDS course on student knowledge of HIV prevention and transmission and of the science of HIV infection. In the second semester of 2009, the Discipline of Pharmaceutical Sciences, School of Health Sciences, University of KwaZulu-Natal, Durban, South Africa began to test the concept of online AIDS education by offering an online course on HIV/AIDS in collaboration with Dr Sharron Jenkins of Purdue University, Indiana, USA. The sensitivity of the topics related to HIV/AIDS, such as sexual preferences, practices and behaviours, could be

108

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

quite uncomfortable for students in the traditional classroom setting. In light of these observations, Purdue University began delivering an onlineonly course, providing students the opportunity to have candid, open, and anonymous dialogue about sex and HIV/AIDS. This approach was also adopted at the University of KwaZulu-Natal (UKZN). The primary purpose of this article is to present and discuss the results of an anonymous online course, the Pre- and Post-Confirmation of Acceptance for Studies (Pre-CAS and Post-CAS, respectively), given to students during the first and last weeks of class, respectively. The surveys were designed to assess the overall effect of the AIDS Online International (AOI) course on HIV-related attitudes, beliefs, knowledge and risk behaviours. Only the effect on knowledge is presented in this article.

Methods

The AOI course was offered as an 8-credit mandatory pharmacy module in the second semester of the first year at UKZN. The course was given over 17 weeks (including the examination week). Ninety-seven students took the course in the first year of its implementation. Thereafter, 101 students (2010), 98 students (2011), 113 students (2012) and 104 students (2013) took the course. The AOI course runs online only through a course management system and consists of weekly quizzes, examinations, discussion board activities, animations, and videos/movies to support the learning objectives of the course. In addition, students participate in several online mock activities, including HIV testing, HIV risk assessment, and mandatory online discussions. Embedded into the online delivery of the course, the material and assessments are designed to reflect two theories: the Health Belief Model and Social Cognitive Theory – two well-established models that attempt to explain and predict health behaviours by focusing on the attitudes and beliefs of individuals, including exploring health behaviours associated with HIV transmission.

Research Over a 17-week period, students are given approximately 14 lessons, covering topics such as HIV/AIDS history/origin, statistics, transmission, prevention, testing, and the science of HIV disease progression/opportu足 nistic infections, AIDS diagnosis, vaccines, and antiviral drugs. HIV prevention activities span 2 - 3 weeks. Halfway through the course students participate in a mid足semester activity to help them to assess their personal risk of HIV infection. The activity is a 20足-question survey of behaviours that may place the student and his/her partner at risk of contracting HIV. Students also watch one movie on AIDS history and several online animations related to testing HIV-positive and the science of infection. The online animations provide case studies that help students personalise their own risk of HIV infection. By participating in online mock activities, such as HIV testing and risk assessment, students are able to evaluate their sexual practices and their perceived susceptibility and vulnerability to HIV

infection. The activities also provide students with skills to practise and negotiate safer sex. To assess the effect of the AOI course on participants, online anonymous surveys, the Pre-CAS and Post-CAS, were administered by means of an online survey tool, SurveyMonkey (USA). The surveys were used to assess the knowledge, attitudes, beliefs, and behavioural practices of course participants. The Pre-CAS was administered online during the first week of class and the Post-CAS during the last week of class. Approximately 100 questions were posed to students in each of the 5-year periods. Pre-CAS and Post-CAS statements discussed here cover four main categories: knowledge of HIV transmission/prevention, attitudes/beliefs regarding AIDS-related issues, science of HIV infection, and HIV risk behaviours. Each survey statement allowed students to respond by selecting one of the following: strongly agree, agree, not sure, disagree, strongly disagree, or skip the question. Paired

Table 1. Demographics of students from 2009 to 2013, % Demographics

2009 (n=97)

2010 (n=101)

2011 (n=98)

2012 (n=113)

2013 (n=104)

Pre-CAS Post-CAS

Gender Female

70.10

60.82

61.39

62.38

76.53

80.61

57.52

59.29

76.00

73.00

Male

29.90

25.77

26.73

13.27

15.31

21.24

23.89

24.00

27.00

15 - 19

80.41

69.07

76.24

74.26

81.63

84.69

55.75

61.95

77.00

76.00

20 - 24

19.59

17.53

9.90

12.87

5.10

8.16

20.35

19.47

24.00

22.00

Age categories, years

Marital status Single

69.07

64.95

62.38

69.31

73.47

83.67

58.41

63.72

77.90

77.00

S ingle, but in mono足gamous relationship

27.84

17.53

22.77

16.83

13.27

9.18

17.70

18.58

19.20

22.00

S ingle with more than one sex partner

3.09

4.12

1.98

1.10

0.00

1.90

0.00

1.90

0.00

Married

0.00

1.10

2.30

3.20

0.00

1.00

eterosexual male (sex with H women only)

26.80

24.74

23.76

13.27

12.24

22.12

23.89

25.00

27.00

omosexual male (sex with H men only)

7.22

2.06

5.94

0.99

6.12

5.10

3.54

2.65

3.80

4.00

isexual male (sex with both B men and women)

1.03

0.00

0.99

0.00

eterosexual female (sex with H men only)

58.76

57.73

56.44

62.38

68.37

72.45

51.33

54.87

68.30

67.00

Sexual preference

Living arrangements Renting house/apartment

27.84

20.79

11.22

12.24

35.40

36.28

41.30

44.00

Own house/apartment

4.10

7.10

9.70

2.20

11.22

10.20

5.80

5.00

5.80

5.00

S taying with family/friends who rent house/apartment

12.40

3.60

5.40

14.30

17.35

12.24

8.70

8.00

8.70

8.00

S taying with family/friends who own house/apartment

53.61

48.45

50.50

51.49

45.92

58.16

28.32

32.74

41.30

41.00

70.10

64.95

65.35

61.39

69.39

82.65

45.13

51.33

61.50

63.00

Access to a regular doctor, nurse or health practitioner Yes

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

109

110

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

2.00

2.04

1.27

1.62

I am reluctant to have unsafe sex because of what I’ve learned from this course

IV tests generally test for H HIV antibodies and not for the virus

n HIV test during the A ‘window period’ could result in a false-negative result

I am aware of at least one HIV/AIDS support group in my area

en who have sex with M men (homosexual men)

1.00

1.38

1.53

1.89

eep kissing or ‘French D kissing’ an HIV-infected person

1.05

1.09

1.24

1.03

1.08

1.43

1.42

1.16

eceiving breast milk R from an HIV-infected mother (mother-to-child transmission)

Which of the following groups of people would be at risk for contracting HIV? Choose all that apply

1.43

ral sex with an HIVO infected person

2.05

Is it possible to contract or transmit HIV through the following ways?

2009 (n=97)

Pre-CAS Post-CAS

I am confident that I know the correct and most effective way to use a condom

Do you agree or disagree with the following statements?

Knowledge about HIV transmission/prevention

0.0000

0.0601*

0.0025

0.0001

0.0049

0.0000

0.0010

0.0029

p-value

1.24

1.99

1.27

1.40

1.57

1.28

1.76

1.87

2.02

Pre-CAS

1.05

1.39

1.06

1.08

1.23

1.14

1.17

1.40

1.30

0.0086

0.0000

0.0072

0.0022

0.0020

0.1017*

0.0000

0.0101

0.0000

Post-CAS p-value

2010 (n=101)

Table 2. Knowledge about HIV/AIDS (as a mean score) from 2009 to 2013, with p-values

1.41

2.00

1.10

1.37

1.73

1.31

1.71

2.06

2.39

Pre-CAS

1.02

1.26

1.02

1.04

1.50

1.14

1.21

1.43

1.32

Post-CAS

2011 (n=98)

0.0000

0.1344*

0.0000

0.1399*

0.0958*

0.0004

0.0005

0.0000

p-value

1.32

2.08

1.33

1.34

1.68

1.39

2.13

2.15

2.44

Pre-CAS

1.04

1.40

1.07

1.11

1.44

1.04

1.20

1.51

1.53

Post-CAS

2012 (n=113)

0.0008

0.0000

0.0008

0.0064

0.0208

0.0005

0.0000

0.0034

0.0000

p-value

3.06

2.07

2.29

Pre-CAS

3.46

1.64

1.44

0.0395

0.0052

0.0273

0.0304

0.0505*

0.0008

0.0000

0.2604*

0.0106

p-value

Continued …

Post-CAS

2013 (n=104)

Research

1.53

1.85

People who inject drugs

person who French kisses A a person with HIV infection

2.48

ouching after sex can D prevent HIV transmission

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

1.66

1.38

2.47

1.44

1.87

2.05

positive HIV test means A that the body is producing antibodies to HIV

A CD4 cell is an immune cell

here are two main types T of HIV, three main groups, and several subtypes

everse transcription is a R step in the HIV life cycle

Integrase, reverse transcriptase, and protease are HIV enzymes

ntiviral drugs such as A zidovudine (AZT) and didanosine (ddI) are reverse transcriptase inhibitors

* Non-significant p-values.

1.33

S ome antiviral drugs can help prevent mother-tochild transmission of HIV

Do you agree or disagree with the following statements?

2009 (n=97)

2.01

1.24

1.19

1.38

1.02

1.01

1.16

1.01

1.00

1.02

1.01

1.02

1.08

Pre-CAS Post-CAS

1.77

lobally, most people G contract HIV through heterosexual contact

Knowledge of the science of HIV

1.87

I know all the ways to prevent HIV transmission

1.44

Do you agree or disagree with the following statements?

2009 (n=97)

Pre-CAS Post-CAS

eople who have had a P sexually transmitted disease

Knowledge about HIV transmission/prevention

0.0000

0.0134

p-value

0.0000

p-value

2.11

1.51

1.19

2.35

1.42

1.64

1.47

Pre-CAS

2.44

1.80

1.62

1.93

1.19

1.26

Pre-CAS

0.0000

0.0005

0.0000

0.0271

0.0027

2.41

1.92

1.65

2.01

1.21

1.18

Pre-CAS

2011 (n=98)

2.04

1.17

1.10

1.20

1.03

Post-CAS

2011 (n=98)

1.16

1.06

1.07

1.05

1.03

1.08

0.0000

0.0001

0.0963*

0.0000

0.0005

0.0000

0.0001

2.21

1.60

1.21

2.47

1.44

1.52

1.30

1.24

1.03

1.05

1.08

1.04

1.12

1.15

Post-CAS p-value Pre-CAS Post-CAS

2010 (n=101)

2.03

1.14

1.18

1.38

1.06

1.03

Post-CAS p-value

2010 (n=101)

Table 2. (continued) Knowledge about HIV/AIDS (as a mean score) from 2009 to 2013, with p-values

0.0000

0.0030

0.0000

0.0424

p-value

0.0000

0.0069

0.0550

p-value

2012 (n=113)

2.06

1.24

1.36

1.38

1.07

1.08

Post-CAS

2012 (n=113)

0.0040

0.0005

0.0000

0.3203*

0.0234

p-value

3.06

Pre-CAS

1.90

1.63

1.45

2.23

1.37

1.66

1.35

1.33

1.07

1.11

1.06

1.11

1.12

0.0000

0.0001

0.0000

0.0007

0.0000

0.0115

3.06

Pre-CAS Post-CAS p-value Pre-CAS

2.30

1.60

1.85

1.98

1.17

1.26

Pre-CAS

0.0001

0.0149

0.0035

0.0003

0.0046

0.0056

p-value

3.46

0.0026

0.0006

0.0079

0.0000

0.0089

0.0000

0.0095

Post-CAS p-value

2013 (n=104)

3.46

Post-CAS

2013 (n=104)

Research

111

Research t-tests were calculated for each year to determine if there were significant differences between the Pre-CAS and Post-CAS statements. Ethical clearance to administer and analyse the questionnaire was obtained from UKZN. Informed consent was obtained from the students, who were aware that they could volunteer for the survey and withdraw from it at any time. Anonymity was maintained and no identifying information on the student was obtained (except gender and age).

Results

The data presented in Tables 1 and 2 are results of the Pre-CAS and PostCAS surveys from 2009 to 2013. Table 1 presents the demographics of the students over the 5-year period. The majority of students were female and single. Most were in the 15 - 19-year age group and in most instances stayed with family or friends. Most students reported being heterosexual and had access to a regular doctor or nurse. In terms of knowledge of HIV prevention and transmission, student knowledge on the methods of HIV transmission improved during the course (Table 2). For instance, correct identification of HIV transmission increased in the Post-CAS for oral sex, mother-to-child transmission during childbirth, and mother-to-child transmission via breastfeeding. Overall, students were more confident in their knowledge of HIV prevention. They also indicated that as a direct result of taking the AOI course they were more reluctant to have unsafe sex and more confident about how to protect themselves. Knowledge of the science of the disease and of medication therapy improved. There were some differences in cohort responses over the 5-year period. Of concern is the 2013 cohort's response to the course in terms of ‘I am reluctant to have unsafe sex because of what I’ve learned from this course’ (p=0.2604), which might indicate that risky behaviour was being practised. This group also seemed unsure if HIV could be transmitted via breastmilk. They had very different responses from the group of the previous year (in terms of mean scores). As this was an online anonymous survey, further investigation as to the reason for the difference could not be undertaken.

Discussion

Although most students were reasonably knowledgeable about HIV trans mission and prevention before the course, Pre-CAS and Post-CAS data indicate that they were more confident about their knowledge after the course. The AOI course did have a significant effect in helping students to understand the science of HIV disease. It is important, however, to continue to investigate if the 2013 cohort responses were an anomaly or if there are other reasons for youth to respond to the questions in such a manner. It could be that there are now too many messages being provided through different media, which creates confusion. This topic must be further researched. Studies found that although students have reasonable knowledge about HIV prevention measures, they do not make behavioural changes because: (i) they lack the technical and/or communication skills to practise safe sex; (ii) they do not personalise the risk by separating themselves from the issue; and (iii) they do not socialise or have discussions about safe sex as there is a homosexual stigma associated with AIDS.[7,8] Therefore, to translate knowledge about HIV into risk-reducing behaviours, prevention initiatives must be able to personalise the risk of HIV infection, the seriousness of becoming infected, the benefits of practising safer sex, and the skills learned so that there is self-efficacy to practise them.[9] The AOI course is designed to reflect these theories by incorporating into the curriculum activities intended to influence specific beliefs towards safer sexual practices.

112

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

Recent studies indicate that computerised, online or internet-based HIV prevention initiatives may be a viable way to promote HIV education, awareness and prevention skills.[10,11] The results of this study suggest that an online course on AIDS may be a viable and effective way to influence students’ knowledge related to HIV/AIDS and HIV risk behaviours. Furthermore, the AOI course proved to be an effective HIV prevention initiative with the potential to make a significant international contribution to HIV prevention initiatives – educating large numbers of young adults about preventing the spread of HIV. This study had a higher proportion of female than male students, which could have affected the responses to the survey, and might be a limitation of this study. A Ghanaian study in 2012,[12] which had a more equitable distribution of male and female students, found that the female students had significantly (p=0.017) more knowledge about HIV/AIDS than their male counterparts. The results were similar to those of studies conducted in Nigeria,[13,14] which found that AIDS knowledge differs on the basis of gender among university students.

Conclusion

The online HIV/AIDS course provided a safe and anonymous environment for students to acquire the knowledge necessary to understand the science of the disease and knowledge on HIV prevention and transmission in terms of their own attitude and behaviour. It can be used across disciplines in the health sciences as a way to engage students in discussions on risky behaviours and provide them with information to protect themselves against infection, or on the management of the infection. Results from UKZN cohorts will need to be compared with cohorts at other universities across the globe that are teaching the AOI course, and to assess whether a class with more male students would alter these results. However, results suggest that this could be a good intervention for higher education. Acknowledgements. Dr S Jenkins of Purdue University, Indiana, USA is acknowledged for access to and use of the AOI course. This article was made possible by grant No. 5R24TW008863 from the US President’s Emergency Plan for AIDS Relief (PEPFAR) and the National Institutes of Health (NIH), US Department of Health and Human Services. Its contents are solely the responsibility of the author and the UKZN MEPI programme and do not necessarily represent the official views of the US government. References 1. Joint United Nations Programme on HIV and AIDS (UNAIDS). AIDS by the Numbers. Geneva: UNAIDS, 2015. http:// www.unaids.org/sites/default/files/media_asset/AIDS_by_the_numbers_2015_en.pdf (accessed 29 February 2016). 2. Ross D, Dick B, Ferguson J. Preventing HIV/AIDS in Young People: A Systematic Review of the Evidence From Developing Countries. Report of the UNAIDS Inter-agency Task Team on Young People. World Health Organization (WHO) Technical Report Series 938, 2006. Geneva: WHO, 2006. 3. Laga M, Schwärtlander B, Pisani E, Sow PS, Caraël M. To stem HIV in Africa, prevent transmission to young women. AIDS 2001;15:931-934. 4. Stirling M, Rees H, Kasedde S, Hankins C. Addressing the vulnerability of young women and girls to stop the HIV epidemic in southern Africa. AIDS 2008;22:S1-S3. [http://dx.doi.org/10.1097/01.aids.0000341772.48382.57] 5. Ibrahim N, Rampal L, Jamil Z, Zain AM. Effectiveness of peer-led education on knowledge, attitude and risk behaviour practices related to HIV among students at a Malaysian public university – a randomized controlled trial. Prev Med 2012;55:505-510. 6. Oppong AK, Oti-Boadi M. HIV/AIDS knowledge among undergraduate university students: Implications for health education programs in Ghana. Afr Health Sci 2013;13(2):270-277. [http://dx.doi.org/10.4314/ahs.v13i2.11] 7. Oakley A, Fullerton D, Holland J, et al. Sexual health education interventions for young people: A methodological review. Br Med J 1995;310:158-162. 8. Mkumboa K. Assessment of HIV/AIDS knowledge, attitudes and behaviours among students in higher education in Tanzania. Glob Public Health 2013;8(10):1168-1179. [http://dx.doi.org/10.1080/17441692.2013.837498] 9. Heeren GA, Jemmott GB III, Ngwane Z, Mandeya A, Tyler JC. A Randomized controlled pilot study of an HIV riskreduction intervention for sub-Saharan African university students. AIDS and Behaviour 2013;17(3):1105-1115. 10. Catalani C, Philbrick W, Fraser H, Mechael P, Israelski DM. mHealth for HIV treatment and prevention: A systematic review of the literature. Open AIDS J 2013;7:17-41. [http://dx.doi.org/10.2174/1874613620130812003] 11. Schnall R, Travers J, Rojas M, Carballo-Diéguez A. eHealth interventions for HIV prevention in high-risk men who have sex with men: A systematic review. J Med Internet Res 2014;16(5):e134. [http://dx.doi.org/10.2196/jmir.3393] 12. Oppong Asante K, Oti-Boadi M. HIV/AIDS knowledge among undergraduate university students: Implications for health education programs in Ghana. Afr Health Sci 2013;13(2):270-277. [http://dx.doi.org/10.4314/ahs.v13i2.11] 13. Aluede O, Imhonde H, Maliki A, Alutu A. Assessing Nigerian university students' knowledge about HIV/AIDS. J Soc Sci 2005;11:207-213. 14. Chng L, Eke-Huber E, Eaddy S, Collins J. Nigerian college students: HIV knowledge, perceived susceptibility for HIV and sexual behaviours. College Student Journal 2005;39(1):60-71.

Research Feedback as a means to improve clinical competencies: Consultants’ perceptions of the quality of feedback given to registrars C I Bagwandeen,1 MB ChB, DHSM, DOH, Dip HIV Man (SA), FCPHM; V S Singaram,2 BMedSc, MMedSc, PhD 1

Discipline of Public Health, School of Nursing and Public Health, College of Health Sciences, University of KwaZulu-Natal, Durban, South Africa

Clinical and Professional Practice, School of Clinical Medicine, College of Health Sciences, University of KwaZulu-Natal, Durban, South Africa

Corresponding author: C I Bagwandeen (bagwandeenc@ukzn.ac.za)

Background. Effective supervision by consultants in postgraduate medical education involves the process of feedback. Giving feedback may be challenging for consultants who have no formal training in this process, which may be further compounded in heterogeneous diverse settings. Objective. To explore consultants’ perceptions of feedback to registrars in a multicultural, multilingual diverse academic hospital setting. Methods. Thirty-seven consultants consented to completing a questionnaire on what, when, where, how often, and how feedback was provided, as well as on the type and effect of feedback to registrars. Descriptive statistics were used to analyse the data. Differences between groups were calculated using Pearson’s χ² test for independent variables, with a p-value of <0.05 regarded as being statistically significant. Results. Only 40% of consultants reported that they provided feedback often or always and 62.2% reported that standards were not predetermined and communicated to registrars. When feedback was provided, it was based on concrete observations of performance (78.4%), it incorporated a plan for improvement (72.9%) and it supplied information on techniques performed incorrectly (72.9%). Only 40.5% of consultants provided feedback on procedures performed correctly. Moreover, only half of the consultants believed they were proficient at giving feedback. Conclusion. Consultants need to develop the art of giving feedback through appropriate training so that they are more comfortable and proficient with the various aspects of feedback, leading to a positive effect on enhancing registrar training. Afr J Health Professions Educ 2016;8(1 Suppl 1):113-116. DOI:10.7196/AJHPE.2016.v8i1.758

Effective supervision in postgraduate medical education involves the process of feedback, which may – in practice – be ineffective or inconsistent.[1] Based on the literature,[2–6] this study defines feedback as ‘a process whereby the desired standard of proficiency in a task has been clearly established. This standard has been communicated to the student. Gaps in performing the task or level of knowledge are identified, and the student is made aware of his or her shortcomings, together with a plan to improve performance.’ As medical education has moved beyond the paradigm of ‘see one, do one, teach one’,[7] consultants need to be capable of providing suitable training guidance to ensure that graduates are clinically and otherwise competent. This should have a positive effect on patient outcomes, foster a life-long love of learning and the process of reflection, and promote good ethical practice. This process involves more than didactic input. It includes suitable feedback, so that competencies may be enhanced and improved and deficiencies corrected.[8] It might be argued that if some (or all) of the elements contained in the definition are missing, feedback is not being adequately provided in clinical settings, thus affecting the calibre of specialists subsequently produced. Feedback has been well recognised as an important component of edu cation and can have an extremely powerful and positive effect on learning.[2,8,9] It is regarded as integral and essential to postgraduate medical education,[4] a concept that is similar to that of serving an old-fashioned apprenticeship in an experiential learning setting. Without feedback, poor performance is not corrected, good performance is not entrenched and magnified, and no plans for improvement are implemented.[10,11] Feedback that meets all the defined criteria can positively influence the performance of doctors.[12] The importance of suitable external feedback by consultants to registrars becomes critical when there is no self-assessment by registrars or if the feedback is inaccurate.[13] Giving feedback may be challenging for consultants who have

no formal training in the process, which may be further compounded in heterogeneous settings involving students of different gender, ethnicity, race, socioeconomic backgrounds, educational levels and home or first languages.[14] Consultants need to be sensitive to the different dynamics at play to ensure that the same message ‘transmitted is received and under stood’[15] by the different groups in the same way. Furthermore, several authors have reported that consultants often believe that they provide adequate, timeous and sufficient quality feedback, despite evidence from registrars indicating the contrary.[4,8,10,13] Given the importance of feedback as an essential component of medical education, this discrepancy is of great concern and needs to be monitored. Hence, this study was undertaken to explore the consultants’ and registrars’ perceptions of feedback given and received. The study focuses on the perceptions of consultants with regard to the quality of feedback they provided to registrars employed at an academic hospital.

Methods

A questionnaire was designed to ascertain the consultants’ perceptions on what, when, where, how often, and how feedback was provided, as well as on the type and effect of feedback to registrars. Sociodemographic information (age, gender, home language, discipline and years of specialisation) was also gathered. A definition of feedback, as discussed above, was also included in the questionnaire to try to prevent any misconceptions with regard to the basic tenets of this process. Responses were reported on a 5-point Likert scale (Figs 1 and 2). Although this observational study adopted a mixed-methods approach, this article focuses on the quantitative data used to survey the overall perceptions of the consultants. The qualitative investigation of these perceptions will be reported in a follow-up study.

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

113

Research 24.3 2.7

I would like to receive peer feedback 40.5

27.0

16.2

27.0

16.2

The registrar agrees with the feedback provided

27.0

8.1

I prefer giving group feedback

43.2

My feedback sessions are always successful − the registrar receives the intended message in the intended manner

37.8

I am proficient at giving feedback to my registrars

29.7

Support is available to the registrar from different sources after both formal and informal feedback sessions 43.2

The effect of feedback on the registrar noted 37.8

Feedback is documented

40.5

16.2 13.5

37.8

2.7

24.3

32.4

Feedback encourages reflection about previous feedback 45.9

2.7 18.9

56.8

Feedback is formal

24.3

100.0

50.0 Rarely

Never

27.0

21.6

27.0 62.2

Feedback is informal 2.7 32.4

51.4

27.0

2.713.5

Feedback is provided in all encounters with a registrar

45.9

16.2 21.6

18.9

24.3 5.4 27.0

43.2 29.7

27.0

Feedback is based on concrete observations of the performance of the registrar 18.9 2.7 29.7

45.9

10.8 13.5

2.710.8 10.8

40.5

54.1

27.0

21.6 2.7 2.7

Feedback is given about procedures and techniques performed incorrectly

Formal feedback sessions are held in an appropriate location

27.0

18.9

Feedback incorporates a plan of improvement 18.9 5.4

Formal feedback sessions are clearly scheduled in advance

40.5

43.2

10.8 5.4

Feedback is given in non-emotive, non-judgemental language

Standards for assessment are predetermined and communicated to the registrar in advance

29.7

8.1 10.8

24.3 5.4

75.7

Feedback is given about procedures and techniques performed correctly

48.6

5.4 13.5

37.8

Formal feedback incorporates new learning objectives Feedback is influenced by race, gender or ethnicity of the registrar 5.4

8.1

40.5

The registrar has an opportunity to respond to the feedback given

64.9

5.4 13.5 24.3

2.7

45.9

5.4 10.8 0.0.0

Sometimes

32.4

48.6 50.0

Always

Often

80.0

100.0

Fig. 1. A divergent stacked bar graph showing consultants’ perceptions on the feedback they give to registrars. 29.7

Feedback is given about how to be a professional

45.9

Feedback is given about how to be a health advocate

43.2

Feedback is given about how to be a manager

Feedback is given about how to be a communicator

5.4

43.2

Feedback is given about how to be a medical expert 80.0

60.0

Rarely

2.7 16.2

2.7

10.8 21.6

48.6

Feedback is given about how to be a collaborator

16.2

8.1 2.7 2.7

48.6

Feedback is given about how to be a scholar

100.0

2.7 8.1

32.4 40.0 Never

43.2 37.8 32.4

24.3

21.6

2.7 21.6

5.4 16.2

2.7

32.4

13.5 10.8 2.7

35.1

20.0

0.0.0

Sometimes

Always

40.0

60.0

Often

Fig. 2. A divergent stacked bar graph showing consultants’ perceptions on the feedback they give to registrars with regard to graduate attributes.

All consultants from the disciplines of Surgery, Internal Medicine, Obstetrics and Gynaecology, Paediatrics, Psychiatry and Family Medicine were invited to participate. Because of a 0% response rate to the online questionnaire, hard copies were distributed at academic day meetings; 62% (n=37) were returned anonymously with informed con sent. Descriptive statistics were used to interpret the responses of the registrars, with mean values calculated. Differences between groups were calculated using Pearson’s χ² test for independent variables, with a p-value of <0.05 regarded as sta-

114

tistically significant. Ethical approval for the study was granted by the Humanities and Social Sciences Ethical Committee, University of KwaZulu-Natal, Durban, South Africa (HSS/1185/013D).

Results

The mean age of the consultants was 37.8 (range 31 55) years. Fifty-four percent were female. English was the first language for the majority of consultants (n=31), while six spoke another language. Six consultants had qualifications other than the Colleges of Medicine of South Africa Fellowship

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

in their respective fields: 1 had a PhD in Surgery, while the other 5 had postgraduate certificates in their respective fields. Thirteen consultants from Paediatrics, 9 from Internal Medicine, 8 from Obstetrics and Gynaecology, 3 from Surgery and 2 each from Psychiatry and Family Medicine responded. As illustrated in Fig. 1, 62.2% of consultants reported that standards for assessment were not predetermined and communicated to registrars in advance – always or often. All consultants reported that they provided feedback, but only

Research ~40% provided feedback always or often. The majority of consultants based their feedback on concrete observations of registrar performance (78.4%), incorporated a plan for improvement in their feedback (72.9%), or provided feedback on techniques performed incorrectly (72.9%). However, only 40.5% provided feedback on procedures performed correctly, while 56.7% thought that feedback encouraged reflection about previous feedback. The vast majority gave informal feedback (94.6%). Only 27% gave formal feedback that was clearly scheduled in advance, given in an appropriate location and that incorporated new learning objectives – only half of the time. Feedback given was influenced by race, gender or ethnicity of the registrars – sometimes (10.8%) and often (5.4%). Seventy-three percent reported that feedback was given using non-emotive, non-judgemental language, and 70.2% of consultants gave registrars an opportunity to respond to feedback, but only 32.4% noted the effect of feedback on the registrar. A total of 51.3% reported that support for registrars was available after feedback. While 73% of the consultants felt that the registrar agreed with the feedback, 54% reported that they were proficient at giving feedback and believed that their intended message was received. Most consultants (83.7%) preferred giving feedback one on one, and would have liked to receive peer feedback (72.9%). Registrars need feedback on both technical and other specific skills and on graduate attributes to improve outcomes. With regard to technical skills, feedback on how to be a professional was provided always or often (59.4%), while the specifics around being a medical expert (56.7%), communicator (64.8%), collaborator (75.6%), manager (75.6%), health advocate (64.8%), and scholar (59.4%) were neglected. Sixty-two percent of consultants believed that they always or often provided feedback about clinical skills, technical skills and evidence-based practice, but feedback about inter personal skills (67.6%), communication skills (59.5%) and ethics (54%) was rarely or never provided. This study found that consultants delivered feedback in a variety of settings. While no consultants provided feedback during group teaching, 84% gave feedback during academic days, 62% made use of side-room settings, 50% provided one-on-one feedback, and 32% gave feedback at the bedside. No statistically significant relationship was observed between the age of consultants and how they perceived feedback to be provided. With regard to gender, male consultants believed that they were proficient at giving feedback, significantly more than their female counterparts (p=0.041, mean 21.91). Consultants whose mother tongue was English showed significant differences compared with other language speakers, as they gave more feedback about how to be a communicator (p=0.031, mean 20.58), a colla borator (p=0.017, mean 20.74) and a manager (p=0.052, mean 20.44). Provision of feedback was significantly influenced by race, gender and ethnicity of registrars, more so in consultants who were English secondlanguage speakers (p=0.05, mean 27.58) than English first-language speakers.

Discussion

The importance of providing feedback in registrar training has been well documented.[1-5] A good approach to feedback is essential and several necessary elements have been identified for successful feedback to occur and ensure that the process attains the desired end result of improving performance.[2,4,16,17] Two of the fundamental requirements for an appropriate and adequate feedback process involve: (i) the development of the desired stan-

dards to be obtained, and for these standards to be clearly communicated to the registrar in advance; and (ii) that the consultant’s feedback be based on direct observation of the registrar’s performance and compared with the desired standard to be achieved. Such feedback must include an improvement plan to overcome any deficiencies between actual and desired performance.[3,4] The majority of consultants did not communicate such desired ‘gold standards’ to the registrars. Furthermore, more than one-quarter did not base their observations on direct observation of performance or provide a plan for improvement in the feedback given. Therefore, registrars did not always have a clearly defined set of rules as a benchmark. These findings highlight essential fundamental flaws in the current practice of feedback across the disciplines at our academic hospital. This study suggests that each department should develop a set of guidelines that should be given to registrars at the beginning of a rotation, and consultants should be made aware that the feedback process hinges on direct observation of performance and incorporates improvement plans. Moreover, while feedback is used to correct deficiencies, it should also enhance good performance. The majority of consultants did not give feedback on procedures performed correctly, hence missing the opportunity to cement good practice.[5] All consultants reported that they provided feedback, but the majority provided it infrequently and informally. With this approach, registrars will not always recognise feedback as feedback, and may not pay as much attention to it as when it is formally scheduled in advance.[1] Owing to the experiential nature of the clinical teaching setting, it is of concern that consultants do not optimise all opportunities with the registrar to provide feedback. This is an indication that many teaching opportunities are being lost. Many consultants provided teaching at the bedside, a valuable setting for practical demonstration of clinical skills. However, academic days, which could be the best time for emphasising ‘softer’ skills, including graduate attributes, professionalism and ethics, were not maximised. A specific time should be set aside for discussions around such aspects on these days. Also, care should be taken to highlight the relevant application of such tenets during case presentations or didactic lectures on the effect that key areas have on clinical care to ensure that graduates are equipped with more than clinical competencies.[17] As feedback has been likened to giving bad news, the effect of the message on the recipient cannot be ignored.[5] This is particularly important in the diverse multicultural setting of this study. Of note, the majority of consultants were not influenced by the race, gender or ethnicity of the registrar. However, this issue needs to be addressed, as not all consultants reported that feedback was given in a non-emotive and non-judgemental way. This is a major concern and counteracts the purpose of giving feedback – to improve performance – as registrars should not be in a position of reacting to how something was being said, rather than what was being said, and so losing the intended message.[5] Similarly, not noting the effect of consultants’ feedback on registrars, could have a harmful result. In the face of negative criticism, some registrars lack the emotional capacity to recover from this and may flounder in their attempts to improve on their performance.[5,18] Conversely, others, particularly those with strong personalities, may choose to believe that their consultants are incorrect and persist in their chosen behaviour.[18] It is therefore vital not only to be cognizant of the effect of both formal and informal feedback, but also to ensure referral to appropriate support structures should these be required; however, only 45.9% of consultants were aware of the support structures that registrars could access or be

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

115

Research referred to. While the majority of consultants gave registrars an opportunity to respond to feedback, when this did not occur misunderstandings and misconceptions were not clarified. Consultants and students need to be skilled in the art of giving feedback.[2,4,6] Only half of the consultants felt that they were proficient in providing feedback and gave feedback often. Also, less than a third gave feedback about technical skills. As providing feedback is key to improving academic outcomes and clinical proficiency, inadequacies in being able to provide feedback generally and about essential competencies highlight a gap in the key performance areas of consultants and indicate the need for staff development, in addition to a possible postgraduate clinical qualification for employment in an academic teaching hospital. It is encouraging that all consultants agreed that feedback was essential to registrar training and the vast majority felt that they should be trained to give feedback. Consultants are aware of the importance of feedback in honing relevant skills and of their own deficiencies and the need to rectify these through appropriate training. The race, gender and ethnicity of the registrars affected the provision of feedback significantly more for English second-language consultants than for English first-language speakers. The latter consultants were probably more aware of the barriers that non-proficiency in the medium of instruction could pose and took care to overcome them. Conversely, given that all communication between registrar and consultant is in English, the consultants for whom English was their home language believed that they were skilled in providing feedback, possibly because of their ease of use of the language. Consequently, they did not pay as much attention to ensuring that feedback was as successful as it should be, especially for registrars who were not as proficient in English as they were. However, regardless of race or language, generally male consult ants believed that they were more proficient at providing feedback than female consultants.

Conclusion

The study found that the art of giving and receiving feedback has to be nurtured so that consultants are more comfortable with and proficient in the process, not only in specific skills, but also with regard to essential graduate outcomes. To train consultants in this process would entail a form of continuing professional development, especially as they are recruited on their clinical skills and the assumption that knowing how to do a procedure equates to being able to communicate it well, without any formal exposure to didactic instruction. This would encourage a process of reflection and seeking feedback from registrars, starting in the preclinical years. An integral

116

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

component of this training would have to be recognising the effect of feedback on registrars, so that any undesirable outcomes could be appropriately dealt with, be it refusal to accept the feedback or negative emotional reactions. Support structures and mechanisms must be developed internally by disciplines and the university at large, and referral pathways must be developed and communicated to consultants and registrars so that they are able to access these quickly and confidentially if and when required. While it is gratifying that most consultants were able to embrace the multicultural and diverse setting, a small majority appeared to be affected by race, gender and ethnicity. We recommend that appropriate programmes addressing diversity issues be implemented so that no-one is prejudiced by these apparent biases. Acknowledgements. This article was made possible by grant No. R24TW008863 from the Office of the US Global AIDS Coordinator and the US Department of Health and Human Services, National Institutes of Health (NIH OAR and NIH ORWH). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the US government. We wish to thank Dr M Muzigaba for help with statistical analysis, and the staff and students who participated in this study. References 1. Busari JO, Weggelaar N, Knottnerus AC, Greidanus P, Scherpbier AJJA. How medical residents perceive the quality of supervision provided by attending doctors in the clinical setting. Med Educ 2005;39:696-703. 2. Hattie J, Timperley H. The power of feedback. Rev Educ Res 2007;77:81-112. 3. Van de Ridder JM, Stokking K, McGaghie W, ten Cate O. What is feedback in clinical education? Med Educ 2008;42:189-197. [http://dx.doi.org/10.1111/j.1365-2923.2007.02973.x] 4. Ende J. Feedback in medical education. JAMA 1983;250:777-781. 5. DeLima TJ, Arnold R. Giving feedback. J Palliat Med 2011;14(2):233-239. [http://dx.doi.org/10.1089/ jpm.2010.0093] 6. Shrivastava SR, Shrivastava PS, Ramasamy J. Effective feedback: An indispensable tool for improvement in quality of medical education. J Pedagog Dev 2014;4(1):12-20. 7. Rodriguez-Paz JM, Kennedy E, Salas E, et al. Beyond ‘see one, do one, teach one’: Toward a different training paradigm. Qual Saf Health Care 2009;18:63-68. [http://dx.doi.org/10.1136/qshc.2007.023903] 8. Menachery EP, Knight AM, Kolodner K, Wright SM. Physician characteristics associated with proficiency in feedback skills. J Gen Intern Med 2006;21:440-446. 9. Norcini J. The power of feedback. Med Educ 2010;44:16-17. [http://dx.doi.org/10.1111/j.1365-2923.2009.03542.x] 10. Lieberman A, Lieberman M, Steinert Y, McLeod P, Meterissian S. Surgery residents and attending surgeons have different perceptions of feedback. Med Teach 2005;27:470-472. 11. Cantillion P. Giving feedback in clinical settings. BMJ 2008;337:a1961. [http://dx.doi.org/10.1136/bmj.a1961] 12. Veloski J, Boex J, Grasberger M, Evans A, Wolfson D. Systematic review of the literature on assessment, feedback and physicians’ clinical performance: BEME Guide No. 7. Med Teach 2006;28:117-128. 13. Kogan J, Conforti LN, Bernabeo EC, Durning SJ, Hauer KE, Holmboe ES. Faculty staff perceptions of feedback to residents after direct observation of clinical skills. Med Educ 2012;46:201-245. [http://dx.doi.org/10.1111/j.13652923.2011.04137.x] 14. Holen A. The PBL group: Self-reflections and feedback for improved learning and growth. Med Teach 2000;22:485-488. 15. Murdoch-Eaton D. Feedback: The complexity of self-perception and the transition from ‘transmit’ to ‘received and understood’. Med Educ 2012;46:538-540. [http://dx.doi.org/10.1111/j.1365-2923.2012.04278.x] 16. Bing-You R, Trowbridge R. Why medical educators may be failing at feedback. JAMA 2009;32:1330-1331. [http:// dx.doi.org/10.1001/jama.2009.1393] 17. Fluit C, Bolhuis S, Grol R, et al. Evaluation and feedback for effective clinical teaching in postgraduate medical education: Validation of an assessment instrument incorporating the CanMEDS roles. Med Teach 2012;34:893-101. 18. Archer J. State of the science in health professional education: Effective feedback. Med Educ 2010;44:101-108. [http://dx.doi.org/10.1111/j.1365-2923.2009.03546]

Research Feedback as a means to improve clinical competencies: Registrars’ perceptions of the quality of feedback provided by consultants in an academic hospital setting C I Bagwandeen,1 MB ChB, DHSM, DOH, Dip HIV Man (SA), FCPHM; V S Singaram,2 BMedSc, MMedSc, PhD Discipline of Public Health, School of Nursing and Public Health, College of Health Sciences, University of KwaZulu-Natal, Durban, South Africa

Clinical and Professional Practice, School of Clinical Medicine, College of Health Sciences, University of KwaZulu-Natal, Durban, South Africa

Corresponding author: C I Bagwandeen (bagwandeenc@ukzn.ac.za)

Background. The training of novices, in any field, to achieve the optimum state of cognitive, clinical, technical and professional development requires a variety of teaching methodologies, including the process of feedback. Feedback is defined as a process where the desired standard of proficiency in a task has already been established and communicated to the student before gaps in performing the task or in the level of knowledge are identified. The process of feedback has often been evaluated and has consistently revealed students’ dissatisfaction with the amount and type of feedback they receive in their clinical and postgraduate training, as they perceive it to be inadequate, inappropriate or non-existent. Objectives. To investigate the perceptions of the quality of feedback received by a diverse, heterogeneous population of registrars in postgraduate training at an academic hospital. Methods. A study was conducted using a questionnaire to determine the perceptions of all registrars in the six major clinical training programmes with regard to the quality, efficacy and effectiveness of feedback received during clinical training. Descriptive statistics were used to interpret the responses of the registrars, with mean values being calculated. Results. Perceptions of the quality of feedback received differed across disciplines. Overall, the registrars rated the feedback they received as poor. The majority (51.4%) reported that both formal and informal feedback was only sometimes, even rarely, received during all encounters with consultants. Others (51.3%) felt that the feedback received was unacceptable, and did not perceive it to be based on concrete observations of performance. The proficiency of consultants in giving feedback was scored as unacceptable by 64.8% of registrars. Conclusion. Registrars in training regard feedback as an essential component of their postgraduate medical education and as an important component of achieving clinical competence. More formalised processes need to be implemented. The majority of registrars agreed that consultants required training in providing feedback effectively. Afr J Health Professions Educ 2016;8(1 Suppl 1):117-120. DOI:10.7196/AJHPE.2016.v8i1.768

The training of novices in medical specialties to achieve the optimum state of cognitive, clinical, technical and professional development requires the use of a variety of teaching methodologies, including the process of feedback. This level of excellence can only be achieved if the gaps between actual and desired performance are reported to the trainee by a more experienced supervisor, together with a plan for improvement.[1] The failure of feedback mechanisms can contribute to incompetent healthcare professionals.[2] This is due to various reasons, including the failure of students to recognise different forms of provision of feedback; when and where feedback is provided; incapacity of the teacher; operational demands of the clinical setting; lack of clearly defined teaching and learning objectives; and inadequate support mechanisms for students not meeting such objectives.[1] Training to become a doctor is almost the same as serving an oldfashioned apprenticeship, during which skills from more experienced seniors are passed on to students in an experiential learning setting.[3-5] As training progresses in postgraduate specialisation, the need for constant, highquality feedback from clinical supervisors/mentors to students intensifies to aid in the development of the trainees’ finely honed competencies in their chosen field. It is only through the provision of feedback that strengths can be identified and amplified, and corrective measures can be put in place to overcome deficiencies.[4,6] Traditionally, consultants in academic teaching hospitals were held in high esteem for their clinical expertise. Students were

expected to learn from observation, rather than being taught by consultants competent in the formal art of teaching. This process is often compounded by service delivery, as tending to patients takes precedence over more timeconsuming explanations about details of and reasons for processes followed. However, the failure to provide this essential component of training may contribute to incompetent and poorly trained clinicians, resulting in poor patient outcomes. Ende,[6] in his seminal article, defined feedback in clinical medical edu cation as ‘information describing students’ or house officers’ performance in a given activity’. Feedback addresses the deficit in meeting a predetermined standard of desired skills by identifying the area of poor performance and devising a means to achieve the standard. It is used to promote the desired, high-quality performance in trainees through raising awareness of current skills in high-level performers.[7] However, to be effective, the process needs to be a formalised assessment of performance, with a corrective plan of action to address deficiencies – a ‘learning through guiding’ process.[5,8,9] In this way, the trainee receives clarification of the process he or she followed compared with what was expected; on how to address the gap between actual and intended performance; and, most importantly, of the consequences for patient outcomes should the current performance go unchecked. Ideally, this should prompt a behaviour change in the trainee to achieve the desired standard.[7,8]

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

117

Research The lack of appropriate feedback in guiding students on when and how to change, and taking note of and acknowledging what is being done well, can soon lead to disillusionment of the student.[10] However, excessively praising or complimenting the student, which is not feedback in the true sense, has been shown to be equally unsatisfactory over time.[11] With appropriate feedback, students develop autonomy in implementing suitable corrective mechanisms to achieve the desired standard of competence,[10] and are able to critically self-assess performance.[11] As the student makes the transition from under- and postgraduate student to independent practitioner, this skill will prove a valuable resource in the context of an evolving set of competen cies, as it aids in a positive approach to lifelong learning, which is expected of a competent doctor. Therefore, even practising physicians, in whom the art of self-reflection (which should be a component of feedback) has been inculcated in their clinical training, are cognizant of the need for continuous medical education and feedback, leading to improved patient outcomes.[12-14] Feedback has often been evaluated, which consistently reveals students’ dissatisfaction with the amount and type of feedback they receive in their clinical and postgraduate training, as they perceive it to be inadequate, inappropriate or non-existent.[14,15] To achieve clinical competence that will result in optimal patient care and outcomes, trainee errors must be rectified and competencies reinforced, especially in the context of workplace ‘experiential’ learning. This study explores registrars’ perceptions of feedback provided by consultants at a teaching hospital across six clinical disciplines, i.e. Internal Medicine, Surgery, Obstetrics and Gynaecology, Paediatrics, Psychiatry and Family Medicine.

Methods

Although a mixed methods approach was adopted for this study, this article reports only on the quantitative data collected. The study population comprised all registrars (i.e. qualified doctors undertaking postgraduate specialisation training) employed at a major teaching hospital attached to the Nelson R Mandela School of Medicine (NRMSM), University of KwaZuluNatal, Durban, South Africa. All registrars from the disciplines listed above were invited to participate in the study via an online questionnaire. Owing to a low response rate, questionnaires were also handed out at departmental academic days. Thirty-seven out of a total of 60 registrars consented to participate in the study. Questions related to various aspects of how feedback was given, including: when (e.g. ‘Feedback is provided in all encounters with a consultant’); where (e.g. ‘Feedback is provided in all settings’); type (e.g. ‘Feedback is informal’); effect (e.g. ‘The effect of feedback on the registrar is noted’); topic (e.g. ‘Feedback is given about clinical skills’ ); and how (e.g. ‘Feedback is given in non-emotive, non-judgemental language’). Sociodemographic information regarding age, gender, home language, discipline and year of study was also gathered. Registrars responded using a 5-point Likert scale (1 (never) - 5 (always)). A definition of feedback was also included: ‘For the purposes of this study, feedback is defined as: A process whereby the desired standard of proficiency in a task has been clearly established. This standard has been communicated to the student. Gaps in performing the task or level of knowledge are identified, based on actual observation of the student, and the student is made aware of his or her shortcomings, together with a plan to improve performance.’ Descriptive statistics were used to interpret the responses of the registrars, with mean values being calculated. Differences between groups were calcu lated using Pearson’s χ2 test for independent variables, with a p-value of

118

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

<0.05 regarded as being statistically significant. Responses were combined to give an overall negative (1, 2 and 3) and positive (4 and 5) response to certain questions. Full ethical approval for the study was received from the Humanities and Social Sciences Ethical Committee, University of KwaZulu-Natal (HSS/1185/013D).

Results

The mean age of the registrars was 32.3 (range 27 - 43) years. The majority were female (64.9%) and first-language English speakers (54.1%). Only two of the registrars had a postgraduate diploma and one had a Master of Medicine (MMed), while the remaining 34 had completed only their basic undergraduate medical degree. Most of the registrars (n=16) were in their 4th year of training, 12 were in their 3rd year, 7 were in their 2nd year, and 2 had just commenced training. The registrars’ specialisations were divided as follows: Paediatrics (n=9), Obstetrics and Gynaecology (n=9), Surgery (n=7), Internal Medicine (n=6), Psychiatry (n=3) and Family Medicine (n=3). Overall, registrars rated the feedback they received as poor, as illustrated in Fig. 1. The majority of registrars (51.4%) reported that both formal and informal feedback was only sometimes, even rarely, received in all encounters with consultants. The location of formal feedback sessions was perceived as appropriate (59.4%), but the advance scheduling of such sessions was not (62.1%). A total of 48.6% registrars rated equally the provision of standards for assessment being predetermined and communicated in advance. However, 51.3% felt that the feedback received was unacceptable, and did not perceive it to be based on concrete observations of performance. The majority reported that they did not receive feedback on techniques performed incorrectly (54.0%) or on those performed correctly (67.5%). Many registrars (59.4%) perceived that feedback was not being documented. More than half (56.7%) of registrars reported that the intended message was not received and the same percentage perceived insufficient opportunity to respond to the consultant. Overall, 54.0% did not agree with the content of the feedback. When formal feedback was given, 64.8% of the registrars believed that a plan for improvement had been given, while 59.5% reported the inclusion of new learning objectives. Some 67.6% of registrars positively reported reflecting on previous feedback as a result of current feedback. The language in which feedback was given was perceived to be non-emotive and nonjudgemental by 64.8%. In a similar vein, 56.7% of the registrars believed that the feedback received was not influenced by race, gender or ethnicity. The proficiency of consultants in giving feedback was scored as unaccept able by 64.8% of registrars, while 59.4% perceived that the effect of feedback on them went unnoticed, and 67.5% noted that there were no support structures for students after receiving feedback. Other results showed that a majority of the registrars (91.7%) believed that consultants should be trained to provide feedback and all agreed that feedback about registrars’ clinical proficiency was important. The exit examination for specialisation – Fellowship of the relevant College of the Colleges of Medicine of South Africa – requires specific preparation in the honing of clinical skills, how to answer written examination questions and make presentations during oral examinations. While 83.3% of registrars agreed that feedback was provided in preparation for these examinations, only 58.8% felt that this feedback was adequate. Some 45.7% thought that it

Research

I would like to receive peer feedback

37.8

8.1 8.1

18.9

27.0

I would like to receive group feedback

29.7

13.5 8.1

18.9

29.7

I would use these techniques when I have students

32.4

5.4 13.5 13.5

I agree with the feedback provided

48.6

My feedback sessions are always successful. I receive the intended message in the intended manner

2.7 13.5

48.6 29.7

10.8

The effect of feedback on me is noted by my consultant

I have an opportunity to respond to the feedback given

16.2

21.6 51.4

8.1 13.5 10.8

Feedback is influenced by my race, gender or ethnicity

32.4 35.1

18.9 10.8 2.7

A component of feedback process is a plan for improvement

29.7

16.2

5.4 2.7 8.1

35.1

Feedback is given in non-emotive, non-judgemental language

18.9 21.6

8.110.8 8.1 10.8 16.2

32.4

Feedback incorporates new learning objectives

29.7

10.8

27.0

40.5 32.4

Formal feedback is documented

35.1

5.4 10.8 13.5

40.5

Consultants are proficient at giving feedback to registrars Support is available to me from different sources after both formal and informal feedback sessions

35.1

2.7 2.7 10.8

29.7 37.8

29.7 5.4 5.4

18.9

24.3 5.4 2.7

18.9

45.9

Receiving feedback encourages reflection about previous feedback 13.5 8.1 8.1 13.5 Feedback is given about procedures and techniques performed correctly

35.1

16.2 16.2

Feedback is given about procedures and techniques performed incorrectly

32.4

8.1 13.5

Feedback is based on concrete observations of my performance

32.4

5.4 13.5

Standards for assessment are predetermined and communicated to me in advance

18.9 13.5 16.2

Informal feedback is provided

80.0

60.0

Rarely

29.7 16.2

27.0 5.4

43.2

48.6 43.2 40.0 Never

48.6 29.7

2.7 8.1 13.5

51.4

Feedback is provided in all encounters with a consultant 100.0

13.5

10.8 13.5 8.1

37.8

Formal feedback is provided

13.5 16.2

24.3 8.1 5.4 10.8

Formal feedback sessions are held in an appropriate location Formal feedback sessions are scheduled in advance

51.4

20.0

24.3

2.7 13.5 8.1 0.0.0

Sometimes

35.1

18.9 20.0 Always

29.7 40.0

60.0

80.0

100.0

Often

Fig. 1. How registrars believe feedback is provided.

was provided in a timely manner. While 61.1% of the registrars reported that feedback was provided on how to obtain their MMed qualification in terms of the protocol preparation, research and administrative processes, less than half thought the information was provided adequately (44.4%) or timeously (41.7%). The majority (70.6%) believed that the esteem in which they held their consultants influenced the manner in which they received feedback. Multiple settings are available for provision of feedback. Registrars reported that feedback was provided in various settings: 25.0% during group teaching, 22.0% at the patient’s bedside, 20.0% during academic days, and 17.0% during one-on-one teaching. No feedback was given in side-room settings. A majority (87.1%) thought that feedback provided by consultants differed between academic hospitals in KwaZulu-Natal.

Discussion

This study explored the perceptions of registrars’ feedback given to them by their consultants at a teaching hospital in six academic disciplines. A fundamental tenet of feedback is the need to improve on performance by identifying the gaps when comparing actual performance with the desired level of competence to be achieved and a previously identified standard. The information with regard to what is missing should be conveyed to the trainee, ideally together with a plan to improve said performance.[4,6] The predetermination of standards to be achieved is a key component.[6] In this study, registrars were divided in their opinions as to whether this

fundamental component of feedback was met. The lack of guidelines that delineate the desired standards and learning objectives to be achieved may leave registrars unsure as to how best to acquire competence in areas of deficiency, as feedback given may be misunderstood. A core component of medical education is the transfer of skills from an experienced senior to an inexperienced novice in a workplace. Assessing performance and providing timeous feedback at the patient’s bedside or shortly thereafter is a type of brief feedback that forms part of a wider spectrum of types of feedback.[1] It is therefore of concern that this study found that feedback was infrequent and not often given at the bedside of the patient, a prime area for highlighting clinical management.[6,7] This finding highlights the need to encourage consultants to provide more feedback, as the experiential clinical setting is particularly conducive to training.[5] As far back as the ‘apprenticeship’ that medical students served in the days of Hippocrates, the importance of feedback has been well documented as a means of ensuring that underperforming students achieve the desired level of competence, informing competent students of the skills that they are using, encouraging increasing use of those skills, and focusing the attention of students on the consequences of not performing optimally.[1,5,7,8,15,16] Of grave concern is that the majority of registrars reported that no feedback was given when techniques were performed incorrectly and that opportunities for entrenching good practice were also missed. It is essential for these deficiencies to be addressed to prevent registrars’ clinical competence and training being compromised.

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

119

Research In a study comparing the giving of feedback to the process of breaking bad news, emphasis is placed on the importance of the supervisor being fully cognizant of the standards against which performance is assessed, and how these apply to learners at different levels.[8] Being able to respond to feedback allows registrars the opportunity to clarify areas of deficiencies and the steps needed to rectify such deficiencies, which increase the possibility of improvement. The majority of registrars reported not receiving the intended message and not agreeing with the feedback. This is further compounded by them reporting that they were not given an opportunity to respond. This is a serious concern, as these findings may have the detrimental effect of causing the registrar to feel the process is unhelpful, with no clear direction of how to improve, leaving him or her inert, demoralised and fearful to proceed in case of erring. Worse still, the registrar may come to believe that the consultant is wrong and so may persist with incorrect clinical manage ment.[6] Consultants need to be made aware of the effect of feedback on registrars and to ensure that various and appropriate support mechanisms are available to prevent any untoward consequences. Registrars must be at ease in seeking such support. An appropriate approach to feedback is essential to ensure that the process attains the desired end result of improving performance. This study found that only a third of registrars felt that consultants were competent in providing feedback. It is in the interaction with patients, under the expert eye of the consultant, that these practical areas, which require a ‘hands-on’ approach and cannot be learnt abstractly or didactically, can be improved, provided feedback is given.[5] Hence, it is critical for supervisors and students to be skilled in the process of giving and receiving feedback. The Fellowship examination is the exit examination for registrars – an external examination after 4 years of training. The examination has as much to do with being able to present competently and have appropriate examination techniques as with being academically competent. It is disconcerting that, although registrars reported that feedback is provided in preparation for the examination, it is only perceived to be adequate and given timeously half of the time. Such preparation should be an integrated and continuous part of a comprehensive training approach, enabling registrars to present with ease and confidence in their final examinations. It should not be a mere add-on during the last weeks before the examination. Furthermore, information regarding the timelines, and academic support for attaining the MMed qualification, should be standardised to prevent unnecessary delays and improve throughput time in the registrar programme. Our study suggests that a component of the discipline academic day be allocated for formal Fellowship examination preparation to ensure that essential feedback is provided to registrars timeously to allow for adequate preparation.

120

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

Conclusion

Good-quality feedback comprising all elements is essential in postgraduate clinical training. While feedback is provided in some meetings with consult ants, the essential elements of feedback appear to be missing, i.e. of having predetermined standards to be achieved established and in place, clearly communicated to the registrar beforehand, based on observed performance and incorporating a plan for improvement. Provision must be made for the registrar to respond to feedback and clarify areas of possible confusion, especially with regard to the improvement plan. The findings of this study highlight the need for appropriate and continuous training programmes that must be developed and implemented for consultants to provide and for registrars to receive feedback effectively in the postgraduate medical training settings. A limitation of this study was the small sample size and possible female bias, which can be addressed in future studies by including more academic hospitals to increase both sample size and equitable gender representation. Acknowledgements. This publication was made possible by grant No. R24TW008863 from the Office of the US Global AIDS Coordinator and the US Department of Health and Human Services, National Institutes of Health (NIH OAR and NIH ORWH). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the US government. We thank Dr M Muzigaba for help with the statistical analysis, and the staff and students who participated in this study. References 1. Shrivastava SR, Shrivastava PS, Ramasamy J. Effective feedback: An indispensable tool for improvement in quality of medical education. J Pedagogic Dev 2014;4(1)12-20. 2. Iobst WF, Sherbino J, ten Cate O, et al. Competency-based medical education in postgraduate medical education. Med Teach 2010;32:651-656. [http://dx.doi.org/10.3109/0142159X.2010.500709] 3. Rodriguez-Paz JM, Kennedy E, Salas E, et al. Beyond ‘see one, do one, teach one': Toward a different training paradigm. Qual Saf Health Care 2009;18:63-68. [http://dx.doi.org/10.1136/qshc.2007.023903] 4. Henderson P, Ferguson-Smith A, Johnson M. Developing essential professional skills: A framework for teaching and learning about feedback. BMC Med Educ 2005;5(1):11. 5. Fluit C, Bolhuis S, Grol R, et al. Evaluation and feedback for effective clinical teaching in postgraduate medical education: Validation of an assessment instrument incorporating the CanMEDS roles. Med Teach 2012;34:893101. 6. Ende J. Feedback in medical education. JAMA 1983;250:777-781. 7. DeLima Thomas J, Arnold R. Giving feedback. J Palliat Med 2011;14:233-238. [http://dx.doi.org/10.1089/ jpm.2010.0093] 8. Bing-You R, Trowbridge R. Why medical educators may be failing at feedback. JAMA 2009;32:1330-1331. 9. Nicholson S, Cook V, Naish J, Boursicot K. Feedback: Its importance in developing medical students’ clinical practice. Clin Teach 2008;5:163-166. 10. Jussim L, Yen H, Aiello J. Self-consistency, self-enhancement, and accuracy in reactions to feedback. J Exp Soc Psychol 1995;31:322-356. 11. Srinivasan M, Hauer K, dDer-Martirosian C, Wilkes M, Gesundheit N. Does feedback matter? Practise-based learning for medical students after a multi-institutional clinical performance exam. Med Educ 2007;41:857-865. 12. Veloski J, Boex J, Grasberger M, Evans A, Wolfson D. Systematic review of the literature on assessment, feedback and physicians’ clinical performance: BEME Guide No. 7. Med Teach 2006;28:117-128. 13. Rogers A, Boehler M, Schwind C, Meier A, Wall J, Brenner M. Engaging medical students in the feedback process. Am J Surg 2012;203:21-25. [http://dx.doi.org/10.1016/j.amjsurg.2011.07.009] 14. McKinley R, Williams V, Stephenson C. Improving the content of feedback. Clin Teach 2010;7:161-166. 15. Hattie J, Timperley H. The power of feedback. Rev Educ Res 2007;77:81-112. 16. Archer J. State of the science in health professional education: Effective feedback. Med Educ 2010;44:101-108.

Research Third-year medical students’ and clinical teachers’ perceptions of formative assessment feedback in the simulated clinical setting R M Abraham, MBBS, PGDip (Anaesthesia), PGDip (Public Health), MMedSc; V S Singaram, BMedSc, MMedSc, PhD Clinical and Professional Practice, School of Clinical Medicine, College of Health Sciences, University of KwaZulu-Natal, Durban, South Africa Corresponding author: R M Abraham (abrahamr@ukzn.ac.za)

Background. Clinical skills training in the clinical skills laboratory (CSL) environment forms an important part of the undergraduate medical curriculum. These skills are better demonstrated than described. A lack of direct observation and feedback given to medical students performing these skills has been reported. Without feedback, errors are uncorrected, good performance is not reinforced and clinical competence is minimally achieved. Objectives. To explore the perceptions of 3rd-year medical students and their clinical teachers about formative clinical assessment feedback in the CSL setting. Methods. Questionnaires with open- and closed-ended questions were administered to 3rd-year medical students and their clinical skills teachers. Quantitative data were statistically analysed while qualitative data were thematically analysed. Results. Five clinical teachers and 183 medical students participated. Average scores for the items varied between 1.87 and 5.00 (1: negative to 5: positive). The majority of students reported that feedback informed them of their competence level and learning needs, and motivated them to improve their skills and participation in patient-centred learning activities. Teachers believed that they provided sufficient and balanced feedback. Some students were concerned about the lack of standardised and structured assessment criteria and variation in teacher feedback. No statistical difference (p<0.05) was found between the mean item ratings based on demographic and academic background. Conclusion. Most teachers and students were satisfied with the feedback given and received, respectively. Structured and balanced criterion-referenced feedback processes, together with feedback training workshops for staff and students, are recommended to enhance feedback practice quality in the CSL. Limited clinical staff in the CSL was noted as a concern. Afr J Health Professions Educ 2016;8(1 Suppl 1):121-125. DOI:10.7196/AJHPE.2016.v8i1.769

A key component of undergraduate medical education is feedback, which is considered the ‘lifeblood of learning’.[1] Several definitions for feedback exist, but all suggest that feedback is an interactive process with the objective of giving students constructive input into their work.[2] Feedback in the context of behavioural science is defined as the ‘provision of information about the gap between the actual level and the reference level of learning that is used to alter the gap in some way’.[3] In medical education, feedback is defined as ‘specific information about the comparison between trainees’ observed performance and a standard, given with the intent to improve the trainee’s performance’.[4] Feedback can be either formative or summative. Formative feedback in clinical assessment may occur during the theme/module or clinical placement. The objective here is to enhance students’ learning ability by informing them of the strong and weak aspects of their clinical performance, and providing suggestions for improvement in preparation for their summative examination. It does not include the rating of clinical skills performance but intends to shape the students’ responses to the task being worked on. Summative feedback takes place at the end of a theme/module to determine whether or not overall goals have been achieved and includes explicit feedback with rating of clinical skills performance. It may help to shape the next performance or task but is often received too late to have an effect on the task being evaluated.[5] Formative assessment, through the provision of feedback and debriefing in the simulated clinical setting, is important to support student learning and clinical skills development. Feedback is considered effective when it is: provided soon after task performance; is presented in a manner sensitive to students’ learning style; clearly identifies strengths and weaknesses;

includes suggestions for improvement; and is constructive, motivating and able to ‘feed-forward’.[6] Students are aware of its importance in improving learning outcomes and value a balanced and structured feedback approach as effective to meet individual needs.[7] However, concerns about the quality of feedback received by medical students have been highlighted.[8] Clinical skills laboratories (CSLs) are educational facilities that provide medical students with opportunities during the preclinical years to learn and practise clinical skills before using them in real clinical settings. Patient history-taking, performing a physical examination and then analysing and presenting this information involve psychomotor and cognitive skills, as well as behaviour acquired through repetitive and systematic training, and depends upon effective teaching, assessment and feedback[9] in the CSL setting. These skills are better demonstrated than described. Unfortunately, a widely reported deficiency is the lack of dynamic assessment, which involves direct observation and immediate feedback given to medical students’ performing these skills.[10] Without feedback on observation, errors go uncorrected, good performance is not reinforced and clinical competence is achieved only minimally. As evidenced from empirical research, students need adequate motivation and belief in their abilities to be able to satisfactorily decode feedback messages to self-regulate their clinical performance.[11] The importance of feedback is also widely acknowledged by clinical teachers. Although they believe they give regular and sufficient feedback, this is often not the perception of learners.[8] In a study conducted to assess the students’ voice, students rated assessment feedback as an aspect in need of improvement across 14 universities in Australia.[12] Even though giving feedback to learners on their clinical performance has been identified as

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

121

Research a major approach to academic teaching and learning in clinical education, it needs to be monitored.[8] The objective of this study was to survey medical students’ and clinical teachers’ perceptions of formative assessment feedback on direct observation of clinical examination skills performed in the CSL.

Method Context

The Nelson R Mandela School of Medicine (NRMSM) at the University of KwaZulu-Natal (UKZN), Durban, South Africa has adopted a 6-year hybrid problem-based learning (PBL) medical curriculum. Clinical skills teaching forms a part of each of the theme-based PBL modules during the first three preclinical years taught in the medical school CSL, using simulated patients. The clinical skills mini-logbook formative assessment session occurs before the summative assessments at the end of each 6- to 8-week theme-based module. During the formative assessment sessions, each student is given 8 minutes to systematically demonstrate examination skills on a simulated patient. The teacher observes each student and rates the performance in the mini-logbook, based on the minimum requirements for the skill to be deemed satisfactory. Performances are rated as ‘inadequate’, ‘satisfactory’ or ‘exceeded expectation’ and verbal and written feedback are provided.

Study design

This mixed methods observational study was conducted with the 2014 cohort of 3rd-year medical students and their clinical skills teachers at the NRMSM. Ethical approval was granted by the UKZN Humanities and Social Sciences Research Ethics Committee (HSS/0084/014M). Third-year medical students (N=183) and clinical teachers (N=5) consented to completing the questionnaires. The research design involved mixing both qualitative and quantitative research data to provide a more comprehensive understanding of the perceptions of both the students and teachers.

Instrument

Two questionnaires, consisting of closed- and open-ended questions, were designed. One questionnaire was designed to explore the teachers’ perceptions of the feedback. The other questionnaire explored students’ responses to the mini-logbook formative assessment feedback received with regard to the value of feedback, preferences for feedback and suggestions to

122

improve feedback. Each instrument consisted of the following components, which formed the different sections of the questionnaire: (i) demographic data; (ii) perceptions of feedback; (iii) value of feedback; (iv) preferences for feedback; and (v) suggestions for feedback. Sections (ii), (iii) and (iv) included closed questions that required students and clinical teachers to indicate their level of agreement with a series of statements on a 5-point Likert scale, ranging from ‘strongly disagree’ to ‘strongly agree’. There were two open-ended questions on the general perceptions of feedback (section (iii)) and suggestions to improve it (section (v)). Minor modifications were made after questionnaires were piloted.

Data analysis

Quantitative data were analysed statistically using the Statistical Package for the Social Sciences (SPSS) (version 21) (IBM, USA) and reported anonymously. Student independent t-tests and analysis of variance (ANOVA) statistical tests were employed to ascertain specific age, gender, language, enrolment status and aca demic performance differences between the demographic groups. Confidence intervals (CIs) were set at 95% and statistical significance at p<0.05. The qualitative data were read and reread for familiarity with what it entailed, paying specific attention to patterns that occurred. The pattern of responses was used to identify

emergent themes, with consensus from both authors. The student and teacher responses were then clustered according to the emergent themes and categorised according to the degree of support that a particular response represented in terms of the total sample. The qualitative data gathered from the students’ and clinical teachers’ responses to the open-ended questions were thematically analysed and will be reported in more detail in a follow-up study.[13] Specific quotations were selected to support or extend the quantitative data that are the focus of this article.

Results

The student sample consisted of 115 (68%) females. The majority (96%) were between 18 and 25 years of age. The teachers consisted of 1 (20%) female and 4 (80%) male clinicians. The students and teachers came from diverse multicultural, language and academic backgrounds.

Students’ responses to feedback received

Illustrated in Table 1, most students believed that receiving feedback had a positive effect on their learning by informing them of what was needed to improve their performance (94%), explaining the performance rating received (90%) and providing an evaluation of their strengths and weaknesses in clinical skills (96%). They also appreciated feedback as it informed them of the teachers’ expectations with regard to the skill

Table 1. Clinical skills logbook assessment feedback: impact of feedback on academic performance (N=183) Response Positive response, %

Neutral response, %

Negative response, %

Feedback informs me what I need to do to improve my performance in clinical skills

Feedback explains the performance rating I received in the logbook

Feedback is an evaluation of my strengths and weaknesses in the skill performed

Feedback informs me what the expectations of the lecturer/teacher are regarding clinical skills performance

I use feedback to try to improve my performance in future logbook assessments and end-of-semester objective structured clinical examinations

Feedback is only useful when I receive a bad performance rating in the logbook

Feedback is only useful when it is positive

Getting a performance rating is more important to my learning than feedback

Statement

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

Research performed (93%) and was useful and relevant to their goals as a student to feed-forward (93%): ‘I really appreciated the fact that we have logbook sessions before the actual exam … It helps me to see my weak points.’ Although feedback was valued, the students did not think feedback was useful if it only reported a bad performance (82%) or a good perfor mance (86%). They were more likely to value balanced feedback with positive reinforcement and constructive criticism indicative of an understanding of their performance: ‘Teachers must give us feedback that is truly indicative of our performance. They must not focus only on the wrong things but also explain how we can improve the things that we did well.’ Students reported that they valued the feedback received (99%) and always ensured they read the feedback provided in the logbook (98%) (Table 2). They believed that they deserved to receive feed back, especially after putting effort into practising clinical skills (96%), as it encouraged (92%) and motivated them to study (93%). Students felt that their teachers’ feedback was a demonstration of them caring about a student’s work (78%). They suggested ways to improve their feedback experience and requested feedback as a tool to gauge their knowledge in summative assessments: ‘We don’t receive feedback for our [objective structured clinical examinations] OSCEs. I believe to improve we should be given this feedback as it is an indication of how much more work you need to put towards your clinical skills.’ An important issue raised was teacher variability when delivering feedback (60%) and the need for more clarity when providing feedback. The criteria with which feedback was provided were of concern. Students linked feedback variation to lack of use of standardised and structured assessment criteria by the teachers. When prompted for recommendations for improving clinical skills logbook assessment feedback they suggested standardisation: ‘Logbook sessions are useful in assessing our skills. However, I feel that teachers should use the same methods of testing to make things fair for all students. Assessment criteria must be available to students as a form of learning objectives and goals.’ Students were satisfied with the timely delivery of feedback (85%). They valued some general feedback (51%) as it gave them an idea about

Table 2. Clinical skills logbook assessment feedback: impact of feedback on personal motivation to learn (N=183) Response Statement

Positive response, %

Neutral response, %

Negative response, %

Feedback is important to me

I always read the feedback on my logbook

I deserve feedback when I put a concerted effort into practising my clinical skills

When I receive substantial feedback I feel encouraged

Teachers who provide feedback care about what the students generally think

Feedback motivates me to study

When I don’t receive feedback I feel that the teacher does not respect me

All the clinical skills teachers follow a similar style and criteria of providing feedback

An important part of learning is being able to discuss the subject with my teacher

I learn more when my teacher focuses on what I did wrong

Table 3. Clinical skills logbook assessment feedback: references for feedback (N=183) Response Positive response, %

Neutral response, %

Negative response, %

Feedback on clinical skills logbook assessments is generally provided immediately

General feedback provided in class helps me learn independently

Individual feedback is better because I can clarify any issues with the teacher or lecturer

Verbal feedback is easier to understand

Specific feedback is better because it helps me understand what I did right and wrong in the logbook session

It is boring when lecturers provide general feedback to the class

I prefer general feedback in class because it is not personal

I prefer verbal feedback because I can communicate with the teacher and clarify information

Group feedback is best because I can see where other students have experienced similar problems

Teachers’ written comments are often difficult to read and inadequately explained

Written feedback is better because I can refer to it much later

Statement

whether other students in the class experienced similar problems (Table 3). One suggestion made was the need for the following: ‘… a little more group feedback to judge where I am in the class and whether or not I am putting enough effort into my learning.’

The students acknowledged individual feedback (88%) as more personal and fitting for clarifying issues with teachers as an important part of their learning: ‘It is always better for one-on-one feedback that focuses on individual needs and allows students

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

123

Research to interact with the facilitator easily, ask questions and receive clarification when necessary.’ The students preferred verbal feedback (75%) specific to their work, including both positives and negatives in the skill demonstrated: ‘More emphasis on verbal individual feedback for me … because it focuses specifically on my performance and accounts not only for the things I got wrong but the things I got right and what I need to improve on.’

Clinical teachers’ perceptions of giving feedback As illustrated in Table 4, teachers were comfortable with providing feedback, and agreed that effective performance feedback improved students’ learning and required their engagement with and necessary skill in the feedback provision process. They all reported providing individual and verbal feedback, while some amount of written and group feedback was also given. Three teachers indicated that the setting in which feedback was provided in the CSL was private; one was neutral regarding this, while another did not think the setting was private enough. Three teachers provided feedback that reflects more general information on the students’ performance. Three teachers allowed the students to self-assess and reflect on their performance to confirm if they agreed or disagreed with the feedback information provided. Most teachers disagreed that the culture and language background of students determined the feedback they give: ‘I do not really take account of culture and background because I am only focused on the topic, the technique and documentation of clinical skills.’ However, one teacher considered language and culture to be an important factor while providing feedback: ‘For second-language students the delivery of feedback is important. Also, from a cultural perspective students may see me as a figure of authority and misconstrue my feedback as “scolding”.’ The majority of teachers felt that staff development is crucial for increasing teachers’ confidence and skill in this area of giving feedback: ‘A workshop on feedback in clinical skills is very important. It will allow all teachers to harmonise the way they have to improve and deliver the feedback.’

124

Table 4. Clinical teachers’ perceptions of feedback provided during a logbook session (N=5) Response Positive response, %

Neutral response, %

Negative response, %

Provision of effective performance feedback to medical students improves learning outcomes

100

Provision of clearly effective feedback would require engagement and skill

100

I often provide the following types of feedback to my students (verbal)

100

I often provide the following types of feedback to my students (written)

I often provide the following types of feedback to my students (group)

I often provide the following types of feedback to my students (individual)

100

The students are aware that they would be receiving feedback after the clinical performance

100

Statement

Feedback is often provided in a confidential setting

The feedback I provide only reflects what the student did satisfactorily

The feedback I provide only reflects what the student did unsatisfactorily

The feedback I provide is usually a balance between what the student did well and what areas require improvement

100

The feedback I provide reflects more specific information about the student’s clinical performance

100

The feedback I provide reflects more general aspects of the student’s clinical performance

Immediately after providing the feedback I would allow the students to self-assess and reflect on their performance to confirm if they agree or disagree with the feedback

I am comfortable providing feedback to students

100

The culture and language background of the students is an important factor when giving feedback

I feel there should be formal training to improve and enhance my feedback skills as an academic

Teachers also indicated some barriers and suggestions to improve feedback in clinical skills: ‘Regular student and staff evaluations and increase time for feedback. These approaches are limited by time and staffing resource con straints.’

Discussion

Lack of effective feedback is considered a serious deficiency in medical education.[7] It is therefore positive to note that the current study found that most students and teachers were satisfied with the mini-logbook formative assessment feedback in the CSL.

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

Similar to previous studies, students valued the feedback received during clinical skills sessions as most of them requested personalised, frequent, immediate and clear feedback linked to specific learning outcomes as a tool to gauge their knowledge relevant to their goals.[14] They confirmed that receiving timely feedback had a positive effect on their learning and clinical performance and hence their future practice. The students preferred balanced feedback as an evaluation of their strengths and weaknesses essential for their growth as a student, to boost their confidence and self-esteem, to increase their motivation to study and to provide them

Research with some direction for learning. This may have a positive effect on their subsequent clinical performance and the development of their clinical competence.[2] Although there were a wide variety of feedback preferences, the medical students who took part in the study preferred personal, individual and verbal feedback. This afforded them an opportunity to communicate with the teacher and clarify information – an important part of learning.[15] Some group feedback was favoured, as it gave students an idea about whether other students in the class experienced similar problems. While some found general comments uninformative, this method of feedback must not be dismissed as students should be able to probe feedback by asking specific questions.[16] The clinical skills teachers in this study all agreed that providing directly observed immediate formative feedback improved students’ learning and allowed enough time for students to make changes to their performance before examinations.[17] Some teachers allowed students to reflect on their performance, providing them with insight into the students’ ability to self-assess and reflect.[6] Although teachers were aware that providing regular, balanced feedback with clear guidelines for improvement is essential and the teachers believed they provided this most of the time, this is not how the students perceived the situation. One of the reasons that may explain deficiency in the delivery of negative feedback in this setting may be the teachers’ desire to avoid upsetting students, leading to ‘vanishing feedback’ and subsequent avoidance in giving any feedback. Even though cultural and linguistic differences did not seem to influence the feedback process, a concern regarding misconceptions was attributed to different cultural and language backgrounds. Staff development workshops on the provision of negative feedback[18] and diversity training to bridge language and cultural differences is recommended to address these challenges. There were other concerns raised by the clinical teachers with regard to the effective delivery of feedback. These included the challenges of teaching and assessing large groups of students within a short time period in an environment that is not adequately private, as well as limited space and time for recording feedback in the mini-logbook. Moreover, a greater emphasis on assessment, as opposed to giving feedback, was noted, together with a request from teachers for training to enhance their skills in giving feedback. The limited clinical staffing was raised as a barrier to providing effective feedback, highlighting the need for more clinical teachers in the CSL. Some students commented on teacher variability and inconsistency in the quality of feedback delivered during clinical skills mini-logbook sessions. These findings are similar to those of other studies revealing that learners often do not feel they receive enough feedback and if they do, they feel that the process is not effective most of the time.[7] Students attributed this variability to the lack of standardised and structured feedback assessment criteria. Another reason contributing to the discounting of feedback by the students may be their unfamiliarity with a particular teacher and their assessment style. Connecting with teachers to create a positive and healthy environment will enhance the feedback process and eliminate barriers to the use of feedback as a tool for self-improvement and development.[19] This will further help students recognise areas for clinical skills development by assisting them to self-regulate and self-monitor their learning processes.[20]

Conclusion

The students recognised the transferable value of the learning skills developed as a result of an effective feedback for processing new learning. Hence, they valued the effect of feedback as an instrument to guide and regulate their learning. Based on information from this study on students’ varied and inconsistent experiences with receiving feedback, feedback briefing or training sessions, together with the development of an appropriate feedback strategy, are recommended for students and teachers. A structured and balanced criterion-referenced feedback process is also recommended to enhance the consistency of feedback practice and provide fair assessment. Moreover, establishing an instructional system and revising the clinical skills mini-logbook to a more specific criterion-based, standardised and structured feedback instrument, as well as extending it to other clinical disciplines, are suggested. Further regular evaluation of the feedback process in the CSL would help to maintain and enhance clinical skills core competencies and provide direction to address any deficiencies in the clinical skills teaching programme. Acknowledgements. This publication was made possible by grant No. R24TW008863 from the Office of the US Global AIDS Coordinator and the US Department of Health and Human Services, National Institutes of Health. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the US government. The staff and students who participated in the study are also acknowledged. References 1. Rowntree D. Assessing Students: How Shall We Know Them? 2nd ed. London: Kogan, 1987:86. [http://dx.doi. org/10.2307/1981340] 2. Eraut M. Feedback. Learn Health Soc Care 2006;5(3):111-118. [http://dx.doi.org/10.1111/j.1473-6861.2006.00129.x] 3. Ramprasad A. On the definition of feedback. Behav Sci 1983;28(1):4-13. [http://dx.doi.org/10.1002/bs.3830280103] 4. Van de Ridder JM. What is feedback in clinical education? Med Educ 2008;42(2):189-197. [http://dx.doi. org/10.1111/j.1365-2923.2007.02973.x] 5. Wiggins G. Feedback: How learning occurs. In: Chaffee EE, ed. Assessing Impact: Evidence and Action. Washington, DC: American Association for Higher Education, 1997:31-39. 6. Hattie JA, Timperley H. The power of feedback. Rev Educ Res 2007;77(1):81-112. [http://dx.doi. org/10.3102/003465430298487] 7. Weinstein DF. Feedback in clinical education: Untying the Gordian knot. Acad Med 2015;90(5):559-561. [http:// dx.doi.org/10.1097/acm.0000000000000559] 8. Edgren G, Haffling AC, Jakobsson U, et al. Comparing the educational environment (as measured by DREEM) at two different stages of curriculum reform. Med Teach 2010;32(6):e233-e238. [http://dx.doi. org/10.3109/01421591003706282] 9. Association of American Medical Colleges. Recommendations for Clinical Skills Curricula for Undergraduate Medical Education. Washington, DC: AAMC, 2005. https://www.aamc.org/download/130608/data/clinicalskills_ oct09.qxd.pdf.pdf (accessed 22 May 2014). 10. Association of American Medical Colleges. Medical School Graduation Questionnaire. 2012. All School Summary Report. Washington, DC: AAMC, 2014. https://www.aamc.org/data/gq/allschoolsreports (accessed 17 February 2015). 11. Wright K. Student nurses’ perceptions of how they learn drug calculation skills. Nurse Educ Today 2012;32(6):721726. [http://dx.doi.org/10.1016/j.nedt.2011.09.014] 12. Scott G. Assessing the Student Voice: A Higher Education Innovation Program Project. Canberra: Department of Education, Science and Training, 2006:8-10. 13. Krippendorff K. Content Analysis: An Introduction to its Methodology. 2nd ed. Thousand Oaks, CA: Sage Publications, 2004:413. [http://dx.doi.org/10.1177/1094428108324513] 14. Hounsell D. Towards more sustainable feedback to students. In: Boud D, Falchikov N, eds. Rethinking Assessment in Higher Education. London: Routledge, 2007:101-113. 15. Parikh A, McReelis K, Hodges B. Student feedback in problem-based learning: A survey of 103 final year students across five Ontario medical schools. Med Educ 2001;35(7):632-636. [http://dx.doi.org/10.1046/j.13652923.2001.00994.x] 16. Dent J, Harden RM. A Practical Guide for Medical Teachers. 4th ed. New York: Elsevier Health Sciences, 2013:273-275. [http://dx.doi.org/10.1136/pmj.78.916.125] 17. Krackov SK. Expanding the horizon for feedback. Med Teach 2011;33(7):570-575. [http://dx.doi.org/10.3109/0 142159x.2011.617797] 18. Ende J. Feedback in clinical medical education. JAMA 1983;250(6):777-781. [http://dx.doi.org/10.1001/ jama.250.6.777] 19. Watling C, Driessen E, van der Vleuten CP, Lingard L. Learning from clinical work: The roles of learning cues and credibility judgements. Med Educ 2012;46(2):192-200. [http://dx.doi.org/10.1111/j.1365-2923.2012.04126.x] 20. Raftery S. Feedback: An essential element of student learning in clinical practice. Nurse Educ Pract 2008;8(6):405411. [http://dx.doi.org/10.1016/j.nepr.2008.02.003]

May 2016, Vol. 8, No. 1, Suppl 1 AJHPE

125