ECRM 2013 proceedings

Proceedings of

The 12th European Conference on Research Methodology for Business and Management Studies University of Minho, Guimaraes Portugal 4-5 July 2013

Edited by

Anabela Mesquita, Instituto PolitĂŠcnico do Porto And Isabel Ramos,, University y of Minho,, Guimaraes Portugal A conference managed by ACPI www.academic-conferences.org

Proceedings of the 12th European Conference on Research Methodology for Business and Management Studies

University of Minho Guimaraes, Portugal

4‐5 July 2013

Edited by Isabel Ramos Conference Chair Minho University, Guimaraes Portugal and Anabela Mesquita Programme Chair Instituto Politécnico do Porto Porto, Portugal

Copyright The Authors, 2013. All Rights Reserved.

No reproduction, copy or transmission may be made without written permission from the individual authors.

Papers have been double-blind peer reviewed before final submission to the conference. Initially, paper abstracts were read and selected by the conference panel for submission as possible papers for the conference.

Many thanks to the reviewers who helped ensure the quality of the full papers.

These Conference Proceedings have been submitted to Thomson ISI for indexing. Please note that the process of indexing can take up to a year to complete.

Further copies of this book and previous year’s proceedings can be purchased from http://academic-bookshop.com E-Book ISBN: 978-1-909507-32-6 E-Book ISSN: 2049-0976 Book version ISBN: 978-1-909507-30-2 Book Version ISSN: 2049-0968 The Electronic version of the Proceedings is available to download at ISSUU.com. You will need to sign up to become an ISSUU user (no cost involved) and follow the link to http://issuu.com Published by Academic Conferences and Publishing International Limited Reading UK 44-118-972-4148 www.academic-publishing.org

Contents Paper Title

Author(s)

Page No.

Preface

iv

Committee

v

Biographies

viii

Selecting Public e-Procurement MCDA Methods Using a Three Tier Evaluation Mixed Method (TTMM) Research Model

Mohamed Adil, Miguel Baptista Nunes and Guo Chao Peng

1

“Experts and Novices”: An Intervention to Support Literature Review and Analysis

Deborah Anderson

9

Evaluating a Newly Implemented Dissertation Supervision System – Views From Both Sides of the Equation

Marie Ashwin and Alan Hirst

15

Barriers to Using Qualitative Methods in Business and Management Research in Ukraine

Yevhen Baranchenko and Andrey Yukhanaev

24

In a Word: Applying Cognitive Interviewing Methods in Legal Market Research

David Bednall, Marie Ashwin and Alan Hirst

32

The Prevalence of Research Methodology Mis/Conceptions Among Business and Management Academics

Frank Bezzina and Mark Saunders

40

Looking Through, Looking at and Looking Behind: Using Photos in Research Interviews

Zelma Bone

48

Ethnography in Information Systems Research

Ann Brown

58

A Methodological Scan of a National Industry Based Research Program for the Rail Industry 2007-2014

Roslyn Cameron

65

Applying the GRAMMS to a Community Engaged Mixed Methods Study of Skilled Migrants in Regional Australia

Roslyn Cameron, Trudy Dwyer, Scott Richardson Ezaz Ahmed and Aswini Sukumaran

Conflicting Conceptions of Business Ethics Methodology

Nicholas Capaldi

85

And we Shall Spend the First Million Together: Narrative as Basis for a Case Study Research Process

Margarida Cardoso and Isabel Ramos

90

Crisis Management Simulations – Narrative Inquiry Into Transformative Learning

David Clemson and Khalid Samara

100

Developing a General Analytical Inductive Qualitative Research Strategy to Investigate Business School Industry Collaboration

Fariba Darabi and Murray Clark

108

Mixed Methods Research: Creating Fusion From the QUAL and QUAN Data Mosaic

Sally Eaves

118

A Multi-Layered Approach to Surfacing and Analysing Organisational Narratives: Increasing Representational Authenticity

Sally Eaves and John Walton

127

Comparison of Methodologies of CSR Index – Application to the PSI 20 Companies

Paula Fernandes, Ana Paula Monte, Rui Pimenta and Sandra Afonso

137

i

,

74

Paper Title

Author(s)

Page No.

Assessing the Learning Style Preferences of Postgraduate General Business Management Students Using VARK

Elena Fitkov-Norris and Ara Yeghiazarian

144

The Limits of Relativism in Managerial Thinking

Simona Hašková and Pavel Kolář

153

Happy on the Year Abroad? A Mixed Qualitative Methods Approach to Evaluating the Worth of Home Institution Staff Visits to Undergraduates on their Year Abroad

Deborah Knowles and Karen Kufuor

162

Understanding the Australian Indigenous Entrepreneurs Through Narratives

Branka Krivokapic-Skoko, Tracey Trudgett, Sonya , Pearce, Mark Morrison Jock Collins and P.K. Basu

169

The Research Journey: Why it is Important to Think About it and Record it, as a Vehicle to Becoming a More Competent and Effective Researcher

David Lamb

178

The use of Storytelling as a Research Method: The Case of the Police Service of England and Wales

Ron Lawson, Gary Shaw, Gail Sanders and Peter Smith

185

Application of the Structural Equation Model to Develop an Employee Satisfaction Diagnostic Model

Ophillia Ledimo and Nico Martins

192

Integrated Evaluation Model Strengthening Effective Management of Strategic Research Programmes

Ludmiła Łopacińska and Adam Mazurkiewicz

202

What They did and What They Said? – An Observed Experience

Arminda Lopes

210

Applying Action Research in the Adoption of Information Systems Security Policies

Isabel Lopes and Filipe de Sá-Soares

219

Why Older People Intend to Learn Computers - a Grounded Theory Study

Eurico Lopes, Luis Barata and Ana Barata

227

Towards The Operationalisation of the Socio-Political Utilisation of Research Findings in Business and Management Studies

Hendrik Marais

237

The Critical Incident Technique as a Tool for Gathering Data as Part of a Qualitative Study of Information Seeking Behaviour

Rita Marcella, Hayley Rowlands and Graeme Baxter

247

Applying Quadrant Analysis to Determine Textbook Learning Enhancement

Nico Martins

254

Using Focus Groups as a Diagnostic Tool With a Positivistic Approach

Ellen Martins and Nico Martins

263

Characterizing Intuition in Information Systems Research

Dan McAran and Sharm Manwani

271

Sharing the Research Journey – Fostering a Love of Research in Coursework Masters Students

Grace McCarthy

281

Narrating an Emerging Cross-Sector Partnership: From Brand Orange to Taste Orange

Cathi McMullen and Ian Braithwaite

289

The Three World Model of Knowledge as a Solution to Failing Research Centres

Graham Trevor Myers

297

Thematic Content Analysis: Is There a Difference Be® tween the Support Provided by the MAXQDA and ® NVivo Software Packages?

Mírian Oliveira, Claudia Bitencourt, Eduardo Teixeira, Ana Clarissa Santos

304

ii

Paper Title

Author(s)

Page No.

A Comparison of Three Projective Techniques: Lessons Learned From a Business Research Class

Phallapa Petison

311

Research Methodological Trends in Business and Management Studies in South Africa: An Exploratory Bibliometric Survey

Magdal Pienaar-Marais and Gerhard Moolman

317

Everyday Ethics for Student Research Projects

Martin Rich

327

Research Methods in Management Academic Programs: From Where we are to Where we Want to go

Silvia Silva and Sara Ramos

333

Co-Word Mapping Using Correspondence Analysis

Anthony Stacey

339

Using Knowledge Cartography Techniques for Conducting a Realist Synthesis on Collaborative Teaching

Leonel Tractenberg

347

Developing Knowledge Sharing Partnerships in the SME Sector: An Action Research Approach

Teresa Waring, Lorraine Johnston, Angela McGrane, Thuyuyen Nguyen and Peter Scullion

354

PHD papers

363

Empirical Examination of Internal Dynamics of Software Project Management: Mixed Methods Approach

Zana Ahmedshareef, Miltos Petridis and Robert Hughes

365

Grounded Theory Application in Doctorate Research

Sjoerd Gehrels

375

Crowdsourcing: Personal Values and Self-Efficacy as Antecedents to Creative Behavior

Susana Morais and Isabel Ramos

383

Work in Progress Papers

389

Influenza Pandemic Outbreaks: Conducting Focus Groups with Rural Communities in Baram, Sarawak

Constance Rinen Justin Wah and Rohaya MohdNor

391

Use of a Learning Management System in Undergraduate Business Communications Courses

Arlene Nicholas

395

iii

Preface The 12th European Conference on Research Methodology for Business and Management Studies is hosted this year by the University of Minho, Guimaraes, Portugal. The Conference Chair is Isabel Ramos, Minho University, Guimaraes, Portugal and the Programme Chair is Anabela Mesquita from the Instituto Politécnico do Porto, Porto, Portugal.

ECRM is a recognised event on the European research conferences calendar and provides the opportunity for individuals, working in the area of business and management research to meet and discuss their experiences of using the varied and expanding range of research methods available to them.

The opening keynote for the conference is from Gill Green, University of Bolton. The second day will be opened by Dr João Álvaro Carvalho from the University of Minho with a talk entitled “Going one step back - what scientific method are you following?”.

In addition to the main themes of the conference there are mini tracks on Structured and Unstructured Observations in Qualitative Research, Construction and Validation of Scales and Indices, Mixed methods research, Teaching Research Methods and Methodologies and Narrative Inquiry. As usual ECRM is incorporating a PhD Colloquium and we have been impressed with the range of research and calibre of work that has been submitted for this.

With an initial submission of 1334 abstracts, after the double blind, peer review process there are 44 academic papers, 3 PhD papers and 2 work in progress papers published in these Conference Proceedings. These papers come from some 14 different countries including Australia, Brazil, Canada, Czech Republic, France, Malaysia, Poland, Portugal, South Africa, Thailand, The Netherlands, Ukraine, United Kingdom, and the USA. A selection of the best papers – those agreed by a panel of reviewers and the editor will be published in a conference edition of the EJBRM (Electronic Journal of Business Research Methods www.ejbrm.com). These will be chosen for their quality of writing and relevance to the Journal’s objective of publishing papers that offer new insights or practical help into the application of research methods in business research. Selected papers that have been presented in the Mixed methods research (MMR) mini track will be considered for publication in a special issues of the International Journal of Multiple Research Approaches published by eContent Management Pty. I wish you a most interesting conference.

Isabel Ramos, Conference Chair and Anabela Mesquita, Programme Chair July 2013

iv

Conference Committee Conference Executive

Isabel Ramos, Minho University, Guimaraes, Portugal Anabela Mesquita, Instituto Politécnico do Porto, Porto, Portugal Dr Ann Brown, CASS Business School, London, UK

Mini Track Chairs

Isabel Ramos, Minho University, Guimaraes, Portugal Dr Arminda Guerra Lopes, Escola Superior de Tecnologia, Polytechnic Institute, Castelo Branco, Portugal Dr Roslyn Cameron , CQUniversity, Australia Dr Paula Odete Fernandes, Polytechnic Institute of Bragança (IPB) – Portugal Dr Marie Ashwin, Normandy Business School, France Prof Mark Saunders, The Surrey Business School, UK Dr Cathi McMullen, Charles Sturt University, Australia.

Conference Committee The conference programme committee consists of key people in the research methodology and business management community, both from the UK and overseas. The following people have confirmed their participation: Khodayar Abili (The University of Tehran, Iran); Bulent Acma (Anadolu University, Eskisehir, Turkey); Hamimah Adnan (Universiti Teknologi MARA, Malaysia,); Gisela Schulte Agyeman (Learning Services, Hull City Council, UK); Maizam Alias (Universiti Tun Hussein Onn Malaysia, Malaysia); George Allan (University of Portsmouth, , UK); Andrew Armitage (Anglia Ruskin University, UK); Marie Ashwin (Ecole de Management de Normandie, France); Gabriela Avram (University of Limerick, Ireland); Joseph Azzopardi (University of Malta, Malta); Sue Balint (Westminster Business School, London, UK); Joan Ballantine (University of Ulster, UK); Heather Banham (Okanagan College, Kelowna, Canada); Edward Barratt (University of Newcastle Upon Tyne, UK); Andrew Basden (University of Salford, UK); David Bednall (Deakin University, Australia); Peter Bednar (Department of ISCA, Portsmouth University, UK); Lau Bee Theng (Swinburne University of Technology Sarawak Campus, Malaysia); Gary Bell (London South Bank University, UK); Angela Benson (University of Brighton, UK); Egon Berghout (University of Groningen, The Netherlands); Frank Bezzina (University of Malta, Malta); Milena Bobeva (Bournemouth University, UK); Stephan Boehm (RheinMain University of Applied Sciences, Germany); Ann Brown (CASS Business School, UK); Cathal Brugha (University College Dublin, Ireland); Roslyn Cameron (Central Queensland University, Australia); Marian Carcary (University of Limerick, Ireland); Sven Carlsson (School of Economics and Management, Lund University, Sweden); Prasenjit Chatterjee (MCKV Institute of Engineering, India); Vikas Choudhary (National Institute of Technology, Kurukshetra, India); Jyoti Choudrie (University of Hertfordshire , UK); Murray Clark (Sheffield Business School, UK); Aileen Corley (Liverpool John Moores University, UK); Felice Corona (University of Salerno, Italy); Reet Cronk (Harding Universtity, USA); Barbara Crump (Massey University, New Zealand); Geoffrey Darnton (Bournemouth University, UK); Linda Dawson ( University of Wollongong, Australia); David De Vaus (University of Queensland, Brisbane, Australia); Verena Dorner (Chair of Business Computing II, University of Passau, Germany); David Douglas (Staffordshire University, UK); Prokopios Drogkaris (University of Aegean, Greece); Ken D'Silva (London South Bank University, UK); Dev Dutta (University of New Hampshire, Durham, UK); Rohit Dwivedi (Rajiv Gandhi Indian Institute of Management Shillong, India); Hossein Ebrahimpour (The University of Kashan, Iran); Jose Esteves (Instituto de Empresa, Madrid, Spain); Mahmoud Fakhra (College of Business Studies, Kuwait); Jason Ferdinand (University of Liverpool, UK); Paula Odete Fernandes (Polytechnic Institute of Bragança, Portugal,); Elena Fitkov-Norris; (Kingston University;UK); Adriana Giurgiu (University of Oradea, Faculty of Economic Sciences, Romania); Gerald Goh (Multimedia University, Melaka, Malaysia); Karuna Gomanee (Regents College, UK); Andrea Gorra (Leeds Metropolitan University, UK); Gillian Green (School of Informatics, University of Northumbria, UK); Sayed Mahdi Golestan Hashemi; (Iranian Research Institute for Creatology; Iran); Yunke He (Okanagan College, Kelowna, Canada); Alex Hiller (Nottingham Trent University, UK); Alan Hirst (London South Bank University, UK); Clive Holtham (Cass Business School, City of London University, UK); Bob Hughes (University of Brighton, UK); Gordon Hunter (University of Lethbridge, Canada); Fahmi Ibrahim (Glasgow Caledonian University, UK); Rahinah Ibrahim (Universiti Putra Malaysia, Malaysia); Mohammad Intezar (King Saud University, Saudi Arabia); Wyn Jenkins (Staffordshire University, Stoke-on-Trent, UK); Kristina Jesperson (Aarhus University , Denmark); Bev Jones (Management Research Centre, Wolverhampton Business School, UK); Patricia Joubert (University of Swaziland, Swaziland); Helena Karjalainen ( Normandy Business School, France); "Diane Keeble-Allen (Anglia Ruskin University , UK);" Habibul Khondker (Zayed University, Abu Dhabi, United Arab Emirates); Cyril Kirwan (Cyril Kirwin Associates, Dublin, Ireland); Rembrandt Klopper (University of Kwa Zulu Natal, South Africa); Jenny Knight (University of Brighton Business School, UK); Deborah Knowles (University of Westminister, London, UK); Karl Knox (Nottingham Trent University, UK); Mortaza Kokabi (Shaheed Chamran, Iran); Branka Krivokapic-Skoko (Charles Sturt University, Australia); David Lamb (Edith Cowan University, Australia); João Leitão (Technical University of Lisbon, Portugal); Arminda Lopes (Instituto Politecnico, Portugal); Eurico Lopes (Polytechnic Institute of Castelo Branco, Portugal); Jonathan Lord (University of Salford, UK); Sam Lubbe (University of South Africa, South Africa); Sandy MacDonald (University College Northampton, UK); Dalgobind Mahto (Green Hills Engineering College, India); Garance v

Marechal (University of Liverpool, UK); Maria do Rosário Martins (Universidade Cape Verde, Portugal); Rachel McLean (University of Bolton, UK); Lisa McNeill (University of Otago, Dunedin, New Zealand); Ian Michael (Zayed University, Dubai, United Arab Emirates); Elisabeth Michielsens (University of Westminister, London, UK); Avinash Mulky (Indian Institute of Management , Bangalore, India); Graham Myers;(Durban University of Technology; South Africa); Emanuela Alia Nica (University Petre Andrei Iasi Romania); Hester Nienaber (Unisa, Pretoria, South Africa); Kersti Nogeste (RMIT University, Melbourne, Australia); Chetsada Noknoi (Thaksin University, Thailand); Miguel Nunes (Sheffield University, UK); Madeleine Ogilvie (Edith Cowan University, Perth, Australia); Abdelnaser Omran (Universiti Sains Malaysia, Malaysia); Steve Page (University College Chester, UK); Noel Pearse (Rhodes University, Grahamstown, South Africa); Rui Pimenta (Escola Superior Tecnologia Saúde Porto, Portugal,); Ganesh Prabhu (Indian Institute of Management at Bangalore, India); Diana Rajendran (Swinburne University of Technology, Melbourne, Australia); Thurasamy Ramayah (Universiti Sains Malaysia, Malaysia); Karsten Boye Rasmussen (University of Southern Denmark, Odense, Denmark); Theo Renkema (Rabobank Nederland, The Netherlands); Martin Rich (CASS Business School, UK); Kristina Risom (The Aarhus School of Business, Denmark); Bob Ritchie (Lancashire Business School, UK); Isabelle Royer (University Lyon 3, France); Maria Ryan (Edith Cowen University, Perth, Australia); Faith Samkange (SHMS Hotel Belvedere, Switzerland); Joseph Santora (ENPC, School of International Management, Paris , France); Risto Säntti (University of Vaasa, Finland); Mark Saunders (University of Surrey, UK); Dr Ousanee Sawagvudcharee (Stamford International University (Phetchaburi Campus) Thailand); Junaid Shaikh (Curtin University, Malaysia); Anshuman Sharma (College of Applied Sciences, Ministry of Higher Education, Oman); Marie Sheahan (Charles Sturt University, Australia); Ali Simsek (Anadolu University, Turkey); Chandranshu Sinha (Amity Business School, Amity University, India); Tammi Sinha (University of Portsmouth, UK); Ibrahim Sirkeci (Regent's College, London, UK); Peter Smith (University of Sunderland, UK); Rajeev Srivastava (Motilal Nehru National Institute of Technology, India); Prof Anthony Stacey (University of the Witwatersrand, Johannesburg, South Africa); Jim Stewart (Coventry Business School , UK); Kerstin Thomson (Stockholm University, Sweden); Claudine Toffolon (Université du Mans - IUT de Laval, France); Shiv Tripathi (Mzumbe University, Tanzania); Jim Underwood (University of Technology, Sydney, Australia); Arvind Upadhyay (University of Bergamo, Italy); Jocene Vallack (CQ University, Rockhampton, Australia); Ana Vasconcelos (University of Sheffield, UK); John Venable (Curtin University of Technology, Perth, Australia); Kevin Voges (University of Canterbury, Christchurch, New Zealand, New Zealand); David Wainwright (University of Northumbria, UK); Teresa Waring (Northumbria University, UK); Jon Warwick (London South Bank University, UK); Christine Welch (University of Portsmouth, UK); Roy Williams (University of Portsmouth, UK); Les Worrall (University of Coventry, UK); Lilia Zerguit (Sheffield Business School, Sheffield Hallam University, UK); Mingming Zhou (Nanyang Technological University, Singapore); Chaudhary Imran Sarwar (Creative Researcher, Lahore, Palkistan); Vibha Bhandari (College of Applied Sciences,Ministry of Higher Education,Oman, Oman); Prasenjit Chetterjee (MCKV Institute of Engineering, India); Branka Krivokapic-Skoko (Charles Sturt University, Australia); Rita Niemann (University of the Free State, South Africa); Nima Shahidi (Islamic Azad University_Nourabad Mamasani Branch, Iran); Ellen Caroline Martins (Organisational Diagnostics, South Africa); Udo Averweg (eThekwini Municipality and University of KwaZulu-Natal, South Africa); Nico Martins (University of South Africa, South Africa); Anabel Gutierrez Mendoza (Regent's College London, UK); Miles Shepherd (Bournemouth University Business School, UK); Leslie Dinauer (University of Maryland University College, USA); Hendrik Marais (University of Pretoria, South Africa);

vi

Biographies Conference Chair Isabel Ramos is an Assistant Professor with Aggregation Degree in the Information Systems Department of the Minho University, Portugal. Isabel is Secretary of the Technical Committee 8 (Information Systems) of IFIP – International Federation for Information Systems, as well as the Portuguese representative. She coordinates the research group “Information Systems and Technologies in Organizations” of the Centre Algoritmi and advises several PhD and Master Dissertations in the area of Knowledge Management and Innovation. She is the principal researcher in several projects in partnership with Portuguese companies and governmental agencies. She is author and co-author of four books and more than 5 dozens of scientific and technical papers in peer reviewed journals

Programme Chair Anabela Mesquita is a professor at the School of Accountancy and Administration of Porto (Politechnic Institute of Porto) member of the Agoritmi Center (Minho University). She has been involved in several projects and has published books and numerous papers in international journals and Conference Proceedings. She has been a member of the Programme and Scientific Committee of several Conferences and serves as Member of the Editorial Board and referee for IGI Global and as Associate Editor of the IRM Journal and is the Editor in Chief of the IJTHI. She has also been evaluator and reviewer for European Commission projects.

Keynote Speakers

Dr Gill Green is a Reader in Business Technologies in the School business and creative technologies at the University of Bolton. She is the co-ordinator of the doctoral school. She has worked previously for The University of Durham, University of Sunderland and Northumbria University with a domain focus on Information systems and research methods. Her current research interests are in the area of open source software initiatives, SMEs, organisational learning and music and creative industries. She is the Treasurer of UKAIS ( UK Academy of Information Systems) Dr João Alvaro Carvalho Department of Information Systems and Algoritmi Center, School of Engineering, University of Minho 4800-058 Guimarães, Portugal, jac@dsi.uminho.pt João Alvaro Carvalho is Full Professor at Department of Information Systems, School of Engineering, University of Minho, Portugal. He holds a PhD in Computing from the University of Manchester Institute of Science and Technology (UMIST), UK (1991). He currently is Head of Department at Department of Information Systems, University of Minho, and Director of the Doctoral Program in Information Systems and Technology at University of Minho. His research and teaching interests include: the foundations of information systems; information systems development; organizational development; meta-modelling; requirements engineering; knowledge management; and research approaches and methods.

Mini Track Chairs Dr Arminda Lopes holds a PhD in Human Computer Interaction from Leeds Metropolitan University, UK, an MSc in Speech and Language Processing from the University of Essex, UK, a Post- graduate degree in Information Systems from Instituto Superior Técnico, Lisboa, Portugal, and a Licentiate degree in Languages and Linguistics from Universidade de Lisboa, Portugal. She is a Professor at the Department of Informatics at Escola Superior de Tecnologia, Polytechnic Institute of Castelo Branco, Portugal.

Dr Roslyn Cameron is a Senior Lecturer in HRM at CQUniversity, Australia. Since using a mixed methods research (MMR) design in her PhD (Sequential Mixed Model Research Design) she has been an advocate for mixed methods research (MMR) and has continued her interests through teaching mixed methods and related aspects of research methods. Roslyn is currently in negotiations/collaborations with the Chairs of the Research Methods SIGs of the British Academy of Management (BAM) and European Academy of Management (Professor Bill Lee, UK and Professor Mark Saunders, UK) to discuss the writing/editing of a Multimethods and Mixed Methods in Management Reader.

vii

Dr Paula Odete Fernandes holds a PhD in Applied Economics and Regional Analysis (2005) is Professor of Management in the Polytechnic Institute of Bragança (IPB) - Portugal. She is researcher of NECE - Research Centre in Business Sciences. Her research interest include: Artificial Neural Network, Tourism, Entrepreneurship, Marketing Research, and Applied Research Methods. Participation in 4 international projects I&D and more than 65 publications in proceedings and scientific journals with referee.

Dr Marie Ashwin With almost two decades of experience in higher education in the UK and the Far East, Marie joined the Normandy Business School two years ago. She works with colleagues from around the world on teaching and research projects to enhance the standing of RM amongst staff and students, and add value to the teaching of methodology. A qualitative researcher by nature, she is currently investigating Social Networking behaviour with an international research team.

Professor Mark Saunders is Professor in Business Research Methods at the University Of Surrey School Of Management. He teaches research methods to undergraduate, master’s and doctoral students and is co-author of six management textbooks.

These include Research Methods for Business Students (2009, 5th edition, Financial Times Prentice Hall), which has also been translated into Chinese, Dutch, Vietnamese and Russian and Statistics: What you really need to know (2008, McGraw Hill). His research focuses upon research methods (including online methods and tools for understanding and improving organizational relationships) and human resource aspects of the management of change.

Dr Cathi McMullen is a Senior Lecturer in Marketing and a qualitative researcher with the Research Institute for Professional Practice, Learning and Education (RIPPLE) at Charles Sturt University, Australia. She utilises a narrative approach in a range of her research projects in marketing and higher education. This work encompasses a variety of contexts and theoretical concerns including professional identity, professional practice, storytelling and branding, and development of collaborative regional branding initiatives. Cathi is co-author of Teaching, learning and research in higher education: A critical approach. Routledge. 2010.

Biographies of Presenting Authors Mohamed Adil is doing research in Information Systems focusing on Decision Support Systems using Multi-Criteria Decision Analysis at the Information School of the University of Sheffield, UK. He holds a BA in Computer Science and Accounting and an MSc in Information Systems Management. He has been teaching in information systems and related disciplines for the last 12 years.

Zana Ahmed shareef received MSc degree in software engineering – distributed computing systems from University of Greenwich, UK. Software engineer and project management practitioner; worked as consultant for global management consulting firms serving clients ranging from financial, government, and telecommunications sectors. Zana is currently a PhD researcher at University of Brighton (U K); research interests include software project management, project risk management, and Case-based reasoning.

Deborah Anderson is a principal lecturer in Marketing at Kingston Business School, Kingston University. She has a PhD in Educational Research from Lancaster University and is particularly keen to develop and research initiatives designed to improve the student experience in learning and teaching.

Dr Marie Ashwin With almost two decades of experience in higher education in the UK and the Far East, Marie joined the Normandy Business School two years ago. She works with colleagues from around the world on teaching and research projects to enhance the standing of RM amongst staff and students, and add value to the teaching of methodology. A qualitative researcher by nature, she is currently investigating Social Networking behaviour with an international research team.

Dr Yevhen Baranchenko is a lecturer at the Department of Management of Sumy State University. Yevhen holds an MSc in Agribusiness from the University College Dublin and a PhD from Northumbria University. He teaches in the areas of project management and administrative management. Yevhen’s research interests include third sector organisations, sustainable development and higher education sector. viii

Graeme Baxter is a Research Assistant and part-time Lecturer in the Department of Information Management at Aberdeen Business School, Robert Gordon University. His research interests include: the provision of, need for, and use of government, parliamentary and citizenship information; the provision and exchange of information during public consultations; freedom of information legislation; and the use of the Internet by political parties and candidates during parliamentary election campaigns.

Dr David Bednall is Associate Professor of Marketing at Deakin University, Australia. His research interests include the legal use of market research, customer analytics and market research. His is editor of Marketing & Social Research which focuses on research methods.

Dr Angela M Benson is a Principal Lecturer in Tourism at the University of Brighton and Adjust Associate Professor at University of Canberra. Angela has published over 20 articles/ chapters on Volunteer Tourism, Sustainability and Research Methods. She is the Founding Chair of the Association for Tourism and Leisure Education (ATLAS) Volunteer Tourism Research Group.

Dr Frank Bezzina, PhD, is the Deputy Dean of the Faculty of Economics, Management and Accountancy, at the University of Malta. He teaches Buiness Statistics, Research Methods, Operations Research and Decision Modelling. He has a keen interest on statistical and methodological misconceptions and on data analysis methodologies applied to buisness and management research.

Zelma Bone is an academic at Charles Sturt University, Australia. She teaches a range of management subjects focussing on management communication and leadership. She has been involved with research projects in the areas of learning and teaching; workplace learning; aged care services; women in leadership; community development and engagement in natural resource management.

Dr. Ann Brown Senior Lecturer, Information Management, Faculty of Management, Cass Business School. Doctorate from City University (2005), based on work into problems and potential of Information Systems applications to create Business Value for organisations. Editor of Electronic Journal of Business Research Methods (EJBRM). Current researching IS implementation; IS exploitation by organisations; research methods for business and exploitation of electronic tools for teaching and learning.

Dr Roslyn Cameron is a Senior Lecturer and Head of Discipline- HRM at the School of Business and Law, CQUniversity, Australia. Ros is a Fellow of the Australian Human Resources Institute (FAHRI) and Co-Convenor of the Mixed Methods Research Special Interest Group of ANZAM. She has been granted $1.09 (AUD) million in research funding and has over 70 publications.

Nicholas Capaldi is Legendre-Soulé Distinguished Chair in Business Ethics at Loyola University, New Orleans. He also serves as Director of the newly created Center for Spiritual Capital. He is Associate Editor of the recently published Springer Encyclopedia of Corporate Social Responsibility.

Margarida Cardoso is a Sociologist with a MA in Anthropology (Social Movements). She’s an Information Systems PHD student at Universidade do Minho. Her work’s about open innovation and identity strategies. In 2012 she published “Rvolta, a case for open innovation: how can a SME be innovative in a competitive industrial environment”, co-authored by her advisor Isabel Ramos, PhD. Murray Clark Principal Lecturer, Organization Behaviour and Research Methodology, leader of DBA programme, Sheffield Business School, Sheffield Hallam University. Prior academic career after completing PhD. in Organizational Behaviour Manchester Business School, was manager British Coal and qualified mining engineer. Co-editor Business and Management Research Methodologies (Sage Library in Business and Management, 2006) and co-author The Dilemmas of Leadership (Routledge, 2006). Researches/publications centred mainly on trust and leadership.

David Clemson Senior Lecturer – Business & Sustainability, London South Bank University, Steering Group Partner London RCE, Co-Chair of LSBU’s Sustainable Development Group and chairs Innovative Teaching & Learning working group of COPERNICUS Alliance. Prior to academia involved in risk and investment management, organisational change & development and management consulting. Main research areas are risk, organisational stability, crisis management and simulations, technical analysis, socionomics, sustainability and transformative education.

Fariba Darabi DBA Researcher and Lecturer in International Business Studies at Sheffield Business School, Sheffield Hallam University. Studied MSc International Marketing at Sheffield Business School and was a practitioner, mainly in Export Industry in semi-public and private companies in overseas. Received best paper award at Sheffield Doctoral Conference 2011. Researches inter-organisational relationship and trust in Higher Education and Industry relationships. ix

Dr Tracey Dickson is an Associate Professor at the University of Canberra. Tracey’s Current research interests in alpine tourism, snowsport safety, mountain bike safety, volunteer legacies of mega sport events such as the Olympics and Paralympics and risk management in outdoor recreation. She is the National Secretary (Australia) for the International Society for Skiing Safety.

Sally Eaves is a Senior Manager in the IT and Telecommunications Sector with certifications in ITIL, Prince2 and Six-Sigma.A committed ‘practitioner‐researcher’, she is engaged in collaborative, intersectional projects with Sheffield Hallam University with whom she obtained a Distinctive MSc in IT and Management in 2012. Interests include methodological innovation, organisational narratives and knowledge management.

Fakis, Mr. Apostolos CSci, CStat, MSc. Senior medical statistician in R&D Department, Derby Hospitals NHS Foundation Trust and doctorate student in Health and Social Care Practice, University of Derby. Ex Vice-Chair of Derbyshire REC and expert reviewer of NIHR funding programs. Research interest lies on mixed methods research and statistical modeling of qualitative information.

Paula Odete Fernandes (PhD in Applied Economics and Regional Analysis, 2005) Professor of Management in Polytechnic Institute of Bragança (IPB) - Portugal. Researcher of NECE – Research Centre in Business Sciences. Research interests include: Artificial Neural Network, Tourism, Entrepreneurship, Marketing Research, and Applied Research Methods. Participation in 4 international projects I&D and over 70 publications in proceedings and scientific journals with referee.

Elena Fitkov-Norris is Principal Lecturer in Information Management at the Department of Informatics and Operations Management, Kingston Business School, London, UK. She received a BSc in Mathematics and Management from King’s College London and an MSc and a PhD from University College London. Dr. Fitkov-Norris’s research interests are the relationship between study habits and learning.

Dr. Mary FitzPatrick (PhD in Marketing) is a Senior Lecturer in the Department of Marketing, Waikato Management School.

Her research explores interactions between the consumers and providers of services, in particular the experience and construction of relationships within healthcare services. Mary is a qualitative researcher experienced in implementing Memorywork.

Dr. Sjoerd A. Gehrels (1959) started in Dutch higher education in 1989, after 10 years in the hospitality industry. In Stenden University (Netherlands) he is currently course leader for the MA in International Service Management, and researcher. Sjoerd holds MSc (University of Surrey 1999), MBA (Oxford Brookes University 2004) and EdD (University of Stirling 2013).

Ing. Simona Hašková teaches and is doing her Ph.D. at the Faculty of Management at the University of Economics in Prague.

In her teaching and scientific activities she focuses on the problems of managerial economics, on case studies of financial analysis which managers have to deal with and on applied logic in management.

Dr Alan Hirst is a visiting professor at the Ecole de Management de Normandie. Previously he was a senior lecturer in marketing at London Southbank University. Amongst his international roles are Visiting Professor at Assumption University, Bangkok, Thailand. His research interests include research design, online shopping and social media.

Lorraine Johnston is a Senior Lecturer teaching strategy and entrepreneurship at Newcastle Business School, Northumbria University. Her research focuses on the impact of national policy at the local level. This research analyses the capability of higher education institutions, local governments and small business communities responding to change and complexity in embedding strategic change management/leadership.

Constance Rinen Justin Wah, Bachelor Degree was in 2006 (Information Systems) and Corporate Master in Business Administration in 2011. Currently, I am a lecturer and my research philosophy encompasses building a bridge between the fields of Learning Organization and I.T Management.

Deborah Knowles is a Principal Lecturer at the University of Westminster. She divides her time among teaching on Organizational Behaviour modules, acting as the Course Leader of the BA International Business degree, and research. The study of home staff visits to international partner institutions combines the interests of all three of these roles.

Dave Lamb is an academic with university teaching and research experience in the UK, New Zealand and now Australia. He is an experienced practioner and qualitative researcher who utilises a range of qualitative techniques in his research and is a keen advocate of journal writing as part of the research experience.

x

Ron Lawson is senior lecturer in strategic operations and logistics management at the University of Sunderland. Ron worked in senior operations management for over 25 years in the public and private sectors before joining the university. Ron's main academic interests lie in storytelling, and exploring methods of strategic reconciliation between market requirements and operational capabilities.

Dr Ophillia Ledimo has a PhD in Industrial and Organisational Psychology and is currently working at The University of South Africa; Department of Industrial and Organisational Psychology. Her field of specialization is Organisational Psychology and has presented papers at several national and international conferences within this field.

Ludmiła Łopacińska - a researcher in the Institute for Sustainable Technologies – National Research Institute in Radom (Poland). Specialist in the evaluation methodology for strategic research programmes. Research interests - strategic management and strategic changes in research organisations. Dissemination of the research results on international conferences and in many journals.

Isabel Maria Lopes has a PhD in Information Systems and Technologies at the University of Minho. It is Adjunct Professor in the School of Technology and Management, Polytechnic Institute of Bragança. Its research interests in security of information systems and research methods in general.

Dr. Arminda Lopes holds a PhD in Human Computer Interaction from Leeds Metropolitan University, U.K.; she is a Professor at Polytechnic Institute of Castelo Branco, Portugal. Research Interests: Information Systems; Human Computer Interaction; Research Methodologies. Other Research Interests: Social Informatics, Creativity and Innovation, Universal Design.

Hendrik (Bok) Marais holds a Ph D in experimental social psychology and specialises in research methodology and management as well as innovation policy. He is currently Professor in the Graduate School of Technology Management at the University of Pretoria.

Rita Marcella is Dean of Faculty at Aberdeen Business School, Robert Gordon University. Her research interests include: the provision of, need for, and use of government, parliamentary and citizenship information; Internet use by political parties and electoral candidates; freedom of information; business information provision; information behavior and information systems in health and safety management.

Dr Ellen Martins Co-Director: Organisational Diagnostics . Co-director of Organisational Diagnostics. She holds a doctorate degree in Information Science focusing on knowledge retention. Her Masters degree (cum laude) focused on the impact of organisational culture on creativity and innovation. She has extensive experience in numerous behavioural assessments and qualitative research methods such as interviews and focus groups.

Nico Martins holds a PhD in Industrial Psychology and is with the Department of Industrial and Organisational Psychology at The University of South Africa and specializes in the field of Organisational Psychology. He published several articles and presented papers at several national and international conferences in organisational culture/climate, organisational trust,organisational diagnoses and research .

Adam Mazurkiewicz - the Director of the Institute for Sustainable Technologies, National Research Institute in Radom, Poland. Coordinator and supervisor of many R&D projects resulting in industrial implementation. Expert in the area of systems engineering, machine construction and maintenance, materials engineering, knowledge and technology transfer. Author and co-author of over 250 publications, including 10 monographs.

Dan McAran is enrolled in the DBA program at Henley Business School (University of Reading). He is employed fulltime as a Product Manager at Do Process Software, a provider of legal technology to the Canadian Market. His doctorate research is titled Integrating the Intuitive into User Acceptance of Technology Theory.

Dr Grace McCarthy is the Higher Research Degree Coordinator at Sydney Business School, University of Wollongong, Australia. In 2012, Grace received an Australian Government Citation for her outstanding contribution to student learning, for ‘using a coaching approach to inspire a love of learning among students and colleagues’.

Angela McGrane is a Senior Lecturer teaching research methods at Newcastle Business School, Northumbria University. She recently returned to this role after some time working as a freelance consultant specialising in research and data analysis.

Her current research uses personal construct theory to explore the effect of work-integrated learning on the self-perceptions of placement students.

xi

Cathi McMullen is a Senior Lecturer in Marketing and a qualitative researcher with the Research Institute for Professional Practice, Learning and Education (RIPPLE) at Charles Sturt University, Australia. She utilises a narrative approach in a range of marketing and higher education projects. Cathi is co-author of Teaching, learning and research in higher education: A critical approach.

Ana Paula Monte PhD Management Sciences – specialization Corporate Finance, 2008) Professor of Management, Polytechnic Institute of Bragança (IPB) – Portugal. Researcher of NECE – Research Centre in Business Sciences. Research; Investor sentiment/Behavioral Finance, Entrepreneurship, Corporate social responsibility and financial performance. Balanced Scorecard and other performance management systems for SME. Participation international projects I&D. Published papers in book chapters, proceedings and scientific journals with referee.

Gerhard Moolman is a statistician and information system development specialist by training. He has experience in designing and implementing information systems and has managed research funding programmes in the NRF. He is currently Manager: Business Intelligence at the National Research Foundation (Pretoria, South Africa).

Susana Andrea Morais research focuses on crowdsourcing. She is particularly interested in a specific kind of on-line initiatives somewhat similar to idea contest used to get creative or innovative tasks done. I’m looking for patterns of online creative behaviors during the interplay between the crowd and those kinds of platforms.

Graham Myers is a retiree from the Durbnan University of Technology. His interest has been in Research Methods for the past 16 years. He also lectures in Financial Management and Taxation which formed part of his doctoral studeies.

Dr. Arlene .J. Nicholas - graduate director of the MBA/MS Management programs and the coordinator of the undergraduate Business Administration major in the Business Studies and Economics Department. Management experience in retail/nonprofit. Papers have been presented at national and international conferences and published materials include a book, a chapter in a book and in scholarly journals.

Miguel Baptista Nunes – BSc, MSc, PhD, MBCS, FHEA, FIMIS, is a Senior Lecturer at the Information School of the University of Sheffield. He has been involved in research in information management and information systems for the last 20 years. He has published widely in these fields (circa 180 articles) and participated in several EU and UK funded projects.

Mírian Oliveira obtained her doctoral degree in Business Administration from the Universidade Federal do Rio Grande do Sul (UFRGS) in 1999. She is a professor and researcher at Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS), Brazil.

Her current research interests include knowledge management, information systems and research method.

Sonya Pearce is an Aboriginal woman from the Gooreng Gooreng nation of Central Queensland and is employed as Senior.

Lecturer/Director of Indigenous Programs at UTS Business School. University of Technology, Sydney. Sonya‘s PhD work is on Aboriginal. Women Entrepreneurship and Social Change.

Dr. Guo Chao Alex Peng is a Lecturer in Information Systems in the Information School at the University of Sheffield, UK. He st holds a BSc in Information Management (1 Class Honours) and a PhD in Information Systems. He has more than 32 publications in the forms of book chapters, high quality journal articles, and referred full conference papers.

Dr. Phallapa Petison is current working as a Business Management Program at the College of Management, Mahidol University. Dr. Phallapa’s primary research interests include qualitative research and localization strategies.

Magdal Pienaar-Marais is a theologian by training and currently enrolled for an MPhil degree in Science and Technology Studies. She has experience in the evaluation of research programmes and is Manager: Strategic Performance Planning and Reporting at the National Research Foundation (Pretoria, South Africa).

Sara Ramos PhD in Psychology and Assistant Professor in the Department of Human Resources and Organizational Behavior at ISCTE-Lisbon University Institute.

Areas of teaching: research methods, qualitative methods, work psychology and human resources. The latest research has focused on the relationship between health, age and work more specifically on the topic of aging and work.

Eurico Ribeiro Lopes My background is Computer Systems and Electrical Engineering (1987) Technical Institute of Lisbon (IST) and a Master in Computer Science (1992-93 ) University of Essex, UK. I have a Pos-graduation in Information Systems (200304) from IST. Since Jun'09 I also have a PhD in information Management from Leeds Metropolitan, University, UK.

xii

Martin Rich initially worked in the information systems sector as a consultant and project manager, before joining Cass Business School as a lecturer. He is currently course director for the BSc in management, and is active in applying innovative approaches backed up by pedagogic research to teaching and learning.

Hayley Rowlands is a Research Assistant at Aberdeen Business School, Robert Gordon University. She holds an undergraduate degree in Historical Studies from The University of Glasgow and an MSc in Information and Library Studies from Robert Gordon University. Her research interests include: information literacy, information systems, oil and gas industry, leadership and organisational behaviour.

Professor Mark NK Saunders is Director of Postgraduate Research Programmes for the Faculty of Business Economics and Law and holds the chair of Business Research Methods at the Surrey Business School, University of Surrey. He research interests include research methods and human resource management aspects of the management of change, in particular organisational trust.

Sílvia A. Silva is a Professor in ISCTE- Instituto Universitário Lisboa in the field of organizational psychology applied to occupational safety and wellbeing. She has been teaching courses related with Organizational Behavior and Research Methods in several Undergraduate Degrees, Master and PhD, mainly in Psychology and Management Degrees.

Prof Anthony Stacey is an Associate Professor in decision sciences and research methodology at the Graduate School of Business Administration, University of the Witwatersrand, Johannesburg. Having trained as an Electrical Engineer, his research interests now include techniques for analysis of ordinal level survey data, multivariate analytical methods, group decision making, group dynamics, and numerical modelling.

Leonel Tractenberg Brazilian Psychologist, PhD and MSc in Education, professor of Organizational Human Resources Psychology at the State University of Rio de Janeiro, Brazil. In 2008 was invited scholar of the Work Organizations and Personnel Psychology Erasmus Mundus Master Programme at the universities of Coimbra and Barcelona. Also worked as e-learning manager, online tutor and instructional designer.

John Walton is a Principal Lecturer at Sheffield Hallam University; teaching and researching in the areas of organisational strategy, cultural dynamics and knowledge management. Aligning with substantial high-technology industry experience, he jointly developed the breakthrough “hybrid manager” IT and Management MSc programme. His research interests embrace third sector sustainable development, organisational change and methodological innovation.

Ara Yeghiazarian graduated in electronic engineering in 1984 and researched in artificial limbs for his MPhil degree. Ara moved to Kingston Business School in 1990, where he has been a course director for the MBA and masters courses since 1998. Ara’s research interests are investigation into learning preference based on VARK amongst university student.

Dr Andrey Yukhanaev is currently employed by Newcastle Business School of Northumbria University. His current role is a senior lecturer in Strategic Management, International Business and Economics. He is an active researcher in the areas of economic development in transition economies, international business strategies, institutional economics, corporate governance and international development.

xiii

Selecting Public e‐Procurement MCDA Methods Using a Three Tier Evaluation Mixed Method (TTMM) Research Model Mohamed Adil, Miguel Baptista Nunes and Guo Chao Peng Information School, University of Sheffield, Sheffield, UK m.adil@sheffield.ac.uk j.m.nunes@sheffield.ac.uk g.c.peng@sheffield.ac.uk

Abstract: Procurement in Public Sector has always been controversial and heatedly discussed. Methods and approaches aiming to support decision making have been the subject of research since the emergence of the Industrial Revolution in the late 1800s. However, and despite efforts of transparency in the public sector, it is not until automatisation of procedures and the systematic use of IT that the procurement decision making has had the need for the support of precise and unambiguous decision making methods. With the emergence of e‐procurement in the public sector, both response to and decision making on tenders is now heavily dependent on Multi‐Criteria Decision Analysis (MCDA) methods. The research that is the base for this paper aimed at identifying a suitable MCDA method for the specific legal and functional needs of the Maldivian Public Sector. In responding to this research question, the researchers developed a three tier evaluation research approach that is reported in this paper. This approach addressed the three different aspects that determine the suitability of an MCDA method to a particular public sector context: the legal framework, the operational requirements and the characteristics of particular MCDA methods. This resulted in the need to identify the specific legal framework in the Maldives that is defined by the National Constitution, the Public Finance Act and Fiscal and Financial regulations. Furthermore, there was a need to understand the operational aspects of the tendering process, namely the requests for tender, the analysis of the tenders and the final decision making process. Finally, it was necessary to analyse how the characteristics of the different MCDA methods perform against both legal and operational constraints and requirements identified. This paper reports on the research design created to support this process, namely a combination of literature review of both academic and grey literature, a set of focus groups interviews led in the Maldives with public servants and public managers and finally a desk‐top comparative analysis of MCDA methods.

Keywords: multi‐criteria decision analysis, public sector procurement, decision model, evaluation research, critical literature review, comparative analysis

1. Introduction “The function of purchasing is almost as old as the history of man” (England 1967, p. 3). Purchasing has been an important function for everyone’s life. (England 1967). Every organisation buys material and services from suppliers and uses these in their operations (Monczka et al. 2010).

The purchasing function has evolved (Leenders & Fearon 1997; Monczka et al. 2010) and purchasing terminologies like purchasing, procurement, supply, supply‐chain, materiel, materials management, sourcing and logistics are used almost as similar terms (Leenders & Fearon 1997; Leenders et al. 2006; Weele 2000).

There is no agreement on the definition of any of these terms described (Leenders & Fearon 1997; Weele 2000) and all the definitions of purchasing are open to criticism (Lysons & Gillingham 2003). For this research purpose the term procurement is used, because research context uses the term procurement.

The research that is the base for this paper focuses on designing a decision model for e‐Procurement Decision Support Systems (DSS) using MCDA for public education sector in Maldives. Public sector procurement has a rigid structure enforced by law and regulations (Leenders & Fearon 1997). The features and characteristics of public sector procurement are based on four major public sector principles; non‐discrimination, equality, transparency and proportionality (Weele 2010). This results in an organised step‐by‐step procedure for public sector procurement. However, this research focuses only on decisions that are based on the performances of the suppliers against a preset list of criteria. These criteria allow public sector institutions to state differentiated priorities when they announce for bids or tenders. This situation creates a context in which MCDA techniques can be applied to the evaluation.

There are multiple MCDA methods, see Table 1. However, the applicability of these methods to the Maldivian procurement processes, having into account the multiple legal and regulatory constraints and expectations of the public sector, has not been researched in depth. The research aimed to study the applicability and suitability of a large set of MCDA methods (Table 1), identified in the literature, to the Maldivian context. The

1

Mohamed Adil, Miguel Baptista Nunes and Guo Chao Peng MCDA methods used in this research involve linear weighting methods, single synthesising criterion or utility theory, outranking methods, fuzzy methods and mixed methods.

To answer the research question, a three tier evaluation model was developed. It involves qualitative comparative analysis of characteristics of MCDA methods against legal requirements and operational requirements of the Maldivian context to filter applicable methods. This was followed by quantitative analysis of the results of the filtered methods by using real procurement data in order to identify the most suitable methods. For qualitative analysis purpose, literature reviews of MCDA methods, and public sector procurement with focus to research context was done. A field research of focus groups were conducted with public education sector bid evaluation committee (BEC). Additionally, for the quantitative analysis, selected documented data on procurement evaluations were collected.

2. Procurement in public sector There are four major principles in public procurement. They are non‐discrimination, equality, transparency and proportionality (Weele 2010). The features and characteristics of public sector procurement are based on these principles.

The objectives of public sector procurement is the same as private sector (Leenders & Fearon 1997) and many authors define procurement objective as to purchase the right quality of material, at the right time, in the right quantity, from the right source, at the right price (Baily et al. 1994; England 1967; Leenders & Fearon 1997; Weele 2000).

Public sector procurement has certain characteristics that are determined by the need to maintain the public sector principles listed above. These characteristics are general and country specific based on legislative requirements of the country. Moreover, public procurement must adhere to the guidelines provided by public authorities. In every country, public procurement must comply with specific legislative requirements (Falagario et al. 2012).

The public purchasing laws require the contract to be awarded to the lowest capable bidder who fits into the requirements laid by invitation for bid (Brown et al. 1984; Falagario et al. 2012; Leenders & Fearon 1997).

Public procurement has limited flexibility and narrow evaluation criteria when dealing with bid evaluation as it has legal bindings (Leenders & Fearon 1997). However, in general, public sector procurement requires selecting the bid which provides the most advantages to stakeholders.

3. Three tier evaluation model The evaluation methodology designed for the research project and proposed in this paper, involves a three tier mixed‐method (TTMM) process of evaluation as shown in Figure 1. In first tier of evaluation, comparative analysis of MCDA methods against legal requirements of the problem context is considered. In the second tier evaluation, comparative analysis of MCDA methods against operational requirements gathered from a field research involving focus groups is considered. The first two tiers of evaluation are qualitative comparative analysis done together to filter applicable methods for evaluation for suitability analysis. The filtered MCDA methods from the qualitative comparative analysis of the two tiers will be used to apply real procurement data sets in the third tier of evaluation. The results of the MCDA methods will be mathematically and statistically analysed and compared with public sector requirements of the research context to identify the appropriate decision model.

For the first tier evaluation is comprised of a critical literature review of MCDA methods emerging in academic sources as well as a review of grey literature concerning both general and Maldivian public sector procurement requirements, regulations and legislation. A list of identified methods is then compared against the results of the grey literature review, resulting in a filtered list of methods.

The second tier evaluation aims at testing the suitability of the filtered methods to the operational context of procurement decision making in the Maldives. A series of focus groups were conducted with public sector procurement decision makers of the education sector. The results of the focus group analysis were then compared with the affordances of the methods in the filtered list resulting in further filtering.

2

Mohamed Adil, Miguel Baptista Nunes and Guo Chao Peng

Legal Requirements

Quantitative Analysis

Qualitative Analysis

MCDA Methods Evaluation T ier 1

MCDA Methods

Operational Requirements Evaluation T ier 2

Public Sector Requirements

Procurement Data Evaluation T ier 3

DSS Model

Figure 1: Three tier mixed‐method (TTMM) of process of evaluation for the selection of MCDA method to serve public procurement The third tier evaluation aims at test the operational performance and robustness of the methods and lead to a final subset of applicable methods. In this case, a selected real‐life procurement data sets were collected from public sector education institutions. The MCDA methods were then tested using these data sets. These test included sensitivity and robustness as well as performance tests against pre‐define performance criteria.

4. MCDA methods MCDA is one of the most well‐known branches of decision making (Triantaphyllou 2000). MCDA is important in decision making when a wide number of factors are concerned for decision (Amponsah 2011). MCDA include wide variety of methods which share two main common features: the concepts of alternatives and attributes (Triantaphyllou 2000). The MCDA methods listed in Table 1 are based on the initial work of (Guitouni & Martel 1998) extended by additional entries found in literature. These methods are the reference methods used in this study.

In the literature, MCDA methods are grouped in categories based on their characteristics. Table 1 lists the major MCDA methods in the five categories found in literature (Figueira et al. 2005; Guitouni & Martel 1998; Ho et al. 2010). The categories are: linear weighting and elementary methods; single synthesising criterion or utility theory; outranking methods; fuzzy methods: and mixed methods.

5. Legal requirements The Maldivian procurement processes are based on legal requirements stipulated by the Dhaulathuge Maaliyyathu Gaanoonu 2006 (literally, Public Finance Act 2006) and Dhaulathuge Maaliyyathuge Gavaaidhu 2009 (literally, Public Finance Regulation 2009). The responsible body for governing public sector finance is assigned by President of Maldives (Dhaulathuge Maaliyyathu Gaanoonu 2006) and the governing body is Ministry of Finance and Treasury (MoFT) (Dhaulathuge Maaliyyathuge Gavaaidhu 2009). As MoFT is the assigned body for public finance regulation by the President of Maldives, MoFT creates and amends Public Finance Regulation requirements (Dhaulathuge Maaliyyathu Gaanoonu).

These two legal documents governing public sector procurement in Maldives are studies to identify the legal requirements of public sector procurement in the research context as education sector also follows the same legal requirements.

3

Mohamed Adil, Miguel Baptista Nunes and Guo Chao Peng Table 1: MCDA methods (Adapted from Guitouni and Martel (1998)) No

Method

Author(s)

6 7 8 9 10 11 12 13

Linear weighting and elementary methods Weighted Sum Lexicographic method Conjunctive method Disjunctive method Maximin method Single synthesizing criterion or utility theory TOPSIS MAVT UTA SMART MAUT AHP and ANP DEA COPRAS Outranking methods

Hwang and Youn (1981) Keeney and Raifa (1976) Jacquet‐Lagreze and Siskos (1982) Edwards (1971) Bunn (1984) Saaty (1980), Saaty (2005) Talluri et al. (1999) Zavadskas et al. (2007); Chatterjee et al. (2011)

14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

ELECTRE ELECTRE I ELECTRE IS ELECTRE II ELECTRE III ELECTRE IV ELECTRE TRI PR OMETHEE PROMETHEE TRI PROMETHEE/GAIA technique NAIADE ELECCALC UTADIS MELCHIOR ORESTE REGIME PROMSORT EVAMIX QUALIFLEX Fuzzy methods

De Boer et al. (1998); Dulmin and Mininno (2003) Roy (1968) Roy and Bouyssou (1993) Roy and Bertier (1971) Roy (1978) Roy and Hugonnard (1982) Yu (1992); Mousseau et al. (2000) Dulmin and Mininno (2003) Figueira et al. (2004) Dulmin and Mininno (2003) Munda (1995) Kiss et al. (1994) Doumpos et al. (2001) Leclerc (1984) Roubens (1980) Hinloopen and Nijkamp (1982) Araz and Ozkarahan (2007) Voogd (1983) Paelinck (1978)

33 34 35 36 37 38

Fuzzy relationship hierarchy Fuzzy set approach Fuzzy suitability index (FSI ) Fuzzy weighted sum Fuzzy miximini AI methods

39

CBR Mixed methods

Lin and Chen (2004) Sarkar and Mohapatra (2006) Bevilacqua et al. (2006) Baas and Kwakernaak (1977) Bellman and Zadeh (1970) Ng and Skitmore (1995); Vokurka et al. (1996); Kwong et al. (2002); Choy et al. (2002); Choy et al. (2003); Choy et al. (2005) Ng and Skitmore (1995); Choy et al. (2003)

40 41

Martel and Zaras method Fuzzy conjunctive/ disjunctive method

Martel and Zaras (1990); Martel and Zaras (1995) Dubois, Prade and Testemale (1988)

1 2 3 4 5

Churchman and Ackoff (1954) Roy and Hugonnard (1982) Hwang and Youn (1981) Chen and Hwang (1992) Hwang and Youn (1981)

5.1 Maldives government guidelines for procurement Dhaulathuge Maaliyyathuge Gavaaidhu prescribes a standard guideline for public sector procurement. The main points in the guidelines provided by the public finance regulation in context of the research dictate that the public procurement should:

be a transparent process;

4

Mohamed Adil, Miguel Baptista Nunes and Guo Chao Peng

be able to create competitive biddings in order to provide fair opportunities to all capable suppliers;

done according to the National regulation and be the same for all bidders;

be able to use modern technology to support the process;

be led by a clearly identifiable and responsible person for the procurement;

request that whenever imported products are to be purchased, the suppliers must have an import licence, be in continued business and conform with Maldivian regulations;

requesting a formal disclosure document to be submitted with the bid indicating if a supplier has any family or business relationship with any senior official;

request that in case of buying bulk of materials a wholesale rate must be provided by the supplier, rather than the retail; and

strictly follow the approved budget and any deviations can only be justified if it risks a person’s life or if it stops any basic public service.

Moreover, every public department should submit their annual budget in advance for the next year and procurements should be listed and approved in this budget.

5.2 Bid evaluation criteria Even though there is a large variety of criteria used in public sector procurement, there are compulsory criteria that must be used in supplier selection in public sector in Maldives (Dhaulathuge Maaliyyathuge Gavaaidhu).

Dhaulathuge Maaliyyathuge Gavaaidhu assigns the responsibility to a BEC or a national level a National Tender Board (NTB) to evaluate the bids or tenders based on price and duration. In addition, it prescribes to look for the following criteria:

Financial capability of the supplier;

Technical capability of the supplier;

Justifiability of the prices submitted by suppliers compared to estimated price of the procurement material or service;

Similar past experiences of the suppliers in terms of size and its execution;

Any other important criteria perceived by the public sector department.

All bids or tenders should follow the above mentioned information requested by tender announcement through a bid or tender submission form. The bids or tenders also should cover a summary of the current work undertaken elsewhere by the supplier and its value (Dhaulathuge Maaliyyathuge Gavaaidhu). Every public sector institution, BEC and NTB is bound to follow these criteria and guidelines for public sector procurement. Since the public sector procurement requires more than one criterion to be considered in evaluation of bids or tenders, an MCDA method needs to be used for evaluation.

5.3 Results of the evaluation based on legal requirements The need to adhere these legislation and procurement regulations led the selection of 4 methods out of more than 80.

For instance and to illustrate the process, MAVT (multi‐attribute value theory) were rejected. MAVT are firmly grounded in von Neumann and Morgenstern’s utility theory and assumes an existence of a value function, based on utility maximisation (Ananda & Herath 2003). MAVT is built up of a few basic axioms and starts from the basis that all things are comparable. DM preferences are represented as a set of estimated scores for the performance of the alternatives on the measurement criteria. Weights must be created based on the relative importance of criteria. This approach fits well with scientific methods and it is a transparent technique (Simpson 1996).

MAVT places all the potential alternatives onto the same scale making it possible to make comparisons globally. MAVT enforces comparability across criteria. MAVT has strict mathematical basis, therefore the data input into the model must satisfy specific conditions, such as transitivity (Simpson 1996). This issue of common scale of comparability in every criterion is not in line with public sector regulations as any

5

Mohamed Adil, Miguel Baptista Nunes and Guo Chao Peng procurement can have diverse criteria mentioned in 5.2. In fact, criteria may not even be possible to represent in a scale, such as Boolean or nominal criteria. Therefore, MVAT was not applicable for public sector procurement for the research context.

6. Operational requirements In order to identify the operational requirements, his field research involved focus group discussions with BEC members of public sector education institutions. Since the aim of using focus groups in this research project was to know how the current process works and the expected characteristics of an ideal decision making method for evaluation, it was expected that group members would contribute to refine and add missed points by colleagues. It was also expected that the focus groups would stimulate the respondents to engage in in‐ depth discussions, help them in recall events and therefore allow richer data than answers from individual interviews (Flick 2009).

6.1 Sample size and sampling strategy Ministry of Education (MoE) represents 213 schools for which most of the procurements are handled by MoE’s procurement department. However, depending on the procurement, MoE decides to have procurement processed at 19 different schools. Therefore, there are 20 distinct BECs for 213 schools and MoE. The participants of the focus groups in this research involved senior officials of the Ministry of Education and other government institutions represented in the BECs. Since the participants are the senior officials and every BEC has a limited number of representatives, the data collection strategy aimed to have a maximum output of the maximum possible number of focus groups. During the initial meetings, the MoE stated that every institution under its responsibility follows the same procedure and have the same expectations in terms of procurement.

The researchers requested to focus groups with at least three BECs in three different locations, with three different institutions, in order to verify and see if there is any difference in operationalization of regulations and to enrich the data. Thus, the focus groups are done with BECs of MoE, a primary school, and a secondary school which covers the majority of the spectrum of procurements under MoE. These three groups cover BEC members who make procurement decisions for 195 schools out of 213 schools, covering 91.5 percent of the schools under MoE.

6.2 Group size and composition Following the principles of group size and composition proposed by Barbour and Kitzinger (1999), this research did not mix BEC members from one institution to the other institution in focus groups. Since the BEC has limited members and the members are senior staff as described above, having multiple responsibilities other than BEC function, it is not practical to get every member of the BEC to the focus group, therefore the size was set to minimum three members and maximum of all BEC members of target focus group. The recruitment was done initially having a meeting with a director of Ministry of Education followed by formal meetings arranged with heads of the chosen institutions to arrange focus group meetings with the BECs of the institutions. The institutions allocated meeting times based on the availability of at least minimum requested size of BEC members for the focus group. The sessions were held in meeting rooms of the institutions making it easier for the participants to attend. As suggested by Barbour and Kitzinger (1999) to have an ideal room, meeting rooms of the institutions were chosen as they are quiet and comfortable, free from interruptions.

6.3 Recording, transcribing and analysing focus group data This research used video recording and this it helped to easily identify individual speakers and in the transcription. It also helped to keep track of the sequence of the talk in transcription. Focus group transcription can be difficult because of sudden leaps, and interruptions by group members (Barbour & Kitzinger 1999). Therefore, video recording helped the researchers to overcome these difficulties in analysis.

Furthermore, these focus groups were to discuss technical procedures and information and in this case video recording helped to recap the verbal and non‐verbal explanations given. Analysing focus group data usually involves the same processes of any other quantitative data analysis. However, group context is referenced by analysis of groups rather than individuals and keeping the balance between concepts provided by the group and individual ideas within it (Barbour & Kitzinger 1999). In the case of this research thematic analysis was used to code the transcripts and establish a list of operational requirements, needs, problems and wishes.

6

Mohamed Adil, Miguel Baptista Nunes and Guo Chao Peng

6.4 Results of the evaluation based on operational requirements The need to support these operational requirements, needs, problems and wishes and procurement regulations led the rejection of additional of 2 methods out of previously filtered 4. For instance and to illustrate the process, Weighted Sum, which in fact is the current MCDA in use by the MoE, was rejected as not suitable for operational reasons. Weighted Sum is likely to be the most common method for supplier evaluation is a compensatory method (Guitouni & Martel 1998). This method calculates the ratio for each attribute for every supplier by dividing the performance values of the attribute by the maximum value of the attribute, in case of input factors, subtracting these ratios from 1. Next, the allocated weight for each attribute is multiplied by the ratio calculated to get the weights for individual attributes. Finally, all the calculated weights for each individual attribute for every supplier is added to get the total figure for the supplier (Falagario et al. 2012).

Due to the rationale followed by the weighting, most people would, at first glance, accept that the procedure is logical and commonsensical. However, the use of this weighed sum procedure is in fact the most common mistake in public procurement procedures (Mateus et al. 2010). Mateus et al. (2010) further explained with an example that the definition of weights is completely arbitrary and inconsistent with the real preferences of the procurement authority. With regard to compensation, these authors provide the following example: In a case of only two criteria selected, if 75% weight for one criterion (A) is defined and 25% weight for another criterion (B) then losing 10 partial points on criterion A (75% x ‐10 = ‐7.5 overall points) is equivalent to gaining 30 partial points on criterion B (25% x +30 = +7.5 overall points). Since the weights embody trade‐offs, the assignment of weights will have to take into account the way those values were identified, that is, the performance levels set for each criterion (Mateus et al. 2010).

It became very apparent that BEC members were keenly aware of these risks, but were struggling to identify an alternative approach. Therefore, despite being one of the most commonly used methods in procurement; weighted sum does not seem to be ideal for public sector procurement in the Maldives. This is specially so due the vulnerability of the method to manipulation by both suppliers and buyers, as well as the very high risk of the tendered products or services approved no meeting the public sector principles and standards behind regulations. This represents of course a dissonant finding with reality of practice and it is expected that this study may contribute to a rectification discussion on current procurement practices.

7. Quantitative analysis The quantitative analysis component of this study uses selected procurement data sets collected from three distinct public sector education institutions working at three different levels. The three different public sector education institutions are a primary school, a secondary school and MoE. These sets of previously evaluated procurement data along with results of the evaluation are collected from the three levels of institutions providing the chance to have more diversity in the data use on the education sector. The data sets were applied to the MCDA methods filtered through by the qualitative comparative analysis done for tier 1 and 2 of the overall evaluation. Since, the final set of competing methods obtained in this case was 2, namely TOPSIS and COPRAS, the final tier of the evaluation became a comparative process where the 2 MCDA methods were evaluated for the significance of the results one against the other. This quantitative analysis took the form of a sensitivity analysis following the method proposed by Insua and French (1991) and encountered the same results as similar studies that tried to compare TOPSIS and COPRAS (e.g. Podvezko 2011), that is, TOPSIS is more robust than COPRAS. The analysis was based on the sensitivity of the 2 methods to changes in data and in terms of variances of alternatives. Robustness measurement was done against changes in the data sets and resulting relative position of possible alternatives to the current best solution, significance of changes in positions and the relative distance of the positions. The comparative analysis showed that TOPSIS is more robust than COPRAS, which is more unstable to data variation. This confirms findings by Podvezko (2011). Due to the better robustness and stability showed by TOPSIS when compared to COPRAS, and since both showed alignment with public sector requirements, legal constraints and operational specifications, TOPSIS was finally determined as the preferable decision model for e‐Procurement DSS for the education sector of the Maldives.

8. Conclusion This paper presented a Three Tier Mixed‐Method (TTMM) approach to evaluate and support the selection of an MCDA method to serve Public Procurement. This approach was described and discussed in the light of a

7

Mohamed Adil, Miguel Baptista Nunes and Guo Chao Peng research project that aimed to select an MCDA method to be applied in the public sector e‐procurement in Maldivian public education sector. The evaluation model involves a qualitative comparative analysis of legal and operational requirements of the research context against the characteristics of MCDA methods in order to filter non applicable methods. Legal requirements are gathered from public sector laws and regulations from the research context. This first tier evaluation allowed the selection of 4 methods out of more than 80. The operational requirements were identified through a series of focus group discussion with public sector procurement bid evaluation officers from the public education sector of the Maldives. The comparison of these operational requirements with the previously obtained 4 methods resulted in just 2 candidates. The third tier quantitative analysis was based on real life sets of collected procurement data. This data was used to perform sensitivity and robustness evaluation and revealed TOPSIS as the ideal MCDA method for the Maldivian context. The TTMM approach seems to have yielded good results and research findings. It would now be interesting if it could be used in other different contexts and potentially in relation to other operational research and decision making models.

References Amponsah, C.T. (2011) "Application of multi‐criteria decision making process to determine critical success factors for procurement of capital projects under public‐private partnerships", International Journal of the Analytic Hierarchy Process,Vol 3, No. 2, pp 107‐129.

Ananda, J., and Herath, G. (2003) "Incorporating stakeholder values into regional forest planning: a value function approach", Ecological Economics, Vol 45, No. 1, pp 75‐90.

Baily, P, Farmer, D, Jessop, D, and Jones, D 1994, Purchasing Principles and Management, 7th edn, Pitman Publishing, London.

Barbour, R.S. and Kitzinger J. (eds.) (1999) Developing focus group research: Politics, theory and practice, SAGE, London.

Brown, R.B., Wright, R.D.J., Cloke, C.G., Morris, T.B., and Trumper, I.F.S. (1984) Government purchasing : a multi‐ department review of government contract and procurement procedures, HMSO, London.

Dhaulathuge Maaliyyathu Gaanoonu 2006 (Maldives).

Dhaulathuge Maaliyyathuge Gavaaidhu 2009 (Maldives).

England, W.B. (1967) The purchasing system, Irwin, Homewood, Ill. Falagario, M., Sciancalepore, F., Costantino, N., and Pietroforte, R. (2012) "Using a DEA‐cross efficiency approach in public procurement tenders", European Journal of Operational Research, Vol 218, No. 2, pp 523‐529.

Figueira, J., Greco, S., and Ehrgott, M. (eds.) (2005) Multiple criteria decision analysis : state of the art surveys, Springer, Boston.

Flick, U. (2009) An introduction to qualitative research, 4th edn, SAGE, London.

Guitouni, A., and Martel, J. (1998) "Tentative guidelines to help choosing an appropriate MCDA method", European Journal of Operational Research, Vol 109, No 2, pp 501‐521.

Ho, W., Xu, X., and Dey, P.K. (2010) "Multi‐criteria decision making approaches for supplier evaluation and selection : A literature review", European Journal of Operational Research, Vol 202, No. 1, pp 16–24.

Insua, D.R., and French, S. (1991) "A framework for sensitivity analysis in descrete multi‐objective decision‐making", European Journal of Operational Research, Vol 54, No. 2, pp 176–190.

Leenders, M.R., and Fearon, H.E. (1997) Purchasing and supply management, 7th edn, Irwin, Chicago.

Leenders, M.R., Johnson, P.F., Flynn, A.E., and Fearon, H.E. (2006) Purchasing and supply management : with 50 supply chain cases, 13th edn, McGraw‐Hill, New York.

Lysons, K., and Gillingham, M. (2003) Purchasing and Supply Chain Management, 6th edn, Financial Times Prentice Hall, Harlow.

Mateus, R., Ferreira, J.A., and Carreira, J. (2010) "Full disclosure of tender evaluation models: Background and application in Portuguese public procurement", Journal of Purchasing & Supply Management, Vol 16 No. 3, pp 206‐215.

Monczka, R.M., Handfield, R.B., Guinipero, L.C., Patterson, J.L., and Waters, D. (2010) Purchasing and Supply Chain Management, Centage, Hampshire.

Podvezko, V. (2011) "The comparative analysis of MCDA methods SAW and COPRAS", Inzinerine Ekonomika‐Engineering Economics, Vol 22, No. 2, pp 134‐146.

Simpson, L. (1996) "Do decision makers know what they prefer?: MAVT and ELECTRE II", The Journal of the Operational Research Society, Vol 47, No. 7, pp 919‐929.

Triantaphyllou, E. (2000) Multi‐criteria decision making methods : a comparative study, Kluwer Academic Publishers, Boston, Mass.

Weele, A.J.van. (2000) Purchasing and supply chain management : analysis, planning and practice, 2nd edn, Business Press, London.

Weele, A.J.van. (2010) Purchasing and supply chain management : analysis, strategy, planning and practice, 5th edn, Cengage Learning, Andover.

8

“Experts and Novices”: An Intervention to Support Literature Review and Analysis Deborah Anderson Kingston University, Kingston upon Thames, UK D.Anderson@kingston.ac.uk

Abstract: This paper reviews academic studies on the writing of a literature review, leading to the conclusion that whilst academics attach importance to the literature review as part of the research process, there is little practical guidance available for students on the required organisation, synthesis and critical evaluation skills. In an attempt to address this gap, staff at a UK university have developed a series of learning and teaching interventions to help students develop such skills. The paper reports on qualitative research amongst masters students, exploring their perceptions of one particular learning and teaching intervention, “Experts and Novices”. This intervention is based on participative approaches to learning where students are required to read and lead a critical discussion amongst peers with a view to identifying key themes and debates in the literature. Findings confirm that students do not always understand what a literature review is and in particular are uncertain of the status of their own opinion in a review. They appear to value the “Expert and Novices” approach, in particular the opportunity to discuss their ideas in a “low‐risk” environment, although the success of the intervention does rely on full participation of all students. Recommendations for educators include devoting more time and practical sessions to ensure that students become better researchers both by clearly understanding the literature review’s role in research and by affording students practical, participative‐based opportunities for “low‐risk” discussions of their reading prior to submitting a final piece of work.

Keywords: literature review, learning and teaching interventions, critical analysis

1. Introduction The role of the academic literature review in masters’ level research is well‐accepted amongst the academic community. Indeed, any guide to writing a dissertation will include a chapter on how to write a literature review (e.g., Biggam, 2011; Murray 2007; Swetnam and Swetnam, 2010 ) However, whilst academics and supervisors are used to critically reviewing and synthesising papers, the task is often quite alien to students.

This is particularly true when students move from undergraduate to postgraduate study.

At Kingston University, a successful MA Marketing course attracts students from all over the world, from a wide range of disciplines and from a variety of educational institutions. Faced with the challenge of ensuring that all students develop the critical analytical skills required for writing a strong literature review, staff have introduced a series of learning and teaching interventions. One such intervention is “Experts and Novices”, an initiative in which students are required to read and lead a critical discussion amongst peers with a view to identifying key themes and debates in the literature. This paper will report on qualitative research amongst the students which investigated how they perceived of literature reviews in general and more specifically of the “Experts and Novices” intervention.

2. The importance of the literature review Defining the scope of a study is a key element of developing a masters dissertation and for most researchers, existing literature will be crucial to help understand what has already been established in the discipline (Boote and Beile 2005). By drawing upon both conceptual and empirical studies, students are also able to establish a theoretical framework for a study. Initially, a review of existing literature can provide a mapping of the territory within which to locate the current study and its contribution to the knowledge base. A review of previous studies can facilitate the development of the research problem and provide guidance in research design. By highlighting techniques which yield different types of data (Merriam 1998), a literature review can expose students to the alternative ways of tackling their own research problem. Overall, there is agreement that a review of literature ensures that the researcher appreciates the debates in a field, helping to develop an argument and an explanation of how they are extending existing knowledge (Zorn and Campbell 2006).

Despite its importance, several studies note that the literature review is often the weakest element in a study; for example, Hart (1998) identifies that poor literature reviews are often no more than “thinly disguised annotated bibliographies” (p1). This is supported by Boote and Beile (2005, p 3) who describe a poor literature

9

Deborah Anderson review as “merely an exhaustive summary of prior research”. Green and Bowser (2006) also note that annotated bibliographies are often presented as critiques of research.

However, despite a broad consensus about the definition and purpose of a literature review, few research methods text books devote substantial sections to the literature review and often the literature review is given much less emphasis than other methodology‐related issues (Green and Bowser 2006). Indeed, despite the literature review being accepted as a part of developing into an academic, of becoming an “expert in the field” (Hart 1998, p1), there is surprisingly little in the way of practical guidance for students.

Nonetheless, some guidance does exist. Help with search and retrieval is available, often emerging from the field of information science (Gordon et al. 2002) and there is often collaboration between faculty and librarians to develop skills in this area (Green and Bowser 2006). Practical skills such as writing and referencing are also well supported, often in dedicated texts or leaflets and guides developed by universities. What appears to be missing are strategies for organising and synthesising materials to avoid accusations of “brief catalogues of previous research” or “article –by‐article reports” (Nairn et al. 2006, p259). Berthon et al. (2003) do offer some guidance by proposing that students use the notion of a “paradigm funnel” to categorise and organise articles and therefore go some way towards providing a practical tool which can develop understanding of some of the deeper requirements of a literature review. The use of exercises to help students understand and develop synthesised arguments has also been proposed (e.g., Zorn and Campbell 2006). Retaining a practical focus, text books and studies which develop and test the use of rubrics and marking criteria for literature reviews are also available (e.g., Hart 2003; Boote and Beile 2005; Green and Bowser 2006). However, whilst it is clear that a rubric can help students understand the criteria for assessing a review, it is unlikely that a rubric alone can tell a student how to synthesise and therefore lead to improved literature reviews (Boote and Beile 2005). The writing process itself is the subject of research by Caffarella and Barnett (2000) who focus on a specific teaching process designed to help students with scholarly writing (the Scholarly Writing Process) identifying that preparing and receiving critiques from scholars and peers was most influential in helping the scholarly writing process. Whilst helpful, this is a rare example of practical guidance for students.

Given the importance attached to the writing of an academic literature review, either alone or as part of a masters dissertation, it is disappointing that so few studies exist to explore students’ understanding of the literature review itself or the process of developing one. An exception is Bruce’s (1994) phenomenographic study of postgraduate students which identified six conceptions of a literature review, concluding that it is impossible to identify any common understanding. Whilst a basic conception of the literature review as a list did exist, Bruce was also able to discern higher levels of conception which included the literature review as research facilitator and ultimately as a report.

In summary, there is clear agreement in the literature about the role and purpose of a literature review in the research process. It is accepted that a review should be more than a simple annotated bibliography and that concepts and ideas should be synthesised to provide a clear understanding of what is already known about a particular topic. In this way, students can define the scope of their study and develop an appropriate research design. However, despite its important role in the research process, guidance for students is limited and appears to be strongest for search, retrieval, writing and referencing with less guidance on synthesis and critical evaluation.

3. The “experts and novices” intervention explained At Kingston University, in an attempt to help masters students develop the critical analytical and synthesis skills required for a strong literature review, several initiatives and interventions have been developed. For example, in addition to dedicated research methods classes, students are also required to produce several literature reviews as part of the coursework for other taught modules prior to starting work on their own dissertations. To help prepare for such coursework, staff have developed tools and interventions such as a method of keeping detailed notes on their reading by compiling a “literature review grid”. By keeping a record which maps emerging themes against authors, students are able to develop an overview of the body of literature rather than simply keep paper by paper notes. “Experts and Novices” is another example of a specific intervention used in one taught marketing communications module on the masters course.

10

Deborah Anderson The original justification for developing “Experts and Novices” emerged from the researcher’s observations, supported by anecdotal evidence from colleagues, that students have a very confused perception of what a literature review is. In brief, the approach involves nominating some students as “experts” and some as “novices” during selected sessions. “Experts” are those students who have completed in‐depth reading of five journal articles on their chosen topic, whilst “novices” have just read a chapter from the course text. The “experts” are then responsible for briefing the “novices” on the topic during the session. The underlying thinking is that by having to sort out emerging themes from their reading and discussing these with peers, students would gain more confidence in their ability to synthesise different authors’ work critically, in particular by having to decide which research/evidence is most convincing. All of this would be completed in a non‐threatening, low‐risk environment which, whilst not attracting a grade, is a required part of the literature review assessment.

The “Experts and Novices” initiative is based on an approach to learning which places emphasis on participation rather than the acquisition of knowledge. Hagar (2004) identified this shift as one from a “standard” paradigm of learning to an “emerging” paradigm. The “standard paradigm” views learners as “acquiring” knowledge by filling up a series of containers with universal context‐free knowledge. In the standard paradigm, there is a view of “learning as product” (Hager, 2004a p.5) where what is learned is relatively stable over time. In the “emerging” paradigm, the main change is not the altered “cleverness” of the learner, but the creation of a new set of relations in an environment.

4. Methodology The research aimed to explore students’ perceptions of the “Experts and Novices” intervention. As the research aimed to investigate the detailed perceptions and opinions of the students, rather than to prove or test a priori theories, an exploratory, qualitative approach was deemed most appropriate. Four focus groups were conducted, facilitated by a member of the teaching team, following a semi‐structured research guide.

All students who were enrolled on the full time MA Marketing and associated programmes at Kingston University in one academic year were eligible as the wide universe for the research. The sample, based on convenience, comprised 28 students from 19 countries, with a diverse range of prior undergraduate experience, subject areas and institutional cultures. Focus group discussions lasted between 45 and 75 minutes and all group discussions were recorded (with the participants’ full knowledge) and transcribed by a professional transcriber. Data analysis followed Glaser’s (1965) constant comparative method in which initial codes were developed, then clustered into categories and themes.

5. Findings Initial discussions focussed on perceptions of the literature review and findings confirmed that there was confusion about understanding exactly what was expected. This will not be a surprise to anyone teaching masters students and given the participants’ diverse backgrounds, both geographically and in terms of the types of undergraduate courses studied previously, this was probably to be expected. However, the strength of the feelings is worth exposing: I didn’t know what a literature review is. In the past I was used to writing reports, and so writing a literature review is completely foreign to me I had no idea how to go about it. Even looking at academic papers was not something I was accustomed to. So that was all brand new to me.

I had no idea what it was. Someone asked me to make an assignment but I had no idea how to complete it or how to start it.

Well I found it was a huge challenge and I did struggle a lot, first because I have never done it before, English is not my first language, on my Degree I did a project instead of a dissertation so I even didn’t know much what was to write a literature review...

I found it slightly overwhelming I was terrified to be quite honest because I had never written anything at a University level, because my Undergrad was practical, 100% Students seemed to be able to grasp that some kind of summary was required: I felt that we read, we review it then we summarise it.

I thought it was a kind of summary.

11

Deborah Anderson However, not all students realised at first that it was necessary to review more than one paper: My problem with the literature review was that I thought it was one article for example and tried to build on the article whereas I should do a mixed up, I should find some ideas and find the support of the articles and summarising. But I thought it was one article.

Even something as basic as understanding the type of literature to include in a literature review was not clear to all students at the start of the course. The status of books, trade journals and business reports was contested. Some international students reported a lack of access to comprehensive databases in their previous institutions, often finding it overwhelming initially when faced with the challenge of identifying which papers and articles to use in their review: For me it was really hard because of my undergrad actually...we did not have any access to any information. I did study international business but then we did not have anything like the websites we have. We did not have access to (databases), we didn’t have access to any of this so I had basically never read any of this kind of article...

Students expressed a preference for being told exactly which papers to read, in particular one student had done much better with this approach: And maybe the first assignment, because it was the first assignment was very clear and I knew exactly what I had to read and I knew who were the authors I had to use to talk about it. So for me it was really clear, whereas the other ones I didn’t know exactly who should I include, is this author really relevant or not? And I think it’s harder when they don’t tell you read this, what’s going to be relevant and use it.

I agree in the sense I think that in terms of improvement, it would be better if we would be given a guideline in terms of....maybe to manage your time or how to do it better in terms of your structure and how it should be written.

A common misconception was the notion that a student’s own viewpoint was not valid. Because the literature review clearly required the reading of several articles, some students could not see how their own ideas could be woven into a review which summarised others’ work. This is understandable; unless it is made clear that students are expected to draw their own conclusions based on their critical view of a range of research, it is easy to see how a student might initially believe that all that is required of them is an overview of who has found out what in a particular topic.

I also wrote reports and we would search about the thing itself and then give our own ideas on that. I had read a few journals, like 15 journals for one assignment. But the problem that I still feel it’s not my opinion at all.

As discussions moved to perceptions of the “Experts and Novices” sessions, comments largely focussed on the positive aspects of the intervention. Students appeared able to begin identifying the debates in their chosen topic area and by having to read five journal articles in advance of the submission of the assessed literature review, several students viewed the session as a beneficial “kick‐start” to the whole process of writing a literature review. The opportunity to explain their reading and answer questions in small groups was seen as particularly beneficial: I read the five journals, read the textbooks and you know brought out the most important parts of the journals and sort of I don’t know I feel like there was actually quite an interesting debate going on, people were asking me and I could actually answer them and things like that...

I think the thing is you can test yourself whether you have understood it when the novice asks very interesting and clever questions, you can then think oh well I haven’t really got behind that yet and I need to focus a little bit more and I had a very good intelligent novice who asked very good questions but because I prepared well I thought okay I can explain this to him and helped him…

For the quieter members of class who do not usually contribute to discussion willingly, there appeared to be particularly strong benefits: I think the system of experts analysis, they’re good for people who are shy. For example during the lecture they don’t understand something and they’re afraid to raise their hand and ask the

12

Deborah Anderson lecturer, and in this way it’s more relaxed situation, you can ask because you’re doing with your class mates.

One of the features of an “Experts and Novices” session is the opportunity to share references of useful journals and other sources. Whilst the writing of a literature review does ultimately need to be the sole individual work of the student, it is clearly beneficial to share sources and debate ideas as preparation: “The other thing I found very useful was actually sharing the best journals because when I get to do my literary review I did have a couple of journals that I really liked and then my talking to other people and they started sharing, oh this is the best one for me and so I got actually the best ones and that’s why I could argue about my topic so I think that was the best part”.

The opportunity to explain and debate different view‐points in a non‐threatening environment appeared to help the process of critical evaluation: I think it’s a good way to develop critical evaluation because you get other ideas…you think in a different way because you get a new idea and have to evaluate it and maybe you didn’t think about it before.

There was also some evidence that the approach did help towards a deeper understanding of a topic: I was thinking about that because for example before the session I read the chapter from the book, and I gathered that it was a bit hard to understand, but then in class when experts explained it I found it really easy to understand. And also before the first session I thought that it could be a bit boring, but then I found it really useful.

Despite some very positive perceived benefits of the “Experts and Novices” intervention, the experience was not seen as helpful by one student: the person who was the expert hadn’t done any reading, he didn’t really know anything about it and we kind of sat there twiddling our thumbs not really knowing what to talk about.

One problem with the intervention is that students are able to compare participants and their contributions.

The participant above had experienced one very well‐prepared “expert” and was able to contrast the excellent learning in this session with the unsatisfactory contribution of the unprepared student. Whilst the experience was very dependent upon the contribution of the “experts”, it is worth noting that when developing the “Experts and Novices” initiative, the author saw the greatest benefits being accrued by the “expert” in that they would have to identify themes and arguments in the literature and be able to explain why they were more convincing than other arguments. However, the focus group discussions revealed that students were expecting to gain as much from the sessions in which they were “novices”. Future developments of this initiative will need to take this into account when briefing the sessions, by clearly identifying the benefits of being both an “expert” and a “novice”.

In summary, discussions amongst this group of masters students revealed a widespread lack of understanding of an academic literature review. Some saw it as simply a “summary”, and most struggled with the status of their own opinion in the review. Past experience was varied (both in terms of the diverse international contexts experienced and in terms of the subject disciplines) and as a result, some students were used to starting with their own opinions, others felt their own views had no place in their work. Views of the “Experts and Novices” were in the main very positive, with particular benefits for preparing early for an assessment, sharing sources, understanding the key debates and critically analysing the literature. Confidence appeared to grow as students became familiar with their topic through explanation and debate. Shy students who would not normally willingly participate in class discussion were also able to contribute to the small group discussions.

6. Conclusions and recommendations We saw from the literature review for this research that academics broadly agree upon the role and purpose of a literature review. However, the focus group findings confirm what many educators observe, that students do not fully understand what the literature review is and more importantly, find it difficult to know how to approach the process. Despite prior university study, few have detailed experience of writing an academic literature review, and even those who have drawn upon academic work are not always comfortable with the combination of critical summaries and the status of their own opinions. Focus group findings also revealed very positive views of the “Experts and Novices” intervention, suggesting that using participative approaches

13

Deborah Anderson to learning can be helpful, even when the final piece of submitted work is an individual assignment. With these thoughts in mind, it would be useful to identify some recommendations for educators.

In order to improve understanding of the literature review, educators should spend time, either as part of research methods classes or integrated into other taught classes, discussing the definition and purpose of a literature review and its role as part of the research process. Clarifying that a review of the literature can help define the scope of a student’s own work and locate it in a wider body of research will help a student to define the parameters of their own research. It is particularly important that students understand that a literature review is more than a simple annotated bibliography or list, and that they should be identifying themes and debates to provide a synthesis of research.

Educators should clarify the status of the student’s own opinion, i.e., that the student is able to support an argument by drawing upon rigorous research and able to refute an argument by identifying evidence from their reading. Whilst unsubstantiated opinions are to be discouraged, students should be clear that a critical review of studies can contribute to their own stance on a topic.

In terms of planning interventions to support literature review writing, it is recommended that educators consider how students can be offered “low‐risk” opportunities to “play” with concepts and theories, discussing views and develop their own ideas. This appears to work particularly well when students are not under pressure to present to a large audience, but rather can discuss and debate in a “safe” environment. Sharing sources and journal articles should also be encouraged where students are working on similar subject areas to ensure comprehensive coverage of the topic.

In summary, it is recommended that educators devote more time and practical sessions to ensure that students become better researchers both by clearly understanding the literature review’s role in research and by affording students practical, participative‐based opportunities for “low‐risk” discussions.

References Berthon, P., Nairn, A. And Money, A. (2003) Through the paradigm funnel: a conceptual tool for literature analysis.

Marketing Education Review. Vol. 13, No. 2, pp. 55‐66.

Biggam, J. (2011) Succeeding with your Master’s dissertation. Open University Press.

Boote, D.N. and Beile, P. (2005) Scholars before researchers: on the centrality of the dissertation literature review in research preparation. Educational Researcher. Vol. 34, No. 6, pp. 3‐15.

Bruce, C.S. (1994) Research students’ early experiences of the dissertation literature review. Studies in Higher Education.

Vol. 19, No. 2, pp. 217‐230.

Caffarella, R.S. and Barnett, B.G. (2000) teaching doctoral students to become scholarly writers: the importance of giving and receiving critiques. Studies in Higher Education, 25 (1) 39‐52.

Glaser, B.G. (1965) The constant comparative method of qualitative analysis. Social Problems, 12 436‐445.

Gordon, W.F. and Stewart, G.D.J. (2002) Report on a Private Project: using a virtual classroom to develop literature review and writing skills. South African Journal of Library and Information Science, 68 (1).

Green, R. And Bowser, M. (2006) Observations from the field: sharing a literature review rubric. Journal of Library Administration, 45 (1), 185‐202.

Hart, C. (2003) Doing a literature review. Sage Publications, London.

Merriam, S.B. (1998) Qualitative research and case study applications in education. Wiley, San Francisco.

Murray, R. (2007) How to write a thesis. Open University Press,, Maidenhead.

Nairn, A., Berthon, P. and Money, A. (2006) Learning from giants: exploring, classifying and analysing existing knowledge on market research. International Journal of Market Research, 49 (2) 257‐274.

Swetnam, D. and Swetnam, R. (2010) Writing your dissertation. How to books Ltd. Zorn, T. And Campbell, N. (2006) Improving the writing of literature reviews through a literature integration exercise.

Business Communication Quarterly, 69 (2) 172‐183.

14

Evaluating a Newly Implemented Dissertation Supervision System – Views From Both Sides of the Equation Marie Ashwin and Alan Hirst Ecole de Management de Normandie, France m.ashwin@em‐normandie.fr a.hirst@em‐normandie.fr

Abstract: In the context of continuous develop to address the requirements of International accreditation a dissertation/supervision system was designed and implemented in the 2011/2012 academic year. This paper presents the findings of preliminary qualitative analysis of data collected from both sides of the supervision process, from supervisors and students. Many universities and Business Schools around the word have been developing their dissertation systems over a number of years, designing and refining them in response to the feedback on experiences in previous years. This research was undertaken to evaluate this first year of transition as the School moved from a situation where most students prepared a Work Placement report as the final evaluation of their five‐year master’s programme, rather than an academic dissertation. In the previous year, under the direction of the Director of Research, the content of the research methodology module had been harmonized across all the programmes, both in English and French, with a small group of lecturers. Within the national educational system students have been habituated to an educational culture where directive learning is predominant. Lecturers are expected, and expect to provide all the resources that the students need to be able to complete the tasks set them. Within the School there is an activity each year when groups of students work on field study projects linked to their studies, acting as junior consultants to address a concern within a company. These activities provide them with a degree of autonomy to work on the project presented to them by the company, but they do not, however, require the level of individual academic application required by a dissertation. In particular, the study asked staff and students to identify any issues that arose during the dissertation period. Prior to this only a few students each year had undertaken research and submitted dissertations, so not all staff had previously been involved in supervision activities. This study set out to identify the experiences, both positive and negative, of staff and students throughout the research process. This paper presents the results of this survey and identifies the impact of the newly implemented centralized and harmonized dissertation and supervision framework and the key issues which staff and students have highlighted. The results of the project reveal that on the whole staff and students found the newly‐implemented structure and system beneficial. Both groups of respondents voice concerns about similar things, in particular; the timing of the research methods classes, clearly defined and understood guidelines for the project, and the quality of contact and communication between the tutors and the students. Another issue clearly identified by students is that of the workload/calendar balance between the dissertation and other programme requirements.

Keywords: research methodology, dissertation process, supervision, independent learning, communication, research skills

1. Background The reform of the French higher education sector is being driven by global forces (Cant‐Sperber 2011; Husson 2011) and the system of international ranking has contributed significantly to the speeding up of the national university reform. The resulting French Excellence Initiative launched in 2010 has as its aim the consolidation of the existing 85 HE institutions into 19 federated universities. The LABEX initiative provided some 10 Bn dollars of funding for development through direct competition between large institutions to stimulate change.

The Higher Education sector in France consists of three main elements, 85 universities, 205 Grande Ecoles (Conférence des grandes écoles, 2012) and a small group of large state‐funded research organisations such as INSERM (Institut national de la set de la recherche médicale), CNRS (Centre national de la recherche scientifique) or CEA (Commissariat à l'énergie atomique et aux énergies alternatives). The Grande Ecoles themselves focus on a wide variety of disciplines, including Business and Management, Engineering or Science and Technology.

Admission is achieved through a rigorous selection process of written and oral examinations, or through the presentation of an application dossier. Closer to the labour market than universities, French graduates from the schools have little difficulty in finding employment upon graduation although there is no evidence of this advantageous position for their foreign graduates (Barsoux and Lawrence 2009).

These schools are not part of the university system where education is funded by the state, and tuition fees are low. For the 2012/2013 academic year the registration fee for Masters‐level programmes was set at €250

15

Marie Ashwin and Alan Hirst by the Ministry for Higher Education and Research (http://www.service‐public.fr). By comparison the annual fees for Grande Ecoles at the beginning of the current academic year ranged from €8,000 to €15,000 (Conférence des grandes écoles 2012).

Although they are smaller in size, with around 5% of the student population, these schools receive 30% of the national higher education budget. For some schools strategies for development have included the development of closer links with the universities in their region. In general it is difficult for the Grande Ecoles to merge due to their semi‐public legal status and their traditional links with local Chambers of Commerce (Cant‐Sperber 2011; Husson 2011). The School in question is in fact the result of a successful merger of two smaller local schools into a larger regional institution of higher education.

2. Drivers As a player in the global higher education market the School has grown through both the increase in home students and the recruitment of fee‐paying overseas students. A global presence requires international accreditation. The School obtained EPAS accreditation in September 2011 and is in the final stages of the process for AACSB accreditation, a journey it started in 2009.

One part of this application has focused upon the development and implementation of a dissertation process for final year students on its five‐year Master programmes. Up until the 2010 academic year most students presented an Internship Report as the final element of their assessment, with very few completing an academic research project. Many of these students had however been involved in undertaking research during their undergraduate year abroad and producing a dissertation. Although a small number of final year students on specialist Double Diploma programmes with foreign partners were required to complete a dissertation in the 2011/2012 academic year it became compulsory for all students to do so. This limited exposure of staff to supervision, and of students to the rigour of completing a piece of academic research at master’s level, has both positive and negative aspects.

In 1997 Bukle‐Young emphasized the value of the project as a means of developing knowledge and understanding of key concepts outside the classroom environment. By independently designing and implementing a research project linked to their final year specialism students can improve research skills for life‐long application. This view fits well with the HEC (Entrepreneurs) pedagogical approach which has been adopted by the School.

The philosophy encourages student interaction with business through field projects/junior consultancy projects and internships in each year of their studies. For all of these activities the students are used to having a mentor inside the host company who guides and encourages them to develop the skills required whilst working in a semi‐autonomous or autonomous manner. This is in direct opposition to the academic culture within the School which is one of a directive learning environment where students attend classes for anything up to 24 hours a week. The result is that for anything seen as internal to the School students expect everything they require to be provided by the tutor. They therefore appear to have poor information collection, critical thinking and synthesis skills, all of which are essential elements of the dissertation process. The role of the supervisor is to provide guidance and support, not so different from that of the business mentors in other activities, to help them develop these skills.

3. Review of developments in 2011/2012 Over the past twelve months a formalized, centralized dissertation process has been developed and implemented in conjunction with the Dean, Director of Programmes, Director of Research and the research methods tutors. This final group worked together with the Director of Research to harmonise the content and evaluation of the methodology classes to ensure a level of consistency across all 17 programmes regardless of the specialism. The assessment of the research methods module, unlike most other institutions, is not the preparation of a research proposal. A dissertation coordinator was appointed to work with the directors and develop a system to meet the requirements of the staff, the students and, most importantly the AACSB requirements. With no pre‐existing system in place the emphasis was upon developing guidelines and implementing a system to manage and coordinate the dissertation process from the allocation of the supervisors through to the collection and review of final grades.

16

Marie Ashwin and Alan Hirst The issues arising included the design of an efficient and effective supervisor allocation system, the production of a single standardised final document, and the agreement and application of one calendar for all programmes.

One year into the reform a number of evaluative actions have taking place to collect feedback from the 54 supervisors and over 400 students involved in the process. This paper considers the qualitative data obtained on their experiences, reviews their comments and suggests how the system might be improved.

4. Literature 4.1 The research process The mapping of a research process to identify and define its various stages has been addressed in many journal articles. Authors such as Slade (1994) and Webster (1999) consider the style and format of the final document, whilst others identify the required content, beginning with the title of the project then progressing through the various chapters to the final Appendices.

In 2004 Mustafa identified a lack of journal publications on the process of the student research project. The many text books on Research Methods fill this gap in the published literature as they develop the concept of the stages of the research process (Bell 2005; Bryman 2012; Saunders et al 2012; Sharp et al 2002). These enable students to develop a sense of the journey through the research process from the conception of the initial idea to the final submission. Working on a research project can help students develop and understanding of the concept of scientific enquiry (Ryder, 2004) whilst placing it firmly, in this instance, in the context of the business environment.

The engagement and motivation of students, as well as their disaffection are all identified as direct outcomes of the massification of Higher Education (Booth et al 2010). An example given is the fact that the context in which Research Methods are delivered appear to increase feelings of irrelevance, leading to the student’s disengagement with the dissertation process.

4.2 Student‐supervisor relationship Anderson et al (2006) report that existing literature on undergraduate research tends to focus upon the outcome, the final assessment, rather than the process which includes the research methods, input and supervision. (Todd et al 2004) They identify the relationship between the supervisor and the student as having content and contextual elements which translates as task‐orientated or process‐orientated interactions. Figure 1 represents the supervisors’ framing perspectives identified during the research process.

Figure 1: Supervisor’s framing perspectives (Anderson et al 2006 p.153)

17

Marie Ashwin and Alan Hirst The study of linked the value of a research project to career prospects as a level of understanding and competence in research was found to be essential to ‘function as a competent professional in a contemporary context’. (Anderson et al 2006 p.155)

4.3 Student‐centred learning In the EUA publication Trends 2010: A decade of change in European Higher Education two key issues were highlighted, the complex nature of student‐centred learning and the problem of how to measure it. One constraint identified was that of the legal and regulatory issues around teaching workloads. To develop student‐centred activities requires a ‘more creative approach … and therefore even more hours spent on developing new ways of teaching’ (Sursock and Smidt 2010, p49). The transformation of the nature of students’ work and the need for academic staff to work together in a coordinated pedagogical team were identified as cultural challenges for some countries.

Students can find independent learning activities uncomfortable and even ‘unsettling’ (Rich 2010) and as a result demand more support and guidance during the research process.

The constructivist approach to learning (Goodyear 2001) encourages students to actively participate in the structuring of the learning experience. They use previous knowledge to develop new ideas, and test what has already been learnt in the new situation. For this to be successful a support framework must be established around them, which in the case of a research project is provided firstly by the methodology classes and then by supervision.

4.4 Communication This basic model of communication developed by Shannon and Weaver (1949) (Figure 1) identifies the issue of noise impacting upon the message.

Figure 2 Shannon and Weaver model of communication (1949) Narula (2006) discussed the concept of noise in interpersonal communication relating to the three components of semantic noise, channel noise and feedback. Channel noise is identified as being the distortion of, or distraction away from the message at the time of transmission. Schramm (1954) identified feedback as the key to assessing the effect of the message and the level of understanding of the receiver as it reveals how the message is being interpreted. The interpretation of a message can be influenced by physical factors such as the means of communication (telephone, email, face‐to‐face). Misunderstanding of the message is labeled semantic noise when due to a lack of comprehension of the frame of reference of the sender.

5. Methodology To ensure that both sides of the supervision model were investigated adequately two questionnaires were developed, based primarily upon the feedback collected from staff and students. Before the questionnaires were designed informal and formal feedback was collected from students and supervisors to ascertain the issues they considered important. From these discussions it became apparent that both sides of the supervision experience were describing the same key elements and voicing similar concerns.

18

Marie Ashwin and Alan Hirst

5.1 Pilot study This feedback was collected in two ways. Several individuals contacted the researcher directly without knowing that a research project was in its formative stages to provide feedback upon their experiences.

Additional data was collected as a result of the researcher contacting a number of colleagues and students who had been working and studying on the different campuses to ensure that all four campuses were represented during the development of the project.

5.2 Instrument development Each instrument was developed using five sections:

Part One – to investigate the supervisor‐student relationship. This included the identification of the average number of staff hours spent on supervision, the frequency of contact and the methods of communication most frequently used.

Part Two – to identify the preparedness of the students to undertake the project in terms of their knowledge and levels of research skills.

Part Three – to determine the perceptions of student skills and ability to manage the various stages of the research process. These were defined as: development of the research aim and objectives, research problem, critical review of the literature, design of the methodology and choice of methods, data collection and analysis, discussion of findings, writing up and oral defense.

Part Four – to measure the perceived level of usefulness of the documentation provided including awareness and use of the guidelines.

Part Five – to measure the overall satisfaction/dissatisfaction in the implementation of the supervision process and the final outcome.

5.2.1 Supervisor questionnaire The questionnaire was developed around the following six area :

Location – of student and of staff member

Communication ‐ methods of communication used and time allocated

Student capabilities ‐

Documentation – student use, usefulness,

Students awareness and understanding of the research process

Satisfaction with the supervision experience

There was also a free response section in which supervisors could give personal views, highlight problems encountered with more details or make suggestions for improvement 5.2.2 Student questionnaire The questionnaire was developed around the following areas:

Demographics – gender, age, location, level of study

Overall satisfaction with the dissertation process

Documentation – usefulness and use of guidelines and handbook

Process – self‐directed learning, research methods, time management, official calendar

Supervision – communication, interactions with supervisor

Overall experience ‐ as with the supervisors, students were provided with a free response section to provide any feedback they wished.

19

Marie Ashwin and Alan Hirst

5.3 Administration of the instruments A narrow window of opportunity was identified from the middle of September 2012 to the end of October 2012 when it was felt students and supervisors would still be connected with the dissertation process and most likely to respond to the questionnaires.

5.3.1 Supervisor questionnaire An email was sent to all supervisors informing them of the project with a request that they participate in completion of the questionnaire. The anonymity of participants was assured by ensuring that no individual distinguishing elements appeared in the items, other than campus of attachment and gender.

On the researchers’ home campus a printed copy of the questionnaire was delivered to each supervisor. When possible this was done in person, providing the opportunity to explain in more detail the aims and objectives of the research. It was felt essential that it was seen as a developmental activity to help improve the process for future years. When personal contact was not possible the questionnaire was put in the supervisors’ pigeon‐ hole with a short explanatory note. The instructions informed them that there was an envelope in the reception area for the return of the questionnaire to maintain anonymity.

On another campus the questionnaires, explanatory note and a list of supervisors were delivered to the receptionist with the request that a copy be placed in the pigeon‐hole of each. Again they were informed of the envelope waiting in the reception for the return of the questionnaires.

The third campus being smaller and lacking a regular courier exchange, it was decided to send the questionnaire electronically with the request that it be printed off and returned to the researcher without any indication of whom the sender might be.

Reminders were sent by emails after ten days and again after three weeks to ensure that all supervisors had the opportunity to participate in the research project. Thanks were given to those who had already completed and returned the questionnaires and others encouraged to do so. At the cut off period all staff received an email thanking those who had participated and assuring them that general feedback on the findings would be disseminated.

5.3.2 Student questionnaire The questionnaire was administered online using a SurveyMonkey link. An email was sent to every final year student on the School system explaining the reasons for the survey and asking them to click on the link to access the questionnaire. Two weeks later a message thanking those who had already completed the survey and prompting others to participate was sent out, again with the link. This procedure was repeated a final time after another two weeks.

A message was sent to all students thanking those who had participated and closing down the project six weeks after initial contact.

6. Findings and discussion The response rate from supervisors was 46 % from across all the campuses. One issue arising from the distribution by the receptionist on one campus was that names were written on the documents to ensure they were distributed correctly. Several supervisors refused to complete the document as it was no longer confidential, some informing us directly whilst others simply did not respond. Others went to the lengths of masking their name, then photocopying the document again to complete it ‘anonymously’ as they wanted their views to be known.

On the home campus many supervisors signed their names at the end of the document whilst others returned the questionnaire directly to the researcher, thereby removing their anonymity altogether. Several added documents and free‐form pages with reflections on the experience and ideas for future development.

The student response rate to the online questionnaire was 19%. In addition to the free‐form responses around 20 students sent additional emails direct to the researcher with detailed feedback on their personal

20

Marie Ashwin and Alan Hirst experiences giving permission for it to be used in addition to the questionnaires if helpful. Most of these revolved around the issues of the relationship with the supervisor, particularly regarding communication throughout the life of the project.

For the preparation of this paper a preliminary analysis of the qualitative data from the free‐form responses has been undertaken in order to extract the key themes.

The predominant theme from both staff and students is the supervisory relationship, and in particular the issue of communication.

6.1 Task orientated relationship The task‐orientated relationship identified by Anderson et al (2006) fits the psychological and educational culture within the School where there are high levels of contact between students and tutors, up to 24 hours of classes per week, and the teaching style is didactic. There is however a disconnect between this environment and the need for independent‐learning skills, and intermittent interactions throughout the dissertation process.

Class contact is rarely individual or in small groups, nor do students experience the rigorous academic demands of research to the same extent elsewhere during their studies. They are used to the tutor supplying everything they might need to complete the work required, and much of the work they prepare outside the classroom is done in groups. There is a lack of experience in managing their learning independently in a self‐ directed learning environment.

6.2 Research methods classes and research skills Research methods classes early in the academic year lost their value as many students did not start working on their projects until well into the second semester. The majority of supervisors received no communication from the Research Methods tutors to inform them of the content of the classes and any resources provided to the students.

The belief of students is that previous classes in earlier years have provided all the tools they need to undertake a piece of research. As a result they do not necessarily connect the research methods module, and the necessity for academic rigour, to the designing of a research project.

Supervisors did not know how the concept of research had been presented to students, nor were the students able to recall much about the classes. There was a disconnection between the preparatory stage and the project itself.

6.3 Supervision process For many universities and business schools the gradual transition from face‐to‐face to remote supervision has been an organic development over a number of years as the characteristics of the student body, course content and technologies evolve.

The School has moved directly to a distance‐learning model without many staff having prior experience of supervision. There has been no gradual development of the framework of competences and understanding of the supervision process inherent in face‐to‐face experiences.

60 % of students said it would be useful to have the possibility to meet with someone who had been through the process and to see some examples of good dissertations. There had been little feedback from alumni about their experiences of the research process as so few had completed a dissertation.

6.4 Independent learning and calendar The considerable differences in nature, content and structure of the various programmes, linked to the timings of the obligatory field projects and internships, resulted in many students working on their dissertations once

21

Marie Ashwin and Alan Hirst they had ‘left the building.’ The result was that they did not have easy access to the resources or their supervisors during the critical stages of the research project.

Independent, self‐directed learning and the research project vehicle are, for the most part, encountered for the first time in this final year of a five‐year programme. It appears that finishing the taught elements and leaving the structured environment of the School amplified students’ concerns about working independently and the research calendar as they felt they should be committing their time to the internship, and not completing a piece of academic work.

Staff confirmed this, with several referring to the students panicking when they realized how much work was required to complete the research and write the dissertation. It was unanimously agreed that most students had not started the dissertation early enough, despite the fact that all the research methods tutors had emphasized the need to do so during their classes at the beginning of the academic year.

6.5 Communication Poor communication was not simply an issue after the students had started their internships. Almost one third of supervisors reported having no initial contact from students whilst they were still on‐site. Over 25% of supervisors reported not having had any contact at all with some students and heavy demand from others.

Some students complain of a lack of response from their supervisor despite repeated messages being sent across a range of media.

Once students had left to work throughout France, Europe or further afield there was a need to coordinate across different regions, countries and time‐zones. Over 80 % of staff and students identified email as the main means of communication throughout the lifetime of the research projects, with 10% using video‐conferencing to have a face‐to‐face meeting at a distance.

The impact of a lack of interaction was supported with evidence from staff and students of dissatisfaction with the relationship, from the experience of a poor relationship to no relationship at all. The inextricable link between the supervisor and student deemed essential during the dissertation process by Armstrong (2004) was not in evidence in these cases.

About one quarter of the students who responded praised the level of access and interaction they had with their supervisors, and the support they received throughout the length of the dissertation process.

7. Conclusions It has been a year of transformation for all concerned and these preliminary findings confirm the importance of the first stages of implementation of the centralized and harmonised dissertation system. A short presentation of findings was made to supervisors during the review of the first semester resulting in further discussion on the way forward to improve the process further.

Strategies needed to improve the relationship between students and supervisors will have to be agreed and implemented. These include addressing the issues of time management, supervisor skills and the levels of autonomy of the students.

It is hoped that the costs, financial and others, are outweighed by the ultimate success of such initiatives as students move forward using their independent learning strategies developed during the dissertation process.

There is a need to emphasis to the students the value of research skills, not only for the dissertation project but also for their future career prospects (Anderson et al 2006).

Acknowledgements The authors would like to thank all those who participated in the survey, many of whom made personal contact to address issues in more depth than the survey permitted.

References Anderson, C., Day, K., and McLoughlin, P. (2006) Mastering the dissertation : lecturers’ representations of the purposes and processes of Master’s level dissertation supervision Studies in Higher Education Vol 31:2, pp149‐168

22

Marie Ashwin and Alan Hirst Armstrong, S. (2004) The impact of supervisors’ Cognitive Styles on the Quality of Research Supervision in Management Education British Journal of Educational Psychology Vol 74, pp 599‐616 Bell, J (2005) Doing Your Research Project 4e, McGraw‐Hill International, Bryman, A (2012) Social Research Methods 3e, Oxford University Press, Oxford Goodyear, P. (2001) Effective Networked Learning in Higher Education Vol. 3 Final Report JCALT, CSALT, University of Lancashire, Lancaster Mustafa, Y (2004) A Systematic Framework for Conducting Research Projects in Undergraduate Courses Journal of Information Systems Education Vol 15:1, 35‐40 Narula, U (2006) Communication Models Atlantic, New Delhi Rich, M (2010) Balancing guidance and independence in student learning of research skills Conference proceedings of the th 9 European Conference on Research Methodology for Business and Management Studies IE Business School, Madrid, Spain 24‐25 June 2010, p445‐450, Ryder, J (2004) What can students learn from final year research project? Bioscience Education E‐journal, Vol 4 Saunders, M; Lewis, P and Thornhill, A (2012) Research Methods for Business Students 6e, Prentice Hall, Harlow Sharp, JA; Peters, J and Howard, K (2002) The Management of a Student Research Project 3e The Open University, Milton Keynes Shannon CE and Weaver W (1949) The Mathematical Theory of Communication. University of Illinois Press, Urbana, IL Sursock, A and Smidt, H (2010) Trends 2010: A decade of change in European Higher Education, EUA publications, Brussels

23

Barriers to Using Qualitative Methods in Business and Management Research in Ukraine Yevhen Baranchenko1 and Andrey Yukhanaev2 1 Department of Management, Sumy State University, Sumy, Ukraine 2 Business and Law Faculty, Northumbria University, Newcastle upon Tyne, UK e.baranchenko@qs.sumdu.edu.ua

Abstract: The Bologna process and modernisation reforms of the higher education system on the post‐Soviet territories have evoked many controversies in the academic communities across the newly independent states. Influenced by the USSR legacy, traditions of scientific achievements and former transformational inventions the present day academics become resistant to changes when it comes to implementation of the contemporary Western European standards. This is of a high relevance for the Ukrainian higher education system as the government call for international harmonisation and synchronisation of the educational practices has been made. Universities across the country have been charged with the task of incorporating certain Humboldtian traditions in the process of teaching, curricula design and research activities.

Thus, assurance of student learning, research‐driven teaching and knowledge creation are of a paramount importance in the modern educational establishments and becoming prerequisites for sustainable socio‐economic development.

However, strong prevalence of the incumbent approaches inherited from the past tends to inhibit achievement of the ambitious goals of aligning the Ukrainian principles of higher education with the best international norms. Adamant academic perceptions regarding the process of conducting research and generation of scientific knowledge obstruct the scientific advancement and integration endeavours initiated by the national government. This paper deals with the distinctive features of the present academic culture and research traditions within the Ukrainian higher education system.

Our study investigates the issues and challenges associated with conducting qualitative research in buiness and management area. Through the application of qualitative approach using 27 semi‐structured interviews with Ukrainian academics, findings from the participant observation and the content analysis of the 6740 refereed journal articles published in 2009‐2011, we attempt to reveal the reasons underpinning the main obstacles of conducting qualitative research in the Ukrainian context. Our findings offer a significant contribution to further understanding of the difficulties involved in implementation of the universal academic standards and highlight discrepancies between the West European and the former Soviet higher education system.

Keywords: qualitative research, interview, higher education system, Ukraine

1. Introduction Traditionally, explanation of the reasons for the divide between proponents of qualitative and quantitative research together with the subsequent debate about philosophical paradigms was explicitly rendered by the various experts in the literature (Burrell and Morgan 1979; Bryman 1984; Guba and Lincoln 1988; Creswell 2003). Despite the fact that business and management studies fall into the social science domain and tend to be highly diversified in nature, the dominant position of positivistic stance in such enquiries, associated with quantitative research, is still evident (Johnson and Duberley, 2000). At the same time, followers of the qualitative research that utilises interpretivist (constructivist) paradigms insist on the potency and further proliferation of value‐bound approach without distinguishing the knower from the known (Ritchie and Lewis, 2003). As a result of this prolonged philosophical confrontation, the scientific community had been divided and the contention reached its apogee leading to “paradigm wars” in the 1980s (Tashakkori and Teddlie 1998; Bryman 2008). Although, in more recent times the paradigmatic tensions among academics have been on decrease, yet there is still distinctive preponderance of a particular approach in certain areas of scientific research.

To a large extent, being influenced by the observation of this phenomenon in the business and management research in Ukraine, the current study explores the challenges of conducting qualitative research and its subsequent recognition by the Ukrainian academic community. Driven by the need to harmonise its higher education standards with the best international practices, the country is currently undergoing the Bologna process of integration into the Western European education system (Kovtun and Stick 2009; Shaw, Chapman and Rumyantseva 2011; Luchinskaya and Ovchynnikova 2011). As a result of the USSR legacy, the Higher Education system in Ukraine is very hierarchical and subject to enormous regulation, intervention and rigorous bureaucratic control by the Ministry of Education and Science, Youth and Sports of Ukraine (MESYSU). Hence, it is the prerogative of the Ministry to select and accredit national journals where all postgraduate and doctoral students are required to publish prior to the completion of their research projects and receiving

24

Yevhen Baranchenko and Andrey Yukhanaev qualifications. Such requirement places tremendous pressure on the aspiring candidates to publish solely in these specifically selected research outlets at the very early stages of their academic and scientific development. Given the tradition of positivistic methods and unwillingness to recognise a wider variety of methodological approaches in the business and management fields, early career researchers are presented with a real challenge of employing alternative philosophical paradigms and “unconventional” research designs.

Therefore, the main aim of the current study is to explore the key obstacles preventing the use of qualitative research methods in the modern Ukraine.

Driven by the need to gain further understanding of the above‐mentioned phenomenon, our study has a qualitative origin and utilises a range of methods (Johnson, Onwuegbuzie and Turner 2007; Tashakkori and Teddlie 2010) including semi‐strcutred interviews with the academics of various levels, participant observation (Bryman and Bell 2007) and content analysis of journal articles containing signs of qualitative research published by the state approved journals.

2. The distinction between quantitative and qualitative research Although qualitative research is not as common as quantitative approach, various researchers in the business and management domain highlighted a steady growth in utilisation of methodologies based on interpretivist positions (Gummesson 1999; Myers 2008; Waring and Wainwright 2008). In the history of philosophical evolution people have always attempted to quantify the world, which echoed in the proliferation of “hard” sciences such as mathematics, physics, chemistry as well as their subsequent influence on the methods employed in other scientific streams. The opposite end of the spectrum is represented by the so‐called “soft” sciences with social and humanitarian predisposition at the forefront of the academic enquiry. As such psychology is traditionally considered as the pioneering science that gave rise to the development and popularity of qualitative research. At the beginning of the twentieth century in German and American sociology most studies were case study based and inductive approach began to evolve. In the 1960s, sociologists started actively criticising the conventional quantitative approaches, advocating non statistical methods that do not seek to correlate or extrapolate, but strive to capture and explain individual scenarios from the perspective of a researcher. As a result, this led to the wider acceptance of qualitative research in social and psychological sciences. Furthermore, the researchers appealed for a need to better understand the study object, which followed by the introduction of the principles of openness and reflexivity. Thus, the possibility of using interviews and participant observations as data collection tools resulted in extensive employment of these qualitative methods in various studies. In the 1980s, the dispute regarding generalisability and validity of qualitative research, which perhaps was unavoidable, took place. Gladly, it was successfully resolved by the introduction of the alternative ways and specific criteria for quality assessment.

At present, the existence of quantitative and qualitative approaches can be regarded as two independently evolving accounts of empirical research within business and management studies that fall into a broader realm of social sciences. However, for many researchers the conception of qualitative approach still lies in the process of opposing the former to the quantitative counterpart. This can be achieved by examining the essential elements of the enquiry and answering the question ‘What is being?’, which was the central idea for many ancient philosophers starting with Aristotle and his metaphysics (Politis 2004). Extrapolating this to the field of research methodology, the term ‘paradigm’ has been gradually accepted and became a common notion in the scientific lexicon (Firestone 1990). Here the meaning of paradigm is traditionally defined as a ‘set of basic beliefs that deals with ultimates or first principles’ (Guba 1990, p.17) and impacts the formation of key elements. The first element inherently influencing the entire set is ontology, which is understood as the nature and form of reality. The second element – epistemology – grasps the nature of the relationship between the knower and the known. The third element consists of methodology and methods guiding the researcher on the cognitive journey in general, and the selection of most appropriate tools (instruments) for data collection and analysis in particular.

For quantitative studies positivism is the overarching perspective, whereas for qualitative research it is phenomenological stance. Originating from the work of Auguste Comte in the 1830s positivism focused on observable facts with assumption that the social world can be studied in the same way as natural world.

Positivist as well as postpositivist stances are embedded in naïve or critical realist positions claiming that there is a reality, which can be studied, captured and understood fully or approximately (postpositivism). Even though postpositivists acknowledge the variety of methods for capturing as much of the reality as possible, yet

25

Yevhen Baranchenko and Andrey Yukhanaev according to their beliefs utilisation of this approach has an indisputable advantage in the form of prediction or control of the natural phenomena. Application of generalizability criteria can be of use in situations, which are similar, hence posing another constraint for the researchers in the social science. The contextual information, therefore, can only be provided and then fully utilised by qualitative researchers, who emphasise the importance of human factor in studying behavioural dimensions, particularly meanings and purposes.

Ontologically, the position of positivism is realism, while epistemology is objectivist without interaction and involvement in value‐bounded system. Hence, methodology is of experimental nature, when questions, assumptions and/or hypotheses are stated prior to the actual enquiry and then tested afterwards. In turn, postpositivism has ontological position known as critical realism. The followers of this stance choose critical approach recognising the human frailty as ultimate truth can never be discovered. Epistemology is based on rejection of the assumption of absolute objectivity as the enquirer in many instances is unable to remain outside of the enquiry itself. Methodologically postpositivism relies on multiple methods (sources of data) allowing to minimise the distortion and misconception.

On the contrary, qualitative research is rooted in constructivist paradigm with the aim of developing a theory by observing and interpreting the reality. Here, constructivist paradigm shares relativist ontological position, grounded on the assumption of many interpretations of the reality. The researcher is in a constant quest for the most informed and sophisticated reality construction with epistemology being subjectively driven enquiry.

Although methodologically the researcher is able to identify many constructions the task is to deliver as much of the consistency and robustness as possible. Initial suggestions that include the notion of interpretations of the world and understanding arising from contemplating, and not only experience, belong to Immanuel Kant and are associated with qualitative research traditions. Creswell (2007) defined key reasons for undertaking qualitative research among which are the need to gain detailed understanding of the phenomenon; willingness to hear the participants’ opinion and perceive the context of the study; extend theories, which are insufficient to capture the complexity of the phenomenon.

In general, the studies of a qualitative nature are more widely accepted in the UK, the Scandinavian counties and Australia, with fewer supporters in Canada and the USA (Maykut and Morehouse 1994).

Mixed method or multi method (Bryman 1998) or even multi‐strategy technique (Bryman 2004) is increasingly used by the researchers and gradually becoming recognised by many experts in the area as a third major paradigm or research approach (Tashakkori and Teddlie 1998, Johnson, Onwuegbuzie and Turner 2007).

Moreover, if the decision is made to follow a mixed method approach, further distinction between quantitative and qualitative stances may not be required (Johnson and Onwuegbuzie 2004; Morais 2012).

However, as the authors believe and it has to be emphasised, that this assumption can only be accepted when the qualitative research coexists with quantitative.

3. The system of higher education in Ukraine More than two decades have passed since the collapse of the Soviet Union and subsequent emergence of fifteen sovereign and independent states. Being one of them, Ukraine had to implement a number of radical and rapid reforms in all sectors of the economy and national governance. The system of higher education was among the main institutions where drastic transformation and modernisation were urgently needed. This was also perceived by the government as a key priority, since the education system as a whole forms foundation of the social development and economic progress. Hence, the Ukrainian government made a decision to join the Bologna Process in 2005. As a result, an agreement was reached with the EU countries to conduct joint actions and design a set of measures facilitating gradual restructuring of the Ukrainian system of higher education and its alignment with the Western European standards (Shaw, Chapman and Rumyantseva 2011). The changes were required to be made in the areas of national curriculum design, adoption of three‐cycle degree structure and creation of uniform standards for teaching quality assurance. However, the inception of these structural adjustments was followed by a number of serious tensions between various parts and bodies of the country education system. As was noted by the several researchers (Luchinskaya and Ovchynnikova 2011; Shaw, Chapman and Rumyantseva 2012) the Ukrainian system of higher education, in contrast to the Western European principle, is very hierarchical and highly bureaucratic. Thus, the national Ministry of Education exerts enormous control over the institutions as well as stipulates the processes of service provision within the entire system leading to significant interference in the governance of both public and private universities. Among

26

Yevhen Baranchenko and Andrey Yukhanaev other key elements of the Bologna system was recognition of research activities conducted within a particular university that occupies important place in the national league table. The higher the research profile of academic staff of the university, the higher is the position of the educational institution, which in turn, should lead to international recognition and facilitate integration into European educational system. The processes such as redesigning of the curriculum, adaptation of the European Credit Transfer and Accumulation System and implementation of cross‐national quality assurance mechanisms were required to be conducted by Ukrainian universities simultaneously with the actions aimed at achieving international recognition of the research undertaking by academic staff. The authors believe that the latter action in particular substantially depends on the attitude of the researchers towards the actual research enquires, study design and the employed methodologies that should correspond with the internationally accepted practices.

While there is a demand for drastic changes within the system, the pressure is still put on academics by the MESYSU to publish only in the approved national journals. The list of these journals goes through the rigorous approval procedure conducted by the Certified Staff Evaluation Department of the Ministry after which the approved journals are given the status of high ranking “authoritative” outlets. Moreover, according to the national requirements, all postgraduate research students must have a certain number of articles published in these specially accredited journals prior to the completion of the research project and its defence. For the doctoral students the number of publications in these journals should be not less than five according to the Ministry of Education decree (MESYSU, 2012). Naturally, this requirement places an enormous pressure on aspiring candidates, at the very beginning of their research career, to produce work worthy to be accepted for publication in these nationally selected outlets.

Given the tradition of positivism and quantitative research methodology together with unwillingness to acknowledge other methodological approaches in the fields of business and management research, young academics are faced with an insurmountable difficulty of using alternative philosophical paradigms and research designs. This feature of the modern Ukrainian system of higher education and scientific research traditions has its origin from the Soviet Union times when all universities (i.e. educational institutes at that time) were to fulfil the main functions of being ‘sites of instruction’ (Shaw, Chapman and Rumyantseva 2011, p.14). Thus, only purposely established research institutions were allowed to focus on real knowledge creation, scientific experiments and innovations. As a result, today there is still a widespread reluctance among national regulative bodies as well as the wider academic community in the country to recognise and embed a differentiated spectrum of methodological approaches in social, business and management studies. This is in turn, becoming counterproductive to the plans of reforming the national education institutions, incorporating internationally recognised standards and increasing the quality of scientific outputs among both the existent and the prospective Ukrainian academics.

4. Methodological approach The role of theory for this research is consistent with Bryman (2004). Aiming to integrate multiple methods within the the enquiry (Creswell 2003; Tashakkori and Teddlie 2010) the authors use inductive approach combining the range of data collection methods and attempting to disclose the phenomenon and then to explore it. Our approach is not driven by the process of mixing philosophical stances or theoretical lenses, but is designed to employ the methods sequentially (Creswell and Clark 2007). Furthermore, the authors underwent through a multiphase research process. The first phase consisted of participant observation and quantitative analysis of the published academic literature, whereas second major phase is of a qualitative nature and strives for enrichment of the research results.

Thus, participant observation is the starting point for this enquiry with its research traditions in educational fields, anthropology and social sciences in general (Maykut and Morehouse 1994; DeWalt and DeWalt 2002).

Being employed as a lecturer by the higher education institutions in the Ukraine, one of the present authors of this paper was able to immerse into daily academic routine and professional activities in order to collect data in naturalistic settings. The results from the participant observation and immersion in the relevant occupational fields induced the further need for more extensive investigation.

Therefore, the researchers undertook further exploration of the popularity and the use of qualitative methods in the business and management studies published by the leading Ukrainian academic journals. Here, the authors acknowledge that national academic journals might not be the only outlets, alongside other sources

27

Yevhen Baranchenko and Andrey Yukhanaev including books and conference papers, where research studies of a qualitative nature can be found.

Nonetheless, it is believed that refereed articles published by the leading academic institutions in the country present the most reliable and comprehensive picture of the true engagement of the Ukrainian scientific community in qualitative research. For the purpose of this study the researchers selected twenty five state accredited journals that are defined by the MESYSU as ‘professional’ and ‘expert’ in the fields of economics, business and management. Our analysis was restricted to the three‐year period of publications in 2009‐2011 bringing the total number of the analysed academic articles to six thousand seven hundred and forty items.

In order to analyse the selected sample of the published journal articles the authors employed the method of content analysis, which is a widely recognised technique for the analysis of all forms or recorded communications in social sciences (Neuendorf 2002). Aiming to quantify the phenomena based on Krippendorff (2004) the researchers created a set of key words relevant to the area of qualitative research and by means of NVivo software analysed the title, abstract and the main body of the text of each researched journal publication. In particular, our study focused on the search of the following words and word combinations within the contents of the papers: ‘qualitative research’, ‘constructionism’, ‘interpretivism’, ‘inductive’, ‘case study’, ‘interview’, ‘focus group’, ‘observation’. The search using the same algorithm was conducted tree times using three languages ‐ Ukrainian, Russian and English.

According to the developed research design the authors were able to proceed to the major qualitative part of the study as soon as the content analysis had been completed. The sequential qualitative element forms the main part of this research and aims to reveal the actual barriers, which are faced by the researchers in choosing and then undertaking the research pass of a qualitative nature. The second part of the study includes semi‐structured interviews with members of academic staff of various positions. The semi‐structured interviews were selected because of the certain degree of freedom allowing the interviewer to omit certain predefined questions as well as the interviewee to elaborate and comprehend a particular subtheme during the interviewing process (King 2004; Saunders, Lewis and Thornhill 2009). Thus, the interviews were conducted with the active researchers from three Ukrainian universities. Although there is no clearly defined category as ‘classic’ or ‘old’ university in the national context, the authors purposefully selected higher education institutions established more than thirty years ago and have a status of the state universities. Within each university the main criterion for selecting academic members of staff for participating in the interviews was their involvement in teaching students of the faculty of management and business. Three sets of interviews were conducted within each of the following groups of the current research participants: 1) PhD students; 2) lecturers and 3) professors. The total number of interviews came to twenty seven sessions. All the interviews were audio‐recorded to allow the researchers to transcribe and analyse them after the data collection stage. Finally, the authors employed thematic analysis (Boyatzis, 1998; King, 2010) technique enabling to structure the findings on the basis of the research focus coding iteratively using top‐down and bottom‐up approaches.

5. Discussion As one of the authors is employed by a higher education institution in Ukraine, it was possible to observe a considerable lack of interest to using qualitative methodologies in management studies. In addition, the description of the qualitative methods in academic books on research methodology published in Ukraine is very limited and concise. Furthermore, interviews as a data collection method are not distinguished from questionnaire, whereby the former is also defined as ‘conversation’.

The content analysis of six thousand seven hundred and forty journal articles brought even more astonishing results. We identified only nine academic journal publications among the studied items that acknowledged and/or used qualitative research methods in their design. These results were further discussed in details during the following interviews with twenty seven academics, who shared somewhat contradicting views on the matter, stating emphatically that these methods of scientific enquiry were never used by any of them in their research career.

The outcomes of the interviews were structured in themes that revealed the barriers to using qualitative research methods in business and management research in the Ukrainian context.

28

Yevhen Baranchenko and Andrey Yukhanaev

5.1 Traditions This was pointed out by all of the participants with the only difference in naming the tradition as ‘custom’, ‘thinking’, ‘mentality’, ‘stereotypes’. The Ukrainian science at the time of Soviet Union was to a high degree influenced by natural sciences (physics and chemistry). The priority for the state was to develop airspace industry and engineering. The philosophy of several generations was based on appreciation of ‘hard’ sciences.

Until early 1990s the lecturers at the department of economics had engineering education, when economics and then management began to evolve, adopting the research principles from fundamental sciences.

“There is a demand for numbers and formulas. If an article does not have these elements, it is a journalistic review.”

“When we mark the undergraduate dissertations and do not see any calculations, the essay is perceived as weak.”

“Even our quantitative research is not perfect, but we have the traditions.”

Three research participants shared the experience of conducting a survey using questionnaires, which were critically evaluated by the colleagues, who admitted intense subjective factor even in those research enquiries.

In addition, many participants acknowledged the specificity of Ukrainian educational process, which was described as ‘overregulated’ in terms of teaching as well application of the methodological approaches.

5.2 Philosophy The second theme relates to the philosophical underpinning. The postgraduate research students admitted this challenge. As the qualitative research requires deeper understanding and subsequent underpinning of the studies, the participants were not able to link the qualitative research to philosophical stances. Moreover, the interviews revealed that the philosophical underpinning often substituted by theoretical.

“The problem that we have got is that we do not try to understand the nature of the phenomenon, but try to get the result”.

“Only pluralism of philosophical stances and further applicability to the research design may lead to acknowledgement of alternative research methods.”

Widely accepted concept of the research paradigm was not familiar to the majority of the participants.

“The elements of the research paradigm in our system are not explained and thus it is difficult to establish the links between them”.

5.3 Unfamiliarity with the methods The methods are not familiar to the participants. Only the professors clearly stated that they are aware of these methods, but never used them themselves. A number of the research participants shared the view that the questions about the alternative methodologies and methods are rarely highlighted in research dissertations. As it was emphasised the senior researchers pose the right to describe in details the methodology.

“I have never met the researcher who used these methods for data collection. If I had a chance to work with somebody (supervisor or mentor) who knows these methods well, I would certainly like to use them in my research project.”

Also the respondents highlighted the lack of the procedures of the qualitative data analysis. Thus, the researchers are not aware about the techniques that exist and are approved.

“It is much easier with quantitative data. You know the algorithm, I know it – our colleagues in other universities now it.”

5.4 Misconception of application, reliability The participants believed that qualitative methods can be used only as the supportive to quantitative at the initial or last stages of the research.

“Interview can be used only in order to prepare for further fundamental study”

29

Yevhen Baranchenko and Andrey Yukhanaev Despite the significant amount of literature on quality in qualitative research, in which the issues around reliability and validity are comprehensively discussed, all research participants mentioned their concern with respect to reliability. However the authors of this study would like to emphasize that this issue is rooted in data sources, which are defined as reliable by the researchers in Ukraine.

“As a researcher you are entitled to use wide range of data and not to be limited by the beliefs and values”.

“How can you believe your participants”?

A number of participants applied the concept of reliability to the qualitative research methods and questioned the interviewers about the total number of the participants in this study, which to their views was not sufficient to be representative.

6. Conclusion Although the researchers in previous studies have identified the challenges for conducting qualitative enquiries the current research has focused on the issues concerning the utilisation of the qualitative research methods in business and management fields in Ukraine. Our exploratory study has identified four key themes that emerged as potential barriers: traditions; philosophy; unfamiliarity with the methods; misconception of application, reliability.

Despite the willingness of the Ukrainian government to expedite the integration of the national higher education and science through participating in the Bologna process, the failure to recognise and accept the qualitative research methods in business and management studies is regarded as one of the key obstacles to the fully‐fledged integration. Moreover, the numerous initiatives of the European Union in support of the integration of the Ukrainian higher education system are focused on student mobility for instance Erasmus Mundus programmes (EACEA, 2010) or modernisation of the entire system or teaching improvements in the Ukrainian universities through participation in TEMPUS programmes (EACEA, 2011) but leave undeservedly little attention to the integration of the Ukrainian research community. Finally, the study reveals that the research participants purposely distance themselves from being close to the study objects. The lack of qualitative studies in business and management area may also be interpreted as barriers to knowledge transfer between universities and businesses. The participating academics gave high priority to being objective in the research activity, but underestimated the value of dealing with informants.

References Burrell, G. and Morgan, G. (1979) Sociological Paradigms and Organisational Analysis, Heinemann Educational Books, London.

Boyatzis, R. (1998) Transforming Qualitative Information: Thematic Analysis and Code Development, Sage Publications, Thousand Oaks, CA. Bryman, A. (1984) “The Debate about Quantitative and Qualitative Research: A Question of Method or Epistemology?”, British Journal of Sociology, Vol. 35, pp 78‐92.

Bryman, A. (1998) “Quantitative and qualitative research strategies in knowing the social world”, in T. May and M. Williams (Eds.), Knowing the Social World, (pp 138–57) Open University Press, Buckingham. nd Bryman, A. (2004) Social Research Methods, 2 ed., Oxford University Press, Oxford.

Bryman, A. and Bell, E. (2007) Business Research Methods, 2nd ed., Oxford University Press, Oxford.

Bryman, A. (2008) “The end of the paradigm wars?”. In P. Alasuutari, L. Bickman and J. Brannen (Eds.), The Sage Handbook of Social Research Methods, (pp 13‐25) Sage, London.

Creswell, J. (2003) Research Design: Qualitative, Quantitative, and Mixed Approaches, Sage Publications, Thousand Oaks, CA. Creswell, J. (2007) Qualitative Inquiry and Research Design: Choosing Among Five Different Approaches, 2nd ed., Sage Publications, Thousand Oaks, CA. Creswell, J. and Clark, V. (2007) Designing and Conducting Mixed Methods Research, Sage Publications, Thousand Oaks, CA. DeWalt, K. and DeWalt, B. (2002) Participant Observation: A Guide for Fieldworkers, AltaMira Press, Walnut Creek, California, USA. EACEA (2010) Erasmus Mundus programme of the European Comission, Education, Audiovisual and Cultural Executive Agency, available online http://eacea.ec.europa.eu/erasmus_mundus/programme/about_erasmus_mundus_en.php EACEA (2011) TEMPUS IV (2007 – 2013): Overview of the Programme, Education, Audiovisual and Cultural Executive Agency, available online http://eacea.ec.europa.eu/tempus/programme/about_tempus_en.php Firestone, W (1990) “Accomodation: towards a paradigm‐praxis dialectic”. In E. Guba (Eds.), The Paradigm Dialog, (pp 105‐ 124) Sage, Newbury Park, CA.

30

Yevhen Baranchenko and Andrey Yukhanaev Guba, E. and Lincoln, Y. (1988) “Naturalistic and rationalistic enquiry”. In J. Keeves (Eds.), Educational Research, Methodology and Measurement: An International Handbook, (pp 81–85) Pergamon Press, London.

Guba, E. (1990) The Paradigm Dialog, Sage Publications, Newbury Park, CA. nd Gummesson, E. (1999) Qualitative Methods in Management Research, 2 ed., Sage Publications, Thousand Oaks, CA. Johnson, P. and Duberley, J. (2000) Understanding Management Research: An Introduction to Epistemology, Sage Publications, London.

Johnson, R. and Onwuegbuzie, A. (2004) “Mixed Methods Research: A Research Paradigm Whose Time has Come”, Educational Researcher, Vol. 33, No.7, pp 14‐26.

Johnson, R., Onwuegbuzie, A. and Turner, L. (2007) “Toward a Definition of Mixed Methods Research”, Journal of Mixed Methods Research, Vol. 1, No 2, pp 112‐133.

King, N. (2004) “Using interviews in qualitative research”. In C. Cassell and G. Symon (Eds.), Essential Guide to Qualitative Methods in Organizational Research, (pp 11‐22) Sage Publications, London.

King, N. and Horrocks, C. (2010) Interviews in Qualitative Research, Sage Publications, London.

Kovtun, O. and Stick, S. (2009) “Ukraine and the Bologna Process: A Case Study of the Impact of the Bologna Process on Ukrainian State Institutions”, Higher Education in Europe, Vol. 34, No. 1, pp 91–103. nd Krippendorff, K. (2004) Content Analysis: An Introduction to its Methodology, 2 ed., Sage Publications, Thousand Oaks, California.

Luchinskaya, D. and Ovchynnikova, O. (2011) “The Bologna Process Policy Implementation in Russia and Ukraine: Similarities and Differences”, European Educational Research Journal, Vol. 10, No. 1, pp 21‐33.

Ministry of Education and Science, Youth and Sports of Ukraine (2012) “Decree on Publication of the Research Results, available online http://zakon4.rada.gov.ua/laws/show/z1851‐12#n26 Morais, H. (2012) “Towards Bridging Quantitative‐Qualitative Divide in Business and Management Research”, Paper th presented at 11 European Conference on Research Methodology for Business and Management Studies, University of Bolton, UK, 28‐29 June.

Myers, P. (2008) Qualitative Research in Business and Management, Sage Publications, London.

Maykut, P. and Morehouse, R. (1994) Beginning Qualitative Research: A Philosophic and Practical Guide, Falmer Press, London.

Neuendorf, K. (2002) The Content Analysis Guidebook, Sage Publications, Thousand Oaks, CA. Politis, V. (2004) Aristotle and the Metaphysics, Routledge, London.

Ritchie, J. and Lewis, J. (2003) Qualitative Research Practice: A Guide for Social Science Students and Researchers, Sage Publications, London. th Saunders M., Lewis, A. and Thornhill, A (2009) Research Methods for Business Students, 5 ed., Prentice Hall, London.

Shaw, M., Chapman, D. and Rumyantseva, N. (2011) “Organizational Culture in the Adoption of the Bologna Process: a Study of Academic Staff at a Ukrainian University”, Studies in Higher Education. DOI: 10.1080/03075079.2011.614336.

Shaw, M., Chapman, D. and Rumyantseva, N. (2012) “The impact of the Bologna Process on Academic Staff in Ukraine”, Higher Education Management and Policy, Vol. 23, No. 3, pp 71‐91.

Tashakkori, A. and Teddlie, C. (1998) Mixed methodology: Combining the Qualitative and Quantitative Approaches, Sage Pubications, Thousand Oaks, CA. nd Tashakkori, A. and Teddlie, C. (2010) Sage Handbook of Mixed Methods in Social and Behavioral Research, 2 ed., Sage Publications, Thousand Oaks, CA. Waring, T. and Wainwright, D. (2008) “Issues and Challenges in the Use of Template Analysis:Two Comparative Case Studies from the Field”, The Electronic Journal of Business Research Methods, Vol. 6, No. 1, pp 85‐94.

31

In a Word: Applying Cognitive Interviewing Methods in Legal Market Research David Bednall1, Marie Ashwin2 and Alan Hirst2 1 School of Management & Marketing, Deakin University, Melbourne, Australia 2 Ecole de Management de Normandie, Caen, France dbednall@deakin.edu.au, M.Ashwin@em‐normandie.fr A.Hirst@em‐normandie.fr

Abstract: Registered trademarks can be worth hundreds of millions to their owners. Many elements of a brand can be trademarked including words such as Microsoft, symbols such as the Cadbury “glass and a half” mark and colours, such as a particular orange for Veuve Clicquot. The registration of a mark (word, symbol or colour) prevents competitors in that category of goods from using it – a considerable commercial advantage to the owner. Applications to register trademarks are therefore often fiercely contested in the courts. To support registration companies will typically adduce survey research. A key component of this research is to test whether the mark has acquired secondary meaning, that is, whether consumers or buyers of the product have learned that the mark stands for a single brand. In the US, where many of these surveys have taken place, legal precedent has been based on whether the mark is strongly associated with a specific company, product or brand and not strongly associated with any rival. It has been argued the test of association can mislead. First the word “association” is symmetric – Cadbury is associated with purple and purple with Cadbury. Yet the appropriate legal test is whether the mark (colour in this case) takes the person to the brand, a non‐symmetric relationship. Second although one mark may be more strongly associated with a brand than others, this may simply be because the consumer cannot recall the other brands. A more appropriate word is "identification", a non‐symmetric relationship that takes the consumer from the mark to the brand exclusively. But this raises the research question, “Do consumers understand the word ‘identify’ in the way intended and does this word have a different meaning to ‘association’?” Although the meaning of a single word is often not critical in most survey designs, in this case it is. In order to test the meanings of the two words, cognitive interviewing techniques (Willis 2005) were employed. This involved administering a survey in the usual manner, then conducting an in‐depth interview with the interviewee immediately afterwards. In this interviewing, the meaning of words can be explored and the conscious thinking processes in arriving at the answer can be revealed. A study was made of six males and nine females using a questionnaire that tested whether the colour purple (as opposed to a control colour lime) identified any particular “brands, products or companies.” The word ‘identify’ mostly worked in the way intended; it took the consumer to the source, i.e. Cadbury in the case of purple, but to no particular source in the case of lime. The word was interpreted correctly. People could distinguish ‘identify’ from ‘associate’ though in a survey, not all might do so. The study demonstrated the worth of cognitive interviewing in a legal survey context.

Keywords: trademark, secondary, meaning, identification, cognitive interviewing, survey, method

1. Introduction A trademark “… is a word, phrase, symbol or design, or a combination thereof, that identifies and distinguishes the source of the goods of one party from those of others.” (United States Patents and Trademark Office 2011). Many elements of a brand can be trademarked, including words such as Microsoft, symbols such as the Cadbury “glass and a half” mark, sounds, smells and colours (Hudock 1996), such as a particular orange for Veuve Clicquot (Dickerson 2006) or a shade of purple for Cadbury (Abbott 2012). Given that registered trademarks identify the source of a product, they have a meaning similar to a brand, originally a mark placed on goods to identify their owner uniquely. Brand equity theory (Keller 2008) talks about the assets of a brand which, although intangible, have considerable commercial worth up to many $US billions (Interbrand 2012).

The registration of a trademark prevents competitors in that category of goods or services from using it – a considerable commercial advantage to the owner. Competing producers who use similar words, symbols or designs may feel that registration will affect their commercial success. Applications to register trademarks are therefore often fiercely contested in the courts. To support registration, companies will typically adduce survey research.

This paper looks at survey evidence designed to assist courts and tribunals in deciding whether a trademark should be registered. It first looks at the legal concept of secondary meaning, that is, whether the mark has acquired distinctiveness in the minds of consumers. It then examines traditional survey questions used to test

32

David Bednall, Marie Ashwin and Alan Hirst for secondary meaning and argues they are deficient. The paper argues for a different approach, then goes on to describe research which tests whether the proposed approach has validity.

2. Survey evidence used to test secondary meaning There are a number of alternative legal grounds for registration. By far the most common approach is to show that the mark has acquired secondary meaning, that consumers or buyers of the trademarked product have learned that the mark stands for a single brand (Swann 2008). There a several variants on what secondary meaning is. One somewhat liberal definition says the “mark or dress identifies one product,” while a more restrictive definition says the “mark or dress identifies the product of one, perhaps anonymous company” and a conservative one that the “mark or dress identifies one company.” (Evans & Gunn 1989, p. 26). Whichever definition is used, the survey evidence serves to test whether buyers or consumers distinguish the brand on the basis of the trademarked element. In the US, where many of these surveys have taken place, legal precedent has been based on whether the mark is strongly associated with a specific company, product or brand and not strongly associated with any rival. Jones (1989, p. 484) suggested one formulation. He proposed that the indicium be "… printed on a display board ... removing all other indicia of source from the packaging" and then asking a question such as, "Do you associate [brand name product] with the [generic product name] of one, or more than one, company?" Or, "Does [brand name product] connote one or more than one source of [generic product name]?" Just how these questions would incorporate colour is unclear. One possible formulation is, "Do you associate this colour with the confectionery of one, or more than one, company?" Or, "Does this colour connote one or more than one source of confectionery?" Jones noted that a follow‐up question "which company?” should be used.

It has been have argued (Bednall et al. 2012) that such a test of association can be misleading. First the word “association” is symmetric – Cadbury is associated with purple and purple with Cadbury. Yet the appropriate legal test is whether the mark (colour in this case) takes the person to the brand, a non‐symmetric relationship.

Second, although one mark may be more strongly associated with a brand than others, this may simply be because the consumer cannot recall the other brands. A more appropriate word is "identify", a non‐symmetric relationship that takes the consumer from the mark to the brand exclusively. Thus purple identifies Cadbury, but Cadbury does not identify purple. But this raises the research question, “Do consumers understand the word identify in the way intended and does this word have a different meaning to associate?” Although the meaning of a single word is often not critical in most survey designs, in this case it is.

3. Problems with survey questions – use of cognitive interviewing While the law attempts to deal with words in a precise ways, designers of questionnaires have long been aware of the difficulties posed by trying to have both precise and simple questions. It was quickly discovered that seemingly unambiguous words such as “and” or “country” could pose problems in interpretation (Payne 1951). While a pilot test is seen as a means to check the validity of questions, it may be more concerned with how a survey is administered and with issues to do with survey length. Information may result on questions with obvious difficulties, but if no problems are apparent, the questions may be accepted. With the work of Belson (1981) and others, the testing of survey questions became more rigorous. Belson invented a double‐ back technique that involved administering a survey then doubling back to explore the responses of interviewees so that their understanding of questions could be tested. This is a very time consuming and expensive technique, but gives the researcher a set of investigative tools which elevate the probability that a questionnaire will produce valid responses. In a legal domain, where the meaning of words is so critical, such testing techniques are crucial.

This paper uses a development of this testing method, based on the work of Willis (2005) and known as cognitive interviewing, in order to test for a question that asks about the identification of a brand based on colour. Fundamentally the technique involves administering a questionnaire, while collecting additional verbal information about the responses obtained (Beatty & Willis 2007). One approach is to ask the interviewee to talk out loud while in the process of answering each survey question, the ‘think aloud procedure” (p. 289). The other approach is to have the interviewee answer the survey questions administered with an interviewer subsequently probing for more information about what each question meant and the basis on which the answer was given, i.e. “follow‐up probes” (p. 290). The first approach introduces an artificial element, the

33

David Bednall, Marie Ashwin and Alan Hirst simultaneous production and reflection on an answer to a survey question. The second approach is now more common (Beatty & Wills 2007). Crucially the questionnaire is administrated in its usual way, be it self‐ completion or interviewer‐administered. Then probing questions are asked, such as “What did the word ‘identify’ mean to you?” or “Can you tell me what you were thinking when you said …?” This should allow a realistic assessment of the task faced by interviewees in the actual survey, at least to the extent that they can comment on and describe what meaning they gave to the words in the questions and the basis for their responses. One potential weakness is the cognitive memory load placed on interviewees if they have to remember how they handled questions asked much earlier in a long questionnaire. Thus the technique best suits testing short questionnaires. Longer questionnaires may be broken into sections or each individual, answered question could be followed by probes. Both approaches may again introduce artificiality as the interviewees know they will be asked questions which may affect their normal patterns of response.

Cognitive interviewing does not test construct validity statistically, but helps produce better questions with face validity and with a higher probability of construct validity. While it can be used for individuals who may have limited cognitive ability (young children, senile adults or the intellectually disabled), it can also be applied to general samples as would be common in most trademark secondary meaning surveys. In this interviewing approach, the meaning of words can be explored and the conscious thinking processes in arriving at the answer can be revealed. Where questions involve non‐cognitive or unconscious processing, it will be harder for interviewees to arrive at a valid account of their responses. Care also needs to be taken to prevent people being forced to justify their responses, in case the cognitive interviewing itself produces invalid responses.

4. Method A short survey questionnaire was designed for testing purposes. Table 1 shows the relevant question sequence. In this particular survey, where the test is for determining whether colour acts as a badge of identity (a trademark) for a brand, it is necessary to conduct face‐to‐face interviews. This is because the rendering of colour must be true to life and approximate the circumstances in which it is shown to interviewees, such as in a shop or supermarket. The use of an internet survey is not advised because most computer monitors are not colour calibrated ‐ hence the researcher cannot be certain that the colour is rendered accurately. This is crucial when the application for registration involves a particular colour shade or several shades as in the case of Cadbury.

In order to answer the research question, cognitive interviewing techniques were employed. Willis (2005, pp.

6‐7) has described various features of this approach. It involves an experienced interviewer (the lead author in this case), use of verbal report procedures, an emphasis on both overt and covert problems with questions, modest sample sizes, the use of a range of probing techniques, iterative testing and a flexible approach.

A study was made of six males and nine females, selected to be either 18‐39 or 40 and over, to give a good spread of interviewees. All people were residents of Melbourne, Australia. Cadbury Dairy Milk is an established brand in this market and used shades of dark purple for decades. As chocolate is eaten by most adults (Morris 2011), no other special selection criteria were required. The study used a questionnaire that tested whether the colour purple (as opposed to a control colour lime) identified any particular “brands, products or companies.” The question sequence is shown in Table 1. Interviews were recorded then transcribed by a professional service agency.

Table 1: Survey question sequence Q1.

Does this confectionery package identify any particular brands, products or companies to you, or not?

Yes

1 ASK Q2

No

2 GO TO DEMOGRAPHICS

Don’t know 3 GO TO DEMOGRAPHICS

Q2.

IF YES TO Q1, ASK, Which brands, products or companies? RECORD VERBATIM. DO NOT PROMPT. DO NOT PROBE.

Q3.

FOR EACH BRAND, PRODUCT OR COMPANY, ASK, Why do you say this confectionery package identifies (NAME OF BRAND, PRODUCT OR COMPANY)? RECORD VERBATIM. DO NOT PROBE.

34

David Bednall, Marie Ashwin and Alan Hirst Figure 1 shows the mocked up blocks of chocolate shown to participants. The blocks were actually a real chocolate pack (a local supermarket home brand) in a size somewhat different to that in current use by Cadbury. This was to avoid the possibility that the block size was cueing a Cadbury response. The purple colour was close to the various shades of purple used by Cadbury.

Figure 1: Blocks of chocolate

5. Results Table 2 illustrates responses to the questions as they were asked.

Table 2: Responses to survey questions Question Q1 Identify?

Q2 Which brands, products, companies ?

Q3 Why do you say that?

Purple pack responses Yes, 13 out of 13 Yeah being Cadbury.

Cadbury. Purple.

Yeah, Cadbury. Definitely Cadbury. Yes. Yeah, I probably think it’s Cadbury.

The purple is the first thing, well because thinking it's the lolly department, that it's confectionery in some way. The size of it looks roughly like a block of chocolate. As I said it's a different colour but it is the telltale purple colour but as I said, with the sideways, particularly if you turned it the other way maybe I mightn't have thought of the other brand but there is another one but I'm not sure whether I'm making it up or it's either a smaller named one or it's the nut free, dairy free one and I will think of the name eventually. Oh, Sweet William, it makes me think of sideways but I'm not even sure that there's a purple but that's what's hit my memory banks.

It’s the colour I’ve seen their products in, in supermarkets, milk bars, etcetera, for many years. Again, in this light it might be a slightly distorted purple.

Because of the colour and I guess the shape. The colour first I would say.

Just purely based on the colour, just that royal purple colour.

Lime pack response No, 2/2 Not applicable

Not applicable

The results show the purple pack led to Cadbury, while the lime pack did not. That shows the lime pack was acting effectively as a control colour. Of course, these are just a few interviews, but they illustrate the type of differences found in a survey of this kind (Hoek & Gendall 2010). In the case of the purple pack, the colour and to some extent the shape were given as the reasons for the response. However, the pack shape itself was not

35

David Bednall, Marie Ashwin and Alan Hirst enough to name Cadbury. The colour was key, based on consumers having a long history of experience seeing purple Cadbury blocks.

The spontaneous responses showed no indication that interviewees were having a problem with the questions. However, it is not safe to assume that interviewees will report problems with survey questions (Foddy 1993). Table 3 shows the responses of one person to the cognitive interviewing approach to the word “identify”. This person had no apparent difficulties in answering the survey question. Although this response indicated that the response was on the basis of the colour taking the person to the brand, Cadbury, it shows that it is difficult for people to put a concept like “identify” into other words. We return to this theme below.

Table 3: Cognitive interviewing outcome Interviewer When I used that word identify, what does that mean?

But if … the purple identifies Cadbury, can you just explain to me a little more?

Yes, so the characteristics that make you think that way, can you just expand on that a bit?

Interviewee I guess it’s asking me to like the reasons why I think what are the key kind of points.

Like relation maybe like how does it relate to Cadbury, yeah. I think relation would be a word that describes identifying, yeah I just guess what characteristics that would make you think that way.

One again being colour, I don’t know it’s hard, it’s kind of like jammed in your head that purple’s Cadbury. I mean I guess it’s something you don’t think about very often and to put it in to a sentence is quite difficult.

Other responses to questions about what the word identify means included: Identify? Identify, it’s Cadbury’s logo, Cadbury’s logo, the only thing missing is probably the writing and the two cups filling each other, that’s about it. …

Well like put a name to it yeah. …

I think it's just an immediate gut feeling, thinking ‐ I didn't think about it at the start, but thinking about it, it's just because it's like it looks like chocolate, it feels like chocolate, and it's ‐ because it's solid and it's purple, and I can't think of any other ‐ what are they called ‐ confectionery manufacturers that have that colour, or use that colour. It's probably ‐ I don't even know if it's an actual packaging colour or whatever … ...

Yeah, you would go for it, because it’s something that’s etched in your mind. …

Identify, it needs to have a name on it, a brand or a description to separate it from something else next to it sort of or does it stand out. If I looked from a distance and if I couldn't read the writing, does that package represent something to me. …

It was an instant thing when I saw it. …

To me identify would mean the fact that if the room was darkened, suddenly they showed me that, I’d identify it, I’d be able to pick the Cadbury one, I’d identify it, I’d be able to, like if there was a line‐up I would identify the Cadbury one. …

The colour immediately throws up Cadbury to me. …

36

David Bednall, Marie Ashwin and Alan Hirst Yeah well I guess it’s like symbols on a toilet door. You don’t need words, you know that that’s Cadbury’s and if you had half a dozen of them I’d be able to definitely put that as Cadbury’s and there’s a few other colours I guess you can link with other companies. …

If you say does this identify with a brand, I would say do I make a connection automatically between that and straight away I would make a connection between that colour and that brand.

There are several themes. First, there was a spontaneous conjuring up of the brand Cadbury, with little room for doubt for many people. The purple colour in particular, but also the purple in conjunction with the block shape, led the person to the origin, that is, Cadbury. Second, no other sources were named. Third, there was generally a high degree of certainty about the path from the colour pack to the source. Fourth, some people were at ease with describing the meaning to them of the word “identify” – “like the symbols on the toilet door” or “put a name to it” were examples. These outcomes are good evidence for the secondary meaning concept required by the courts – the indicium took the person to the source, few (or in this case no) other sources were named and people had no great issue with the survey question.

One comment illustrates a further issue with the questions in this area. The comment “the only thing missing is probably the writing and the two cups filling each other” reflects the knowledge that the interviewee had of the Cadbury branding that typically includes the “glass and a half” logo shown in Figure 2. In asking consumers to retrieve memories, associative memory networks are involved through what has been termed spreading activation (Anderson 1983). Here one memory triggers another and that spreads to further memories. So while the purple colour in the context of confectionery took the interviewee to “Cadbury”, the mental schema (Merrill 2000) that the person had about Cadbury then took them to the memory of the logo. Cadbury has also registered the logo as a trademark. Potentially it may also have sought survey evidence to support its claim.

This would require very careful framing of the survey topic (Bednall et al. 2012) and strict experimental controls over what is shown to interviewees to ensure that the pathway is from the indicium to the source – the essence of a trademark which has acquired secondary meaning – rather than through a less direct semantic pathway.

Figure 2: Cadbury pack with “glass and a half” logo While there was no indication in the interviews that the word “identify” was misunderstood, interviewees had difficulty expressing the concept in other words.

Interviewer: So the word, identify, sort of says you know what it is, or something like that.

Interviewee: Yes, you’ve grown up with it, it’s etched in your mind, it’s something you’re familiar with, it’s a familiarity.

Yeah, identify I haven’t looked it up recently in the dictionary so it’s hard when you actually get asked to define [an] everyday word that you just take for granted.

Identify, I mean that word identify I’m still thinking, identify, how can you identify something which is purple and square or rectangle. I mean it’s got no eyes or teeth or mouth or it’s not even related to me. But I can relate to it because I do eat it and I do like it.

Identify, colour, shape, it’s a little bit confusing. Identify? Identify? A bit confusing.

In order to explore this further, interviewees were asked about another context in which they might encounter the word “identify”. This was to quote the popular television series CSI (Nine Network 2013) and ask a question such as:

37

David Bednall, Marie Ashwin and Alan Hirst Interviewer: Just imagine a scene in CSI and a body turns up in a river and then [they] call in somebody to identify the body, what does identify mean then?

Interviewee: Put a name to it.

Similarly, some were asked about police asking someone to identify a body and as it happened one interviewee had actually needed to do this. The concept was clearly understood. The “put a name to it” example quoted above was the clearest representation of the identify concept. It takes them clearly to a singular source.

A final issue to emerge from the interviews was the use of the word “associate” given that it is this concept that has been used in many trademark surveys accepted by the courts.

Interviewer: If I'd used the word associate rather than identify, would that have made any difference? Do you associate this confectionery package with any particular brands?

Interviewee: It kind of removes it one step for me. I'm sort of thinking it sounds a trickier question that I've actually got to think more about, which might not get your initial response as quickly but that could just be me I suppose but I'm sort of thinking, I'm feeling like I'm not supposed to say Cadbury Dairy Milk I'm feeling it's making me think, okay Cadbury or Nestle or something, if you said associated with, I probably still would have blurted out Cadbury Dairy Milk because it does yell at you or it yells at me but I think identify, I would have had to puzzle over associate. I just would have thought associated with, not meaning it, it means something a little bit further removed. The partner not the associate.

In this case it is clear that “associate” has a different and somewhat less precise meaning. However, other interviewees might regard the words “identify” and “associate” as approximately equivalent. One exchange illustrates this: Interviewer: So what about the word identify? I asked you does this confectionery package identify any particular … so what does the word identify mean to you?

Interviewee: Associated with. I guess the best way to describe is like a memory game like how you’ve got a picture that matches a word, so the word would be Cadbury and that’s the picture and they put them together. Yeah matching I guess. …

Interviewer [In the context of CSI]: I’m just trying to see if you might use the word differently in different situations.

Interviewee: I think I do know because you can’t associate a dead body. I understand that.

In a true survey situation where the questions should be administered strictly as written, multiple interpretations are still possible. Thus some interviewees are likely to respond as if the word “identify” actually meant “associate”.

6. Conclusions It was found that the word “identify” mostly worked in the way intended, that it took the consumer to the source, that is Cadbury in the case of purple, but to no particular source in the case of lime, and that it was interpreted correctly. People could distinguish "identify" from "associate" though in a survey not all might do so. It is not a legal requirement in such surveys that every qualified person (that is, a buyer or consumer) reacts in the same way to a trademarked element, only that a sizeable group (usually a majority) do so (McCarthy 2012). In addition, few people should react in the same way to a control stimulus such as a lime pack. Surveys are used in many other types of litigation and here the questions, such as estimating periods of elapsed time, are even more problematic. Given that courts and tribunals need to rely on survey evidence to produce useful information, further cognitive interviewing studies are required to investigate the validity of the standard approaches taken.

In the context of a legal survey especially, where the meaning of particular words is crucial, cognitive interviewing methods can help establish the validity of the particular words in a survey. While it may be argued that in other commercial research, there is a tolerance for a degree of imprecision or ambiguity in survey research, such is not the case with legal surveys. The party presenting research needs to provide evidence of the rigour and validity of their approach because, if there are flaws, their opponents will surely discover them.

38

David Bednall, Marie Ashwin and Alan Hirst

References Abbott, K (2012), Should Cadbury be able to trademark the colour purple?, <http://www.guardian.co.uk/artanddesign/2012/oct/03/cadbury‐trademark‐colour‐purple‐open‐thread >.Accessed 24 January 2013.

Anderson, JR (1983), 'A spreading activation theory of memory', Journal of Verbal Learning and Verbal Behavior, vol. 22, no. 3, pp. 261‐95.

Beatty, PC & Willis, GB, ‘Research sythesis: The practice of cognitive interviewing.’ Poblic Opinion Quarterly, vol. 71, no. 2, 287‐311.

Bednall, D, Gendall, P, Hoek, J & Downes, S (2012), 'Color, champagne, and trademark secondary meaning surveys : devilish detail ', Trademark Reporter, vol. 102, no. 4, pp. 967‐1013.

Belson, W (1981), The Design and Understanding of Survey Questions, Gower, Aldershot, United Kingdom.

Dickerson, J (2006), Veuve Clicquot celebrates orange colour mark registration, World Trademark Review, retrieved September 19 2010, <www.worldtrademarkreview.com/daily>.

Evans, LEJ & Gunn, DM (1989), 'Trademark Survey Evidence', Texas Tech Law Review, vol. 20, no. 1, pp. 43‐7.

Foddy, W (1993), 'Checks to ensure that questions work as intended', in Constructing Questions for Interviews and Questionnaires: Theory and Practice in Social Research, Cambridge University Press, Cambridge, United Kingdom, pp.

181‐8.

Hoek, J & Gendall, P (2010), 'Colors, Brands, and Trademarks: The marketing (and legal) problems of establishing distinctiveness', Journal of Advertising Research, vol. 50, no. 3, pp. 316‐22.

Hudock, DC (1996), 'Qualitex Co. Vs Jacobson Products Co.: Color receives trademark protection and the courts receive confusion', Journal of Law and Commerce, vol. 16, pp. 139‐53.

Interbrand (2012), Best Global Brands 2012, <http://www.interbrand.com/en/best‐global‐brands/2012/Best‐Global‐ Brands‐2012.aspx>. Accessed 15 January 2013.

Jones, LC (1989). ‘Developing and using survey evidence in trademark litigation’, Memphis State University Law Review, vol.

19, no. 4, pp. 471‐490.

Keller, KL (2008), Strategic Brand Management: Building, Maesuring and Managing Brand Equity, Third edn, Pearson Education Inc., Upper Saddle River, NJ. McCarthy, JT (2012), McCarthy on Trademarks and Unfair Competition, Fourth edn, Thomson‐Reuters, Danvers, MA. Merrill, MD (2000), 'Knowledge objects and mental models', in Advanced Learning Technologies, 2000. IWALT 2000.

Proceedings. International Workshop on, pp. 244‐6.

Morris, N (2011), Women more likely to have a sweet tooth for chocolate, Roy Morgan Research, <http://www.roymorganonlinestore.com/News/Women‐more‐likely‐to‐have‐a‐sweet‐tooth‐for‐chocol.aspx>.

Nine Network (2013), New Season CSi:, Accessed 26 January 2013, <http://channelnine.ninemsn.com.au/csicrimesceneinvestigation>.

Payne, SL (1951), The art of asking questions, Princeton University Press, Princeton, NJ. Swann, J (2008), 'Swann on U.S.trade‐mark surveys', in RM Corbin & AK Gill (eds), Survey Evidence and the Law Worldwide.

A Reference Text For Lawyers, Jurists and Social Scientists, LexisNexis, Markham, Ontario.

United States Patents and Trademark Office (2011), What is a trademark?, Accessed 3 January 2011, <http://www.uspto.gov/faq/trademarks.jsp#_Toc275426672>.

Willis, GB (2005), Cognitive Interviewing: A Tool for Improving Questionnaire Design., Sage Publications, Inc., Thousand Oaks, California.

39

The Prevalence of Research Methodology Mis/Conceptions Among Business and Management Academics Frank Bezzina1 and Mark Saunders2 1 Department of Management, Faculty of Economics, Management and Accountancy, University of Malta, Msida, Malta 2 Surrey Business School, Faculty of Business, Economics and Law, University of Surrey, Guildford, UK mark.saunders@surrey.ac.uk

Abstract: This study examines the extent to which misconceptions and conceptions about research methodology are held by academics involved in its teaching. Following consideration of the nature of misconceptions, the literature on misconceptions of research methodology is reviewed. Whilst the notion of students’ misconceptions of research methodologies has been reported reasonably widely, little research explores the extent to which these are endorsed or contested by academics. This raises two research questions: “Are misconceptions regarding the nature of research prevalent among academics” and “To what extent do academics in the qualitative, qualitative and mixed methods traditions hold differing conceptions of research practice?” Using a web based questionnaire (comprising of 35 Likert‐type items on the nature of research and mis/conceptions of the research process, and 6 demographic questions) data were collected from a purposive sample of Business and Management academics with an expressed interest in research methodology. This resulted in 190 useable responses, a response rate of 26.5%. Data reveals considerable variability regarding what academics consider the purpose of research; in particular some academics contest whether it is a tool to answer questions, to find out more information about something that is already there, or to find solutions to problems.

Such misconceptions did not differ significantly between research traditions. Academics’ conceptions of research practice varied, the most salient differences being between those in the qualitative and quantitative traditions. Particularly evident were the contesting of commonly held beliefs about qualitative research among quantitative researchers. It is argued that, where academics pass such thinking on to students, problems of misconceptions and contested of beliefs are likely to pervade. Finding fault with one tradition on the basis of the standards of the other does little to promote understanding.

This emphasises the importance of academics involved in methods teaching recognising and express the legitimacy of different research traditions.

Keywords: research methodology, misconceptions, academics, nature, practice

1. Introduction The notion of misconceptions regarding some aspects of research has been reported in edited collections and academic papers reasonably widely. Within this, a clear thread of research has focussed upon highlighting misconceptions (often of students) regarding quantitative methodologies and specific methods, in particular techniques associated with statistical analyses (Huck, 2009), advice being offered regarding how to anticipate and address issues relating to specific techniques in the classroom (for example: Smith, 2011). Research suggests misconceptions also encompass both the nature of research (Meyer et al., 2005) and qualitative methodologies (Eby et al., 2009; Harper and Kuh, 2007). Such misconceptions have been argued to hinder meaningful learning, impede research progress and interfere with the decision making processes (Eby et al., 2009; Huck, 2009). Success of interventions to address these misconceptions is however dependent upon awareness, understanding and by those involved in their delivery. Yet, despite the interest in student misconceptions, we have been unable to find research that explores the extent to which methodological misconceptions are endorsed, addressed or left unchallenged by academics involved in the teaching of research methodology and methods.

The aim of this research is therefore to determine the extent to which misconceptions and conceptions about research methodology are held by those involved in its teaching. Following a brief consideration of the nature of misconceptions and the role of academics, we offer a review the literature on misconceptions of research methodology. We then outline our method, in which data were collected from a critical case purposive sample of Business and Management academics. This is followed by our findings in which we give particular attention to differences in conceptions in relation to stated research tradition. We conclude with a discussion of the implications of our findings.

40

Frank Bezzina and Mark Saunders

2. The nature of misconceptions and the role of academics Misconceptions are views or opinions that are incorrect due to faulty thinking or misunderstanding. They arise from prior learning or from interacting with the social and physical world and are reflected in contextualised beliefs, ideas or understandings of those engaged in research (Meyer et al., 2005). Whilst some misconceptions are grounded in human intuition resulting in faulty thinking, others are generated by inconsistencies in textbooks and well as in oral presentations in classrooms (Huck, 2009), including those by academics. Where these misconceptions have been used by students without correction, such as in assessed work, they can be considered faulty extensions of prior knowledge. Misconceptions that are deeply engrained in the students’ underlying knowledge base “represent knowledge that is functional but has been extended beyond its productive range” (Smith et al., 1993, p. 152), resulting potentially in poor research.

Some researchers have argued that successful instructional confrontation can replace faulty misconceptions with new expert knowledge in a short period of time (Brown and Clement, 1989). Others have shown that, where deeply engrained, exposure to more advanced instruction is not sufficient (Mevareck, 1983). In such instances, misconceptions are often so strong and resilient, that students are unwilling to alter their opinion even when confronted with contrary evidence (Garfield, 1995). However if academics also hold and pass on misconceptions, it is likely that the problem will pervade, students’ misconceptions being generated and supported by their mentors (Huck, 2009).

3. Mis/conceptions about research methodology Within this research, we adopt Saunders et al.’s (2012) definition of research methodology: the theory of how research should be undertaken rather than specific techniques used to obtain and procedures used to analyse data. Consequently we are concerned with misunderstandings relating firstly to what constitutes research and, secondly, to how research should be undertaken rather than misconceptions or misuse of particular data collection procedures or statistical analysis techniques.

4. Misconceptions regarding the nature of research Definitions of research abound, particularly in text books; those used emphasising consistently that it has clear purpose. This is operationalised as answering a question or solving a problem (Saunders et al., 2012), finding out more information about something that is already there or providing a deeper insight and understanding of a particular topic (Meyer et al., 2005). Research is defined as comprising systematic data collection and interpretation of data, being based on logical relationships rather than beliefs (for example Ghauri & Grönhaug, 2010; Saunders et al., 2012; Walliman, 2011). Although presenting research as a series of linked stages, such texts warn students in varying degrees of detail that research is rarely a linear process, emphasising that data analysis may result in contradictory and unclear findings which can be subject to multiple interpretations dependent upon the theoretical lens used (Saunders et al., 2012). They remind the reader to be aware of how their own and others’ biases can impact on research (Robson, 2011).

Literature relating to misconceptions regarding the nature of research focuses mainly on students. Meyer et al.

(2005) identify eight misconceptions in their exploration of the dimensionality of students’ conceptualisations of research. These they argue express faulty views, although no justification is given as to why these views should be considered ‘faulty’. In particular, students misconceive research as being about gathering data to support preconceived ideas, about collecting data which back up arguments, that correctly followed research procedures will always yield clear results, that if conducted properly research findings’ will never be contradictory, that when undertaken by qualified people (such as academics) research will always produce unbiased results, that it is acceptable to alter research data, that there is only one way to interpret research findings and that research becomes true after it is published. Not surprisingly, the converse of each is, as indicated earlier, often repeated in general discussions regarding the nature of research in research methods texts. Hence our first research question:

Are misconceptions regarding the nature of research prevalent among academics?

5. Differing conceptions of research practice Social science researchers often distinguish between quantitative and qualitative research traditions (Robson, 2011) and within this the role of theory, epistemological issues and ontological concerns (Bryman, 2012). The

41

Frank Bezzina and Mark Saunders quantitative tradition is likened to research in the natural sciences emphasising measurement and quantification and, usually, employing a deductive approach to the testing of theories. It is argued to incorporate the practices of positivism (Bryman, 2012), some advocates claiming that quantitative research is the only way to conduct serious research (Robson, 2011). In contrast, the qualitative tradition tends to employ a very different methodology to that of the natural sciences emphasising non‐numeric data (often words), tends to a more inductive approach and theory development rather than testing. Within this tradition, many advocates have rejected the practices of positivism emphasising the importance of context and openness and receptivity rather than objectivity; the latter being considered to distance the researcher from participants (Robson, 2011) and be unobtainable.

Despite the differences outlined above, the status of the quantitative/qualitative distinction is ambiguous, there being on‐going debate as to whether it is crucial or simply false (Bryman, 2012). For some, particularly those at the forefront of the “paradigm wars” (Hammersley, 1992, p. 131), quantitative and qualitative traditions are so different in their epistemological and ontological assumptions as to be incompatible. An advocate of research from one of the two traditions might therefore consider research in the other tradition misconceived.

Alongside the paradigm wars is a growing recognition of the value of combining quantitative and qualitative traditions in multi‐strategy or mixed methods research (Bryman, 2006). This combination of traditions is methodologically eclectic, often selecting and integrating those techniques and procedures that are considered most appropriate (best) to answer the research question or solve the problem be it theory testing or theory generation (Teddlie and Tashakkori, 2010). Mixed methods therefore focus upon the research question, drawing upon the practices of more than one philosophical position and using both inductive and deductive logic (Teddlie and Tashakkori, 2010). Within this, there is a focus upon compromise (Teddlie and Tashakkori, 2010), suggesting that advocates of mixed methods are less likely to consider conceptions of other research traditions (both quantitative and qualitative) as misconceived. Hence our second research question: To what extent do academics in the qualitative, quantitative and mixed methods traditions hold differing conceptions of research practice?

Harper and Kuh (2007), whilst focussing on qualitative assessment, highlight 10 key differences between quantitative and qualitative traditions focussing upon what they term “common misconceptions” (p. 6) about using qualitative methods. These reflect some of the epistemological and ontological differences between the quantitative and qualitative traditions, such as objectivity and subjectivity and associated superiority of quantitative (or qualitative) methods, the focus on generalizability rather than transferability of findings, the need for and role of numbers in research, as well as the overall ease of use of associated methods.

Similar aspects are considered by Eby et al. (2009) in their refuting of commonly held beliefs about qualitative research relating to the use of scientific method, methodological rigour in particular validity of findings, and the utility of the research findings. Based on an evaluation of 241 articles in nine top social science journals, they argue that the key steps of scientific method (observation and description) are “always present” in quantitative, qualitative and mixed methods approaches whether inductive or deductive logic is used. Given these arguments, it is not surprising that they conclude that like quantitative research, qualitative research also contributes to the advancement of knowledge.

6. Method Data were collected from Business and Management academics with an interest in research methodology using a web questionnaire. Interest was signified by membership of the Research Methodology Special Interest Groups (RM SIGs) of either the British Academy of Management (BAM) or the European Academy of Management (EURAM), or by attendance at the European Conference on Research Methodology (ECRM) at least once in the past two years. As a consequence, respondents can be considered to be a purposive sample comprising critical cases (Patton, 2002).

7. Design The questionnaire was designed using the SurveyMonkey online software tool. Following information about the survey (including assurances of anonymity) and a question requiring potential respondents to indicate

42

Frank Bezzina and Mark Saunders consent, the main survey contained 35 randomly ordered Likert‐style questions for which respondents were requested to indicate their level of agreement on a five point scale (strongly disagree, disagree, neither agree nor disagree, agree, strongly agree). These comprised 13 statements about the nature of research and 22 statements of conceptions about quantitative and qualitative traditions. Statements about the nature of research included eight statements of misconceptions identified by Meyer et al., (2005), a definition of research (derived from Saunders et al., 2012) and five statements (derived from Meyer et al.’s factor analysis) capturing additional aspects not included in the definition. 22 statements offering conceptions of quantitative and qualitative traditions included: 15 statements based on Harper and Kuh’s (2007) 10 ‘common misconceptions’, amended to avoid conflating two concepts in one statement; five further statements derived from Eby et al.’s (2009) research, as well as Bryman’s (2012) assertion that there is no universally superior research design. Following pilot testing, a statement was added contrasting the utility of secondary and primary data (derived from Cowton, 1988). The final section contained six closed questions requesting personal information including highest academic qualification, expertise in methods, and involvement in research methods teaching.

8. Procedure The questionnaire was distributed via an email invitation with a Web link to 540 people on the two RM SIGs electronic mailing lists as well as 196 past ECRM conference attendees compiled from the delegate lists, there being 18 non contacts. For the RM SIGs, the invitation was circulated by the membership secretary, whilst for past conference attendees, it was emailed by one of the researchers. Subsequently each potential respondent received a personal follow‐up/reminder re‐emphasizing the deadline for returns and restating the Web link.

Respondents were able to amend their responses until the questionnaire was ‘done’, the software allowing one respondent per work station, helping prevent multiple completions. The questionnaire took approximately 10 minutes to complete.

9. Participants 224 persons responded between March 7 and April 10, 2012. Of these, 34 chose not to complete the survey despite consenting to take part, their responses being discarded. The remaining 190 questionnaires (26.5% response rate) formed the basis of the analysis. 118 (62.1%) were members of the at least one RM SIG while the remaining 72 (37.9%) had attended the ECRM. The single largest groups were in possession of a doctoral degree (66.3%), based in a United Kingdom institution (63.1%), involved in research methods as project/dissertation supervisors for taught Master’s degree programmes (58.4%), and with mixed methods expertise in research (45.8%). 37.8% had expertise in qualitative methods and 16.8% had expertise in quantitative methods.

10. Data analysis procedure To answer our first research question, we generated a frequency report and descriptive statistics for the 35 ordinal statements. For our second research question, we used the Kruskal Wallis H test to determine whether significant differences existed in the responses of academics in the qualitative tradition, the quantitative tradition and the mixed methods tradition. Where the Kruskal Wallis H statistic was significant, the Mann Whitney U test was used as a post hoc test to distinguish between the three groups, with the Bonferroni correction being used to counteract for the problem of multiple comparisons among subgroups inflating the Type I error (Miller, 1991), effect size being computed to determine the magnitude of observed significant effects (Cohen, 1988; Rosenthal, 1991).

11. Findings Academics’ Mis/conceptions regarding the nature of research

Our first research question asked “Are misconceptions regarding the nature of research prevalent among academics?” Variations in academic’s responses (Table 1) highlighted clear differences with regard to this. In particular, variations were most marked with regard to both the purpose of research: “good research specifically gathers data that will support the researcher’s preconceived ideas” and “research is basically a tool about answering questions”; and process of research “if followed correctly research procedures will always yield clear results.”

43

Frank Bezzina and Mark Saunders Analysis revealed that such differences were only significant between academics from the quantitative, qualitative and mixed methods traditions for one item: “When academics do research, the results are always unbiased” (χ²=6.53, d.f.=2, p=.04), with those from the qualitative tradition disagreeing more strongly.

However, post‐hoc analyses revealed that this significant difference could not be attributed to a significant difference between these two groups, after the Bonferroni correction was applied (U=894.5, z=1.96, p>0.017).

Consequently whilst there are differences between academics in what they consider to be the nature of research these are not significantly related to their research tradition.

Table 1: Academics’ mis/conceptions regarding the nature of research Level of Agreement (%) (N=190)

Descriptive Statistics

strongly disagree

disagree

neither

agree

strongly agree

Md

IQR

Statement

2.1

10.0

12.6

44.2

31.1

4

1.25

8.9

19.5

22.1

40.0

9.5

3

2

11.1

26.8

32.1

25.8

4.2

3

2

.5

3.2

4.7

33.2

58.4

5

1

4.2

16.3

32.1

36.8

10.5

3

1

49.5

26.3

8.9

6.8

8.4

2

2

73.2

15.8

7.4

2.6

1.1

1

1

Research becomes true after it is published

46.8

39.5

7.9

4.7

1.1

2

1

There is one way to interpret research findings If followed correctly research procedures will always yield clear results If research is conducted properly then contradictory research will not occur When academics do research the results are always unbiased Research is about collecting data which back up your argument

65.3

25.8

5.8

1.6

1.6

1

1

10.5

49.5

16.3

16.3

7.4

2

1

27.4

51.6

11.6

7.9

1.6

2

1

51.1

39.5

6.3

2.6

.5

1

1

26.3

36.8

17.9

13.2

5.8

2

2

Research is the systematic collection and interpretation of data with a clear purpose, to find things out Research is basically a tool about answering questions Research means finding out more information about something that is already there Research provides a deeper insight and understanding of a particular topic Research is about finding solutions to problems Good research specifically gathers data that will support the researcher's preconceived ideas It is quite acceptable to alter research data if it does not look exactly right

Descriptive Statistics measures: Md = median, IQR = inter quartile range. Percentages may not sum to 100 due to rounding.

Table 2: Academics’ conceptions of research practice Level of agreement (%) N=190

agree

28.4

16.3

7.4

3

1

21.1

35.3

25.3

11.1

3

2

There are superior research methods*

65.3

27.4

4.7

1.1

1.6

1

1

Research is not the quest for truth**

14.7

32.6

29.5

11.1

12.1

3

1

Organisational decision makers respond only to numbers Objectivity is the gold standard in research

15.3

38.9

26.8

15.8

3.2

2

1

20.5

23.2

24.2

21.6

10.5

3

2

44

IQR

Neither

25.8

7.4

Md

disagree

22.1

The perspectives of few do not represent many

Subjectivity compromises trustworthiness

strongly agree

strongly disagree

Statement

Descriptive statistics

Frank Bezzina and Mark Saunders Level of agreement (%) N=190

Descriptive statistics

Statement Subjectivity compromises accuracy

17.9

30.5

28.9

18.4

4.2

3

1

Qualitative research lacks internal validity

49.5

28.9

13.2

5.3

3.2

2

1

Qualitative data are easy to collect; Anyone can do it!

Qualitative research lacks construct validity

61.1

24.2

7.4

3.2

4.2

1

1

40.0

34.7

18.9

5.3

1.1

2

2

67.4

22.6

6.3

2.1

1.6

1

1

70.0

22.1

4.2

2.6

1.1

1

1

28.4

44.2

19.5

5.3

2.6

2

2

28.4

44.7

19.5

6.8

0.5

2

2

55.3

32.1

8.9

2.1

1.6

1

1

30.0

43.2

15.3

8.4

3.2

2

2

27.4

47.4

17.4

6.8

1.1

2

2

40.0

38.9

12.6

6.3

2.1

2

1

23.7

34.7

31.6

8.4

1.6

2

1

31.1

28.4

21.1

15.3

4.2

2

2

21.6

53.2

16.3

6.8

2.1

2

1

30.5

48.4

16.3

4.2

.5

2

1

28.9

54.2

13.2

2.6

1.1

2

1

Qualitative data are easy to analyse; Anyone can do it!

Qualitative research contributes little to the advancement of knowledge Qualitative research does not utilize the scientific method Qualitative methods are too labour intensive to be practical for students’ assessed work Qualitative research lacks methodological rigour Only research findings that are generalizable can inform policy and practice Qualitative methods are too cumbersome to be practical for students’ assessed work Qualitative data are useful only when corroborated by numbers Self‐reported data are unreliable Quantitative data are useful only when corroborated by qualitative data Secondary data are mere substitutes for ’better’ primary data Quantitative methods are too labour intensive to be practical for students’ assessed work Quantitative methods are too cumbersome to be practical for students’ assessed work

*question reversed coded from “There is no superior research method”; ** question reverse coded from “Research is the quest for truth”; Descriptive Statistics measures: Md = median, IQR = inter quartile range.

Percentages may not sum to 100 due to rounding.

12. Differences in conceptions of research practice Our second research question asked “To what extent do academics in the qualitative, quantitative and mixed methods traditions hold differing conceptions of research practice?” (Table 2). As with items regarding the nature of research, responses ranged from strongly disagree to strongly agree.

Variation in responses was prevalent for all statements there being statistically significant difference (at p<0.017, after applying Bonferroni correction) between academics in the quantitative and qualitative traditions regarding “qualitative research lacks internal validity” (U=637.5, z=‐3.88, p<0.017, effect size r=0.38), “qualitative research lacks construct validity” (U=736.5, z=‐3.02, p<0.017, effect size r=0.30), “qualitative research contributes little to the advancement of knowledge” (U=702.5, , z=‐3.91, p<0.017, effect size r=0.39), “qualitative research lacks methodological rigour” (U=791.5, z=‐2.76, p<0.017, effect size r=0.27), “qualitative data are useful only when corroborated by numbers” (U=702.0, z=‐3.34, p<0.017, effect size r=0.33), “objectivity is the gold standard in research” (U=539.5, z=‐4.36, p<0.017, effect size r=0.43), “subjectivity compromises accuracy” (U=772.0, z=‐2.69, p<0.017, effect size r=0.27), “there are no universally superior research method” (U=828.5, z=‐2.60, p<0.017, effect size r=0.26), and “research is the quest for truth” (U=648.0, z=‐3.59, p<0.017, effect size r=0.35).

Academics in the mixed methods tradition differed significantly from those in the qualitative tradition in only two items. These were: “objectivity is the gold standard in research” (U=2074.5, z=‐3.64, p<0.017, effect size r=0.29), and “subjectivity compromises accuracy” (U=2199.5, z=‐3.21, p<0.017, effect size r=0.25). Academics in the qualitative tradition disagreed more strongly with these statements than those in the mixed methods

45

Frank Bezzina and Mark Saunders tradition, relative effect sizes indicating that the differing conceptions are more pronounced between academics in the qualitative and quantitative traditions than between the qualitative and mixed methods traditions. There were no significant differences in the item responses of academics in the mixed methods tradition and those in the quantitative tradition.

13. Discussion and conclusion This study provides empirical evidence that differing conceptions about the nature of research regarding research methodology are prevalent among some business and management academics with an expressed interest in research methodology. Although no significant differences were found between research traditions, we consider it worrying that even a minority of academics agree (at least to some extent) that “good research specifically gathers data that will support the researcher's preconceived ideas”, “it is quite acceptable to alter research data if it does not look exactly right”, “when academics do research the results are always unbiased” and “research becomes true after it is published”. It has been shown that such biases impact negatively on research (Robson, 2011) since they interfere with the proper interpretation of findings and consequently decision making. It seems likely that if academics with a special interest in research methodology hold such methodological misconceptions, other academics are also likely to hold them.

Responses to the statements regarding conceptions of research practice were highly dependent on the preferred research tradition. This finding supports the claim that interpretations depend upon the theoretical lens used (Saunders et al., 2012) and the meta‐theoretical assumptions that underlie the research (Weber, 2004). The most salient differences emerged between the proponents of qualitative and quantitative research.

Particularly evident was the contesting of commonly held beliefs about qualitative research by those who prefer quantitative research (Eby et al, 2009). In particular “qualitative research contributes little to the advancement of knowledge”, qualitative research lacks methodological rigour”, “qualitative data are useful only when corroborated by numbers”, “objectivity is the gold standard of research design”, and “subjectivity compromises accuracy”. These differences in conceptions appear to be strong and resilient and appear unlikely to be easy to alter. However, finding fault with one approach on the basis of the standards of another does little to promote understanding. As Siegle (2002) notes, each approach should be judged on its own theoretical basis. Consequently it is important that academics involved in the teaching of research both recognise and express the legitimacy of the different traditions.

For students, understanding both the nature and practice of research is essential if they are to develop their own research skills. Yet, academics involved in their learning hold both differing conceptions about the nature of research and differing conceptions regarding methodology, dependent upon their preferred methodological tradition. Consequently students’ conceptions may, as suggested by Huck (2009), be generated and supported by their mentors. Such misconceptions may interfere with their learning of concepts (Smith et al., 1993) related to one or more of the traditions. We would therefore argue that it is essential that those teaching research methods are able to articulate the reasoning behind differing conceptions of research within quantitative, qualitative and mixed methods traditions, recognising the utility of each. Alongside this there is we believe a need to identify innovative ways of “strengthening cohesion among all forms of research methodology” (Kelly & Kaczynski, 2007, p. 31), rather than supporting the divisions between them.

Acknowledgements Both authors contributed equally to this paper.

References Brown, D.E. & Clement, J. (1989) “Overcoming misconceptions via analogical reasoning; abstract transfer versus explanatory model construction”, Instructional Science, Vol. 18, pp. 237‐261.

Bryman, A. (2006)”Editor’s introduction: mixed methods research”. In. A. Bryman (ed.) Mixed Methods: Volume 1, London: Sage, pp. xxv‐lii. th Bryman, A. (2012) Social Research Methods.(4 edition) Oxford: Oxford University Press.

Cohen, J. 1988. Statistical power analysis for the behavioral sciences. Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Cowton, C.J. (1998) The use of secondary data in business ethics research. Journal of Business Ethics Vol 17, pp. 423‐434.

Eby, L.T., Hurst, C.S. & Butts, M.M. (2009) “Qualitative Research: The Redheaded Stepchild in Organisational and Social Science Research”, In. C. E Lance & R.J. Vandenberg, Statistical and Methodological Myths and Urban Legends: Doctrine, Verity and Fable in Organisational and Social Sciences, Routledge, Taylor & Francis Group, New York & London pp 219‐246.

46

Frank Bezzina and Mark Saunders Garfield, J.B. (1995) “How students learn statistics”, International Statistics Review, Vol. 63, pp. 25‐34.

Ghauri, P. and Grønhaug, K. (2010) Research Methods in Business Studies: A Practical Guide (4th edn). Harlow: FT Prentice Hall.

Hammersley, M. (1992) “The paradigm wars: Reports from the front.” British Journal of Sociology of Education.Vol 13, No. 1, pp. 131‐143.

Harper, S.R. & Kuh, G.D. (2007), “Myths and Misconceptions About Using Qualitative Methods in Assessment”, New Directions for Institutional Research, No, 136, Winter 2007.

Huck, S. W. (2009) Statistical Misconceptions, Psychology Press, Taylor & Francis Group, New York.

Kelly, M.A. & Kaczynski, D.J. (2007) “Misconceptions Students Brong to Qualitative Research”, Qualitative Research Journal, Vol. 6, No. 2, pp. 31‐44.

Mevareck, Z.R. (1983), “A deep structure model of students’ statistical misconceptions”, Educational Studies in Mathematics, Vol. 14, pp. 415‐429.

Meyer, J.H.F., Shanahan, M.P. & Laugksch, R.C. (2005), “Students’ Conceptions of Research. I: A qualitative and quantitative analysis”, Scandinavian Journal of Educational Research, Vol. 49, No. 3, pp. 225‐244.

Miller, R. G. (1991) Simultaneous Statistical Inference. New York: Springer‐Verlag.

Patton M (2002) Qualitative Evaluation and Research Methods. London: Sage.

Robson, C. (2011) Real World Research: A Resource for Users of Social Research Methods in Applied Settings (3rd edn).

Chichester, John Wiley. nd Rosenthal, R. (1991), Meta‐Analytic Procedures for Social Research (2 ed.) Newbury Park, CA: Sage. th Saunders, M., Lewis, P. and Thornhill, A. (2012). Research Methods for Business Students (6 edition). Harlow: Pearson.

Siegle, D. (2002) “Qualitative versus Quantitative”, Available at: http://www.gifted.uconn.edu/siegel/research/Qualitative/quanlquan.htm (Accessed 12 December, 2012).

Smith, J. P., diSessa, A. A., & Roschelle, J. (1993), “Misconceptions Reconceived: A constructivist analysis of knowledge in transition”, Journal of the Learning Sciences, Vol. 3, No. 2, pp. 115‐163.

Smith, M.A. (2011) “Missteps in multiple regression student projects: beyond association and not causation”, The American Statistician, Vol. 65, No. 2, pp. 190‐7.

Teddlie, C. & Tashakkori, A. (2010) “Overview of contemporary issues in mixed methods research” in Tashakkori, A. and nd Teddlie, C. (eds). The Sage Handbook of Mixed Methods in Social and Behavioural Research (2 edn.). Thousand Oaks, CA: Sage, pp. 1‐41.

Walliman, N. (2011) Your Research Project: A Step by Step Guide for the First‐Time Researcher (3rd edn). London: Sage.

Weber, R. (2004) “The Rhetoric of Positivism Versus Interpretivism: A Personal View”, MIS Quarterly, Vol. 28, No. 1, pp iii‐ xii.

47

Looking Through, Looking at and Looking Behind: Using Photos in Research Interviews Zelma Bone Charles Sturt University, Orange, Australia zbone@csu.edu.au

Abstract: Visual methods, for example, photographs are used in many disciplines such as anthropology, sociology, and psychology, but they are not widely used in business research. Four years ago, photographs were taken to capture the writing group experiences of nine business academics at an Australian university. Partly assisted by this photographic archive, the researcher re‐ visited and interviewed the writing group participants. The aim of this study was multi‐faceted: to capture the experiences of being part of the writing group and the writing retreat; to evaluate the value of the photograph in assisting recall of these experiences; to introduce business colleagues to the use and potential of photos in the research interview; and ultimately, to consider and encourage wider use of photos in their business research. This study uses photo elicitation interviews (PEI) using a phenomenological and narrative perspective with business academics to evoke recall of the writing retreat. The PEIs investigated the images to construct the internal (content of the image) narrative and the external (social context and social relationships) narrative. „Looking through refers to the image content as if looking through a window at a set of objects (Parker, 2009). What can we see? „Looking at is an examination of the way the content of the image is presented, for example, the use of light, arrangement of items. Finally, we gain understanding the context by „looking behind ; looking more deeply at the photographs and stories they are eliciting from the participants (Parker 2009). When introducing photographs part‐way through the interview process the immediate „rush of emotion was tangible. Responses became more graphic and more detailed. The photographs, taken by one of the participants, had the power to elicit information that would not have been gained in a traditional verbal interview. The potential for photographs to be used in business‐related research seems endless, as photographs can capture the historical and contemporary world of business. The photograph is applicable to business research because accounting, management and photographs share common features of portraying people, events and processes, and communication between present and past. As a result of this project, the business academics are more informed about the use and value of PEIs as they have had first‐hand experience. Many have expressed an interest in considering using photographs in future projects.

They have seen that this methodology would be useful to extract meaningful information from people and to minimise the awkwardness that sometime happens when interviewing. Revisiting the writing group and the associated writing retreat through the use of PEI highlighted the importance of collegiality and collaboration in the potentially isolating academic environment. Activities like this emphasise the key objectives of furthering the research agenda within the university, sharing ideas with colleagues, benefits of learning about different research methods, and collaborating on research projects. This study also emphasised the importance of considering academics interpretations of their experiences when generating policies about developing quality teaching and learning in business disciplines.

Keywords: photo‐elicitation, business, visual methodologies research interviews

1. Introduction The use of visual stimuli in the form of photographs, drawings, art and video recordings have been increasingly used for research in the fields of ethnography, anthropology, sociology and psychology (Harrison 2002; Frith & Harcourt 2007). However, visual methods such as photographs have not been widely used in the business disciplines (Parker 1999; Parker 2009; Petersen & Oestergaard 2003; Warren 2005). Hurworth (2003) found that introducing photographs into research interviews has shown to have many advantages, including it can be used at any stage of the research; produces unpredictable information; promotes longer, more detailed interviews in comparison with verbal interviews; and provides a component of multi‐methods triangulation to improve rigour. The potential for photo‐elicitation interviews (PEI) to be applied to the business research field, where appropriate, would appear to be endless.

This paper reports on the use of photographs to capture the writing group experiences of business academics in an Australian university. The researcher is re‐visiting the writing group experience four years after the writing retreat took place. The aim of this research was multi‐focussed: to capture the experiences of being part of the writing group and the writing retreat; to evaluate the value of the photograph in assisting recall of these experiences; to introduce business colleagues to the use and to recognise the potential of photos in the research interview; and ultimately, to consider and encourage wider use of photos in their business research.

48

Zelma Bone

2. Background In 2008 three academics, two from the Faculty of Business and one from the Faculty of Education, were successful in gaining a Scholarship in Teaching (SiT) grant to initiate a SiT writing group for academic staff within the Faculty of Business. Nine academics volunteered to be part of the writing group and the group was facilitated by the three academics. The academic from the Faculty of Education acted as a „broker (Wenger 1998) in the writing group community of practice to introduce members to the concept of scholarship in teaching. The writing group met on a fortnightly basis during the year and culminated in a three day writing retreat which formed the context for the PEIs.

Our focus for the SiT project was on scholarly writing, a core capability for academics that has traditionally been seen as a solo, often lonely, endeavour. We challenged this representation by demonstrating how a writing group can create for early career academics a cooperative space for situated learning within an academic workplace. Building on the work of Lee (1997) we revisited questions such as: How is the knowledge in a field produced? What models of research are available for generating new knowledge? What are productive ways of bringing together theory, practice and research in developing academic writing and publication?

There were both „hard and „soft outcomes related to the writing group project. The hard outcomes were measurable, observable and brought recognition to the university. These outcomes included the articles accepted for publication in journals, the conference presentations and peer‐reviewed conference papers which were products of the writing group process. The softer outcomes while immeasurable were perhaps the most valuable in terms of the initial purpose of the writing group. All of the members of the group have increased in their confidence as writers, and as reviewers of their colleagues work. They feel comfortable sharing their ideas and work‐in‐progress with the writing group community of practice, and can confidently reflect on and initiate ideas around scholarship of teaching. One participant took numerous photographs of the event.

As it was a very successful exercise we wanted to „tell the story‟ of the writing group as a means to generate more interest in scholarship in teaching. One way of capturing the experiences of the writing group and writing retreat was by revisiting the exercise through this study and by using the photographs as a stimulus to evoke the reflections and the memories.

3. Photo elicitation interview (PEI) Photo elicitation involves „the simple idea of inserting a photograph into a research interview‟ (Harper 2002:13). The earliest forms of photo elicitation can be traced to the work of photographer and researcher John Collier (1957) at Cornell University. Collier and his research team used PEI to examine mental health in changing communities in Canada. Harper (2002) reported that Collier and the research team did PEI as well as non‐photographic interviews with the same families to see how each method worked. The PEI was found to elicit longer and more comprehensive interviews with richer data being collected (Collier 1957:858).

The introduction of the photograph into the interview can make the interview more relaxed. Schwartz (1989:151‐152) highlighted that interviewees respond to photographs: ... without hesitation. By providing informants with a task similar to viewing a family album, the strangeness of the interview situation is averted.

Hurworth (2003) also identified that PEIs assisted with building trust and rapport by bridging psychological and physical boundaries as well as giving the researcher the opportunity to collect a combination of visual and verbal language.

Parker (2009:11‐12) focussed on opportunities for the visual methodologies in business research, particularly in the accounting and management areas. Firstly, photographs of people, places and objects can illustrate human activity in the business world and can reveal more of the many experiences and behaviours that may not be captured purely through textual accounts (Petersen & Oestergaard 2003; Warren 2005). Secondly, unlimited opportunities in exploring the symbolism in historical commercial and business architecture; and studies of the context in which „scientific management and work study was photographically applied to factory and work functions for analysing their process and improving their efficiency (Parker 2009:12). And thirdly, searching archives for photographs and using contemporary photographs for research purposes.

49

Zelma Bone These potentially offer opportunities for deconstruction and critique of various situations, challenging past beliefs and interpretations of events.

4. Methodology This was a qualitative study using photos as a visual research method. This method is subjective as the photographs elicit individual and social constructions and how to visualise the past and present it in new and different ways (Parker 2009). PEIs give the „potential for contextualising, interpretative and critical discovery using a phenomenological and narrative perspective (Epstein, Stevens, McKeever & Baruchel 2006:2).

Phenomenology and narrative inquiry are both forms of qualitative research.

Phenomenology is the study of „phenomena: appearance of things, or things as they appear in our experience, or the ways we experience things, thus the meanings things have in our experience (Smith 2011:1). Lester (1999) identified that phenomenological methods are particularly effective at bringing to the fore the experiences and perceptions of individuals from their own perspectives. The photograph has the ability to reflect and evoke feelings, signification, and multiple voices that offer understandings and critiques that go beyond the confines of representation through language (Parker 1999: 10). In this way the photo can augment the textual analysis.

According to Clandinin and Connelly (2000) narrative inquiry is an understanding of narrative as both phenomena under study and method of study. Narrative inquiry uses many forms of text materials but also can include photographs as units of analysis. It is an approach to understanding and researching the way people create meaning of their lives as narratives. The collection of photos used in this research project was chosen to convey a strong implied narrative: the content of the photographs providing an internal narrative and the context of the photographs providing the external narrative (Parker 2009; Banks (in Epstein et al.

2006)). Banks (in Epstein et al. 2006:5) also advised researchers to distinguish between the content of a photograph and its context. Photographs can be interpreted internally (content: what message has been sent) and externally (context: what we see in the photo). Interpretation of the photographs was extended to „looking through (content), „looking at (the content) and „looking behind (the context) (Parker 2009) to assist in understanding the main themes. The photographs used in this study are attached in Appendix 1.

In each interview, the writing retreat participants were asked about their recollections of the writing retreat at Millamolong and for their motivations for being part of such an exercise. The aim of the retreat was to get far away from the work environment, to Millamolong homestead, (built environment); to a situation with nice surrounds (natural environment); and with a group of people that had common ground (social environment). The photographs were categorised into those representing the built, natural or social environments.

The photographs were introduced into the interview part way through. Key questions used in the interviews to prompt responses and gain recollections were:

Why do they exist? Who created them? What is the biography? What do they represent?

The photographer – under what circumstances were the images created? By whom and for what audience? Who had control of the image content?

The built environment (Appendix 1: Photos 1‐4). What did you like/dislike about the place?

When you were there how did you feel? Any sounds? Smells? What do you think your colleagues liked about this place? What would you change?

a farm, far away from work? What are the things you did here – in the natural environment? What did you like/dislike? What would you change?

The natural environment (Appendix 1: Photos 5‐8). What was it like being close to nature, on

The social environment (Appendix 1: Photos 9‐13). What do you think of these photos? What

were the good and not so good things about having time together in this type of activity? What were you doing when others were in discussions?

50

Zelma Bone

5. The photographer and the photographs The photographs represented the experience of the writing retreat through the „lens of one of the participants and thus were significant to the interviewees (Keller et al. 2008). The photographer was a keen amateur photographer with a strong sense of the aesthetic. The interview participants acknowledged that he has a great deal of empathy and respect for people. This comes through in the content and context of the photos. He was trying to „capture the essence of the people who were there and he took the photographs because he wanted to give the participants a record of the time together (Participant response).

The original photographs were in colour. However, these were photocopied in black and white for the interviews as they tended to be „more evocative than colour photography. It stimulated a faster onrush of memories because „less has been given, more has been left out ... (Berger 1992:192‐193). The photographs were a visual reminder of the experience of the writing retreat and aided the participants in the telling their story of this event. The photographs captured many different emotions, situations, locations, and relationships that had been „snapped in the image. For example, one of the most „popular photographs was the meal time around the kitchen table (Figure 1).

The photos were taken at the writing retreat and then stored with the organiser. Four years later the photographs were reviewed and used for this project s interview to evoke oral narratives. Dominant themes were identified from the interview narratives. Parker (2009:10) suggested one way of interpreting photographs to capture the layering of image viewing and reading was by looking through, looking at and looking behind.

Figure 1: The kitchen table – smiling faces and friendly gestures

6. Looking through and looking at: The content or internal narrative „Looking through‟ refers to identifying the image content, i.e. the internal narrative, as if looking through a window at a set of objects (Parker 2009). What can we see through the window? What are people wearing?

What/how are they eating/discussing/working? Facial expressions? Gestures?

„Looking at‟ is an examination of the way the content of the image is presented, for example, the use of light, arrangement of elements (Parker, 2009). What does the image imply? What is represented? What is absent?

Why?

The dining room table in the kitchen was a favourite „place‟ to gather for informal conversations. Interactional mannerisms indicated a group that knows each other well, are comfortable with each other and are pleased to be spending time together. The casual attire of the participants reflected the relaxed atmosphere of the place.

These photos represent the social environment (Figures 1&2; Appendix 1: photos 10‐13).

51

Zelma Bone

Figure 2: (Appendix 1: Photo 11): Around the kitchen table – „Relax! Everyone is here, everyone has arrived safely! Lunch on the first day, our first meal together. We have worked up an appetite. The agenda for the retreat – we have no agenda. Time to write; time to discuss; time to reflect. When we are ready we will eat and have breaks. Great conversations, great company.‟ (Participant comment).

The built environment is represented in Figure 3 and Appendix 1 (photos 1‐4). Millamolong homestead with its native garden setting was the location for the writing retreat (Figure 2). It was a large nine bedroom homestead ninety minutes drive from campus. It provided space apart but space to be together; it was off campus with no „work distractions; with comfortable surrounds; and pleasing aesthetics.

Figure 3: Millamolong homestead. „Stunning. It was spring. The rural surrounds were idyllic .„Nothing to dislike about the place . „Loved the architecture of the house. Bits added on. It was an organic house . (Participants comments) The bucolic location was peaceful and quiet with no distractions from everyday work pressures. The natural environment (Appendix 1: Photos 5‐8) gave many the opportunity to walk the undulating hills of the farm. A vineyard (Figure 4) was part of the property. It was arranged for the vineyard manager to give a wine tasting on the veranda one evening. The group learned more about the history of Millamolong and grew more attached to the „place‟.

The social environment (Figure 5 and Appendix 1: Photos 9‐13) depicted the numerous discussions, eating times, relaxing in the formal lounge and walking in the grounds. Figure 5 shows the warmth of the light streaming in the window of the formal lounge room. It was a situation conducive to collegial discussions and relaxation.

52

Zelma Bone

Figure 4: (Appendix 1: Photo 8) Horse and vineyard and (Photo 5) Busy bees and flower –„All of the vineyard is above 600 metres in altitude. Wine tasting at 6.30pm on the front veranda. We learned about the history of Millamolong. We grew more attached to our place – we were putting the treat into retreat. „Busy as a bee! Again we reflected what was happening in nature. Busy bee in the garden; busy as a bee in the homestead. Heads down and writing. Making the best use of our time.

(Participants comments)

Figure 5: (Appendix 1: Photo 12) Discussion point with sun streaming through the window –„The formal lounge gave us a nice, comfortable space to work and to discuss things. The sun coming in the window is warming, as was the discussion. We have separate corners to work in this room but we also have room to congregate near the window. (Participant comment)

7. Looking behind: The context or external narrative „Looking behind signified the focus on image context as the construction of an external narrative (social context and social relationships) (Parker 2009). This involved looking more deeply at the photographs and the stories that they elicited from the participants. The external narrative woven from the participants stories gave a rich perspective on the people present, their feelings about the location and the experience of the event (Beilin 2005). Understanding this context was vital to the interpretation. A strong sense of collegiality and collaboration was portrayed. The colleagues „welcomed the peace and being able to get away from „crowded days (Figure 6).

A common reaction from the participants when shown the photographs was delight as the memories of this time were brought to the fore. The photographs „spurred meaning that otherwise might have remained dormant in a face‐to‐face interview (Clark‐Ibanez 2004:1513). The participants responses lengthened and became more detailed as they explored the photographs. The colleagues spoke warmly of their shared focus on teaching and learning research and acknowledged their differing approaches as the nine academics came from varying business disciplines ‐ accounting, finance, management, and marketing.

These multiple voices were contributing to the reflections. Accountants in the group did things „differently to colleagues from the other disciplines. They preferred to get straight to work and have

53

Zelma Bone discussion later on. Others liked to discuss first and then get to work on their individual projects or collaborations.

The photographs were also re‐examined with a view to interpreting the detail and nuances of the organisational life, the writing retreat portrayed and the changes that have occurred since the photographs have been taken. Davis (1973:18) described the sense of nostalgia that can be evoked when using PEI: „a positively toned evocation of a lived past in the context of some negative feeling toward the present of impending circumstance . The study elicited strong feelings of nostalgia from all participants: „Reminds me of a nice time. Sad that we will not have this again . „Belonged to an era that has past . „Something like this would not get funded now . (Participants comments)

Figure 6: Night time in the lounge – „Big comfy chairs for all in front of the open fire. Well fed, a couple of wines and all is right with the world. It is a very collegial atmosphere. „Glorious photos. Great photos of a very positive experience. People were engaged and supportive. „Most valuable time.

Very happy social experience. I enjoyed it very much. „Warmth. Just being together with a group of people doing, thinking, reflecting about writing. Peacefulness. (Participants comments) The writing retreat, four years in the past, was associated with positive affects, while the present was viewed in a relatively negative way. The current climate in higher education was seen as a time of increased compliance with greater accountability, greater demands on academic‟s time and more emphasis on discipline‐related research outcomes in performance management. The participants acknowledged that research outcomes were important but not nearly as important as the social components of the workplace. They emphasised the significance of achieving outcomes and improving writing skills as well as output. The writing retreat gave them the time and space to put many of the writing skills they had learned over the year into practice. The motivations for being part of the writing group and retreat were to improve writing skills to increase publication and to spend time with colleagues in a pleasant environment away from workplace pressures.

Figure 7: The front veranda – „Cane chairs for relaxed meetings, quiet readings. Wine tasting in the evening, early morning cuppas. Reflective practice on the final day. A strong communal space.

(Participants comments)

54

Zelma Bone Which group of photographs did the participants like best? All of the participants agreed that it was the „social environment group of photos that were their favourites (Figure 7; Appendix 1 Photos 9‐13). Why? In their words: „Because it reflects the collegiality, contact with people . „Reminder of the very good vibe, the participation, friendly atmosphere, a very cohesive group . (Participants comments)

8. Issues and implications Revisiting the writing retreat through the use of the PEI highlighted the importance of collegiality and collaboration in an academic environment which often can be isolating. Activities such as the writing group and writing retreat emphasised the key objectives of furthering the research agenda within the university, sharing ideas with colleagues, benefits of learning about different research methods, and collaborating on research projects. The benefits of having the experience recorded in photographs which the group share are ongoing.

The PEI gave the participants a chance to reflect on their writing group experiences and continue the conversations around learning and teaching and leading to further collaborative projects. Harper (2002:23) identified the significance of photographs: My enthusiasm for photo elicitation also comes from the collaboration it inspires. When two or more people discuss the meaning of photography they try to figure out something together. This is, I believe, an ideal model for research.

From a research perspective, the photographs generated data on the writing retreat that the interviewee could see but was invisible to the researcher (Schwartz 1989). The photographs were a powerful tool to elicit richer information about the experience. The greater detail in responses was due largely to the introduction of the photographs into the interview. Berger (1992:192) found that „the thrill found in a photograph comes from the onrush of memory‟. In this study, the participants‟ responses gave different perspectives on the experience with differing emphases on the significance of events.

However, the reflections identified an activity like this is almost a thing of the past (particularly in the learning and teaching areas). Writing workshops and retreats are now more strictly focussed on specific discipline‐ related research, achieving publications in the highest ranked journals and gaining the university kudos in the research arena.

9. Conclusion The use of the photograph in the research interview with the business colleagues was very successful in evoking powerful recall of the writing group experience and, in particular, the writing retreat at the end of the year. When the photographs were introduced part way through the interview the immediate „rush of emotion was tangible. Responses became more graphic and more detailed. A strong sense of nostalgia, relating to good times in the past was evident. The photographs had the power to elicit this information that may not have had been gained in a traditional verbal interview. As a result of this project, the business academics are more informed about the use and the value of the PEI as they have had first‐hand experience. Many have expressed an interest in considering using photographs in future projects. They have seen that this methodology would be useful to extract meaningful information from people and to minimise the awkwardness that sometime happens when interviewing.

Although relatively under‐utilised in business research, the potential that photographs could bring to projects are almost endless. However, before incorporating PEI researchers must be able to justify that PEI is the most appropriate method to generate data. The photograph is very applicable to business research because accounting, management and photographs share common features of portraying people, events and processes, and communication between present and past (Parker 2009). Many advantages have been highlighted in the paper that would support the use of photographs, or other visual methods, becoming an integral part of both research methodology and its outputs.

55

Zelma Bone

Appendix 1: The photographs of the writing retreat at Millamolong Homestead Built Environment – buildings, physical structures

Natural Environment – natural surroundings, animals, birds, trees

1. The homestead

5. Busy bees and flower

Social Environment ‐ people, communal spaces, discussion points, meal times 9. Communal kitchen

2. Sculpture in the garden

6. Bird in trees

10. Night time in the lounge

3. Kitchen with Aga#

7. Mare and foal

11. Around the kitchen table – first meal together

4. The formal lounge

8. Horse and vineyard

12. Discussion point with sun streaming in

13. The front veranda

References Beilin, R. (2005) “Photo‐elicitation and the agricultural landscape: „seeing‟ and „telling‟ about farming, community and place”, Visual Studies, Vol. 20, No. 1, April 2005, pp 56‐68. Berger, J. (1992) Keeping a Rendezvous, Vintage International, New York.

Clandinin, D. J. and Connelly, F. M. (2000) Narrative Inquiry: Experience and Story in Qualitative Research, Jossey‐Bass Publishers, San Franisco.

Clark‐Ibanez, M. (2004) “Framing the Social World with Photo‐Elicitation Interviews”, American Behavioural Scientist, Vol. 47, No. 12, August 2004, pp 1507‐1527.

Collier, J. (1957) “Photography in anthropology: a report on two experiments”, American Anthropologist, 59, pp 843‐859.

Davis, F. (1973) Yearning for Yesterday, Free Press, New York.

Epstein, I., Stevens, B., McKeever, P., and Baruchel, S. (2006) “Photo elicitation interview (PEI): Using photos to elicit children‟s perspectives”, International Journal of Qualitative Methods, 5 (3), Article 1. Retrieved 22/10/2008 from http://www.ualberta.ca/~iiqm/backissues/5_3/pdf/epstein.pdf Frith, H. and Harcourt, D. (2007) “Using photographs to Capture Women‟s Experiences of Chemotherapy: Reflecting on the Method”, Qualitative Health Research, Vol.17, No. 10, pp 1340‐1350. http://qhr.sagepub.com. Accessed 21/10/2008.

56

Zelma Bone Harper, D. (2002) “Talking about pictures: a case for photo elicitation”, Visual Studies.Vol. 17, No. 1. Harrison, B. (2002) “Seeing health and illness worlds – Sing visual methodologies in a sociology of health and illness: A methodological review”, Sociology of Health and Illness, 24(6), pp 856‐872.

Hurworth, R. (2003) “Photo‐Interviewing for research”, Sociology at Surrey: Social Research Update, University of Surrey, Issue 40.

Keller, C., Fleury, J., Perez, A., Ainsworth, B. and Vaughan, L. (2008) “Using Visual Methods to Uncover Context”, Qualitative Health Research, Vol. 18, No. 3, March 2008, pp 428‐436. http://qhr.sagepub.com. Accessed 21/10/2008.

Lee, A. (1997) “Working together? Academic Literacies, Co‐production and Professional Partnerships”, Literacy and Numeracy Studies, 7, (2), pp 65‐82.

Lester, S. (1999) “An Introduction to phenomenological research”, Stan Lester Developments, Taunton, UK. www.sid.demon.co.uk/resmethy.pdf. Accessed 8/10/2012.

Parker, L. D. (1999) “Historiography for the New Millenium: Adventures in Accounting and Management”, Accounting History, 4, 2, pp 10‐42.

Parker, L. D. (2009) “Photo‐elicitation: An ethno‐historical accounting and management research perspective”, Accounting Auditing & Accountability Journal, 22:7, pp 1111‐1129.

Petersen, N. J. and Oestergaard, S. (2003) “Organisational Photography as a Research Method: What, How and Why”, Academy of Management Conference Proceedings, Submission identification number 12702, Research Methods Division.

Schwartz, D. (1989) “Visual Ethnography: Using Photography in Qualitative Research”, Qualitative Sociology,12, 2, pp 119‐ 154.

Smith, D. W. (2011) “Phenomenology”, The Stanford Encyclopedia of Philosophy (Fall 2011 Edition), Edward N. Zalta (ed.). <http:??plato.stanford.edu/archives/fall2011/entries/phenomenology/> Accessed 4/10/2012 Warren, S. (2005) “Photography and Voice in Critical Qualitative Management Research”, Accounting, Auditing & Accountability

57

Ethnography in Information Systems Research Ann Brown Cass Business School, City University, UK a.p.brown@city.ac.uk

Abstract: This paper reviews and assesses the quality of the research process and description displayed by five recent papers using ethnographic methods, published in major IS journals. The IS ethnographer has two tasks – to carry out good research and to write it up the work in such a way as to convince readers as to the value of the results. The aim of this paper is to assess to how well these two aims have been met by the five papers.

Keywords: ethnographic methods, IS research

1. Introduction Ethnographic research methods can yield extra‐ordinarily insightful results but are costly for the ethnographer in terms of time and are complex in application. Hence for any researcher in IS, the decision to use these methods has serious consequences.

Ethnography has its roots in the study of social groups within unfamiliar cultures. The method yields large benefits for research questions addressing inexplicable or puzzling or unwelcome group behaviour. Since organisations can be viewed as societies with their own peculiar customs and practices, ethnographic research has great potential for the study of management in organizations (Myers, 1999; Evered and Louis, 1981). For a similar reason it has an important place in studying the impact of new Information systems on organisations.

Nonetheless, it still forms only a small subgroup of empirical studies in Information Systems (Mingers, 2003; Leidner and Kayworth, 2006; Tan and Hall, 2007).

This paper reviews five recent papers that apply ethnographic methods in IS research, taken from leading IS journals. The IS ethnographer has two tasks – to carry out good research and to write it up the work in such a way as to convince readers as to the value of the results. The aim of this paper is to assess to how well these two aims have been met by the five papers.

The next section outlines the characteristics of ethnographic research methods. Section three considers the five papers use of ethnography with a view to establishing the type of Information Systems research question for which this technique is well suited. Section four analyses how the papers dealt with carrying out and presenting the main steps of an ethnographic research study. The problems of evaluation are presented in section five with an assessment of how well the papers dealt with this aspect.

2. Ethnographic methods Ethnographic research has its roots in anthropological studies, where researchers would travel to faraway places to study the customs and practices of less well known societies (Sanday, 1979; Fetterman, 2010). It is a methodology based around extensive empirical research. ‘Fieldwork is the heart of the ethnographic research design’ (Fetterman, 2010). The ethnographer can expect to spend considerable amounts of time in the field, typically between 6 months to a year (Sanday, 1979; Van Maanen, 1988; Yin, 1994; Klein and Myers, 1999). He or she immerses themselves in the field so that they come to see how life and the phenomena under investigation are viewed by the social group and/or organisation. The researcher seeks to see what people and the organisation actually do as well as what they say they do. A key aim is to understand how the subject group interacts with each other and with the specific social and cultural context of the time and place of the research, when dealing with the phenomenon under investigation. Hence the study results will be critically affected by the contextual circumstances. When behaviour fails to match expectations, this research method enables the ethnographer to critically assess the taken for granted assumptions on which such expectations are based and perhaps to create a better understanding of the drivers for individual and group behaviour (Myers, 1999). First‐hand knowledge of what happens in the field may contradict commonly held assumptions, highlight hidden agendas and reveal fundamental misconceptions (Myers, 1999; Schultze, 2000). Although ethnographic methods are most closely associated with the phenomenological philosophic paradigm, in fact these methods can be and have been applied within all three of the dominant philosophical perspectives

58

Ann Brown accepted by Information Systems researchers ‐ positivist, interpretivist and critical (Orlikowski and Baroudi, 1991).

Ethnographic methods include a wide range of possible approaches and the researcher has to make a series of decisions on these, early on in the project. Fetterman (2010) suggests that a key choice set by the original aim is between basic research (concerned with theory development) or applied (with direct practical implications).

The main schools of thought vary in their view as to what type of relationship the ethnographer should aim for when immersed in the social setting (Sanday, 1978; Van Maanen, 1988). The options vary from ‘going native’ to soak in the way of life of the target group to the semiotic school in which the interpretation of the symbols used by the social group is the paramount objective.

Data collection, analysis, interpretation and reporting are carried out in parallel. The direction of the research (even the research question and initial boundaries) may undergo a number of changes as the project progresses. The ethnographer is making choices and applying personal judgements at all stages of the project and hence his/her role becomes an important element in the final shape of the project and the results obtained.The challenge for the ethnographic researcher is to carry out and report on the research in such a way as to convince his or her audience of the value and credibility of the work.

3. Using ethnographic methods in information systems research I have selected five recent academic papers to analyse, each of which reports on the application of ethnographic methods in Information Systems Research. These are taken from three leading IS academic journals ‐ MIS Quarterly, IT and people and Information and Organization. The papers are: ‘Using Performance ethnography to explore the human aspects of software quality’ by Neil McBride (2008); ‘A confessional Account of an Ethnography about Knowledge Work’ by Ulrike Schultze (2000); ‘The sociology of a market analysis tool: How industry analysts sort vendors and organize markets’ by Neil Pollock and Robin Williams (2009); ‘Telemedecine in the Upper Amazon: Interplay with Local health Care Practices’ by Gianluca Miscione (2007) and ‘Hidden Agendas, power and managerial assumptions in information systems development; An ethnographic study’ by Michael Myers and Leslie Young (1997).

The research work for three of the papers focused on the impact of the introduction of a new IS. Schultze (2000) uses the implementation of a new IS tool (KnowMor, a knowledge management technology) in a major US company, to investigate Knowledge Work in order ‘to develop a grounded understanding of what producing information entails and what Knowledge Work is like’. Miscione (2007) joined a project team tasked with the job of evaluating the new Telemedecine system (RISS) implemented in Peru, in terms of its effects on a wide number of stakeholders (public health care system, local health care practice, local population, patients, traditional healers). Myers and Young (1997) investigate the steps in the development process of a new Management Information System being designed for the New Zealand Health Sector. One paper (by Neil McBride, 2008) is an extra‐ordinary piece of performance ethnography based on the author’s experience as a software engineer and is concerned with the factors affecting software quality. Pollock and Williams (2009) investigate the role played by IT research firms employees (industry analysts) in the IT procurement market, specifically the power of these analysts to affect buyers assessment of the functionality of vendor IS. All papers have an IT artefact at the heart of the research. The driver for the research was either the unexpected or unwanted behaviour generated by the use of new IS artefacts or the failure of traditional formal methods (software quality) or the complexity surrounding the evaluation of systems functionality. This suggests that the IS ethnographic researcher tends to have a relatively specific phenomenon in mind at the early stages if the research, which he or she wishes to investigate.

What were the reasons given by these authors for adopting ethnographic research methods? The reasons are closely tied to the research aims. Several papers made no explicit case for adopting ethnography. The overarching research aims were stated and ethnography (sometimes with other qualitative methods) was assumed to be the most appropriate approach. So Miscione sought to understand what happens when a Telemedecine system has been implemented – to obtain a ‘better understanding of the processes of health care institutionalization through telemedecine’ (p404, Miscione, 2007). With a research focus on the social aspects of the implementation of the RISS system, the use of ethnographic methods was stated as the method to be applied and (presumably) did not need justifying. Pollock and Williams (2009) also spend little time on justifying the use of ethnography. Their study was sparked by their perception of the power exerted by

59

Ann Brown industry analysts on the IT vendor industry. They explain that this led them to focus on the interactions of IT research firms’ staff with players in other organizations, in particular IT managers who were potential buyers of the IT systems. Myers and Young (1997) are concerned with the social and political factors that often seem to drive IS development. Their research aim was ‘to explore’ these aspects in a case example of the development of an MIS in the NZ health care system. The choice of critical ethnography is discussed in detail and they show how well this method supports the research aim and the investigation. McBride (2008) and Schultze (2000) both applied a personal approach to ethnography, hence it’s use was in a sense an integral part of the research study and aims. McBride(2008) uses his personal experience of software quality issues to apply autoethnography. The impact of his personal creativity, the contextual cultural and political factors on his software projects are shown through his use of performance ethnography. Schultze aims to ‘explore knowledge work in situ’ (p4, Schultze, 2000). She spends considerable space in establishing the significance of the individual role in knowledge work and the case for ethnographic research methods. Taking a confessional writing approach allows her to include her own experience of knowledge work as a researcher.

What does this suggest about the type of IS research problems for which ethnographic methods may be valuable? For these papers the research aim seems most closely linked to basic research concerned with theory development. The research problem involves an IT artefact, which is likely to be the trigger for or central to a significant change in the organization. A core issue for the problem is the social and political context and in particular the key role that individuals can play in the successful exploitation of an IT artefact.

The research aim is most often that of understanding these complex factors affecting or affected by the IT artefact.

4. The research process 4.1 Practical problems The main practical problems associated with ethnography include time, access to the case site(s), the potentially ambiguous role and/or isolation of the participant observer (the ethnographer) and ethical issues (Myers, 1999; Schultze, 2000; Fetterman, 2011; Harvey and Myers, 1995).

Access is discussed in all five papers. Myers and Young (1997) draw on existing material from a research project carried out in previous years and use this to reinterpret the political role played by the new MIS. McBride (2008) draws on his lifetime’s personal experience of working in software quality and his ongoing work as a performance ethnographer. Miscione (2007) joined the pre existing RISS Development project in Peru and developed his research study using access provided by this project. Pollock and Williams (2009) explain how the research came out of other work being carried out by one author. Only Schultze (2000) has to search for a site, but even she starts with the IT artefact of interest and locates very quickly a company that is initiating the process of installation and hence her research is of relevance to their situation.

The description of the role of the participant observer varied widely between papers. Only Schultze (2000) describes her role in detail. She was in the field (the US company) for 10 months and worked most closely with three groups of knowledge workers. She discusses her contribution to the work carried out by these three groups and how her work relationships developed and varied between the three groups of knowledge workers. Full acceptance took time but does not seem to have been a problem for at least two of the groups of Knowledge workers. Pollock and Williams (2009) research required them to be in a position to eavesdrop on the activities of representatives of the two key groups – industry analysts and IT manager’s when involved in the activity of evaluating IT vendor systems. They explain how one of them came to be in the position to observe one particular IT manager and his interaction with one Industry analyst in particular their email exchanges. The role seems to be more observer than participant observer and hence the role issue does not appear to have been significant. Miscione (2007) worked with the RISS project team for over a year but spent no more than 2 months in the field collecting data mostly through interviews with the various groups of healthcare workers who used the telemedicine system. He does not discuss his role as participant observer except to note that a significant part was through virtual participation in the online groups. It was also implied that his contribution as a RISS project team member qualified as participant observation. There was no discussion of the potential problems. The paper by Myers and Young (1997) offers no information on who took the role of participant observer or what if any problems were encountered by the participant observer during the period of the development of the new MIS. It is possible that this aspect had been discussed in detail in a

60

Ann Brown previous paper on the original research study but we are given no information on this. McBride’s (2008) study was focused on himself and hence his personal role was the only significant aspect. None of the papers raise ethical issues.

4.2 Research process The period during which the empirical work is carried out constitutes a major (often the largest) part of the project in terms of elapsed time and project work. The activities in the research process include decisions as to: what pre‐existing theory to apply, what type and amount of data to collect, the range of analytical tools to use, approaches to analysis and interpretation and type of reporting to adopt. The inclusion of participant observation poses problems of assessment that differ from all other research methods.

All five papers make extensive use of theory in formulating the research question and directing the data collection and choice of participant observation and finally in writing up. But the way in which theory is utilised varies considerably. Myers and Young (1997) make intensive use of theory. In their paper Habermas model of societal development drives the research question and is explicitly used in interpretation of the data and the writing up of the case. Whereas for Schultze (2000), Miscione (2007) and Pollock and Williams (2009) theory drove the research aim and hence the direction of data collection, but the type of data collected determined the analysis and interpretation. McBride (2008) is the least dependent on theory.

Participant observation presents the researcher with unique challenges (Evered and Louis, 1981). These include: finding a role acceptable to the social group or organization, accurately assessing the effect that the presence of the investigator has on the informants, and managing the analysis and reporting of the findings, so as to gain the insight without losing sight of the need for some degree of rigour and objectivity. Participant observation carries with it the concern that the presence of the investigator may influence the way informants behave. Informants may be suspicious of the researcher and reluctant to participate or be eager to please etc.

Personal relationship between researcher and informants may also influence the interaction (e.g. the researcher may empathise with his/her informants and vice versa). The informant may mislead the researcher in many ways and for many reasons, from self‐preservation to peer protection, ignorance, etc. (Van Maanen, 1979). Schultze (2000) goes into great detail on many aspects of her 10 month work in the field. She discusses her relationships with co‐workers but even she fails to address the concerns raised by Van Maanen (1979) as to bias. For the other papers (apart from Myers and Young, 1997) the participant observer role is described more briefly. The various ways, described in the papers, in which access (and hence work roles) was obtained suggests that that the ethnographer within organisations may have limited power to control these problems.

The analysis and interpretation stage pose the greatest challenge. The data collected tends to be predominantly qualitative, relatively unstructured, and, as is often the case with qualitative research, the sheer volume of data (potentially far more than that produced in case studies) can be overwhelming and difficult to manage (Miles and Huberman, 1984). There are no clear paths. All five papers noted the importance of the specialist knowledge of the IT artefact, displayed by the ethnographers. Each paper gave detailed information on how the analysis was carried out. Each research project used analysis tools dictated by the context and problems to make use of the data collected. Hence each was distinct and differed from any other paper.

Writing a case study is a process of selecting, ordering, editing, shaping the case material into a narrative which makes sense and serves a purpose – to describe, explore or explain the phenomenon, depending on the aim of the research. No two studies are the same, and ethnographers have experimented with different forms of writing. Van Maanen’s ‘Tales of the Field’ (1988) is generally regarded as the definitive work on ethnographic writing. The author identifies three narrative forms, namely, realist tales, confessional tales and impressionist tales, reflecting the choices the researcher as writer needs to make with regards to how to present the findings, what to include or omit, and what perspective to adopt, the researcher’s or the actors’.

Thus, realistic tales provide a direct, factual representation of the phenomenon under study; confessional tales reflect the researcher’s personal experience and perspective, and impressionist tales are dramatic reconstructions of fieldwork stories. For the writing up, two papers stand out for their use of the personal voice – Schultze (2000) and McBride(2008). Both adopt the method of telling us a story that is shaped by their personal experience and approach. Miscione (2007) adopts the realistic approach giving us for example a great deal of factual information on actual electronic exchanges. Pollock and Williams (2009) also adopt a realistic

61

Ann Brown approach. Whereas Myers and Young (1997) produce a case description that follows traditional descriptive methods told in the 3rd person, but heavily affected by the Habbermasian theory applied. Although this paper refers to the ethnographic research work on which it is based, the paper itself seems to make use of the results in a way more consistent with case methods. As for choice of analytical tools, each paper is written in a form that appears best suited for the topic and author(s) concerned.

5. Evaluation Klein and Myers (1999) call on authors to carry out critical evaluation of their own research work. They also suggest that we, the readers, should have enough information to critically evaluate the work described in academic papers. But convincing readers, that the work reported has met agreed standards of rigour and produced valuable results, of relevance to them, presents a complex problem for ethnographers.

Traditional research methods (of for example positivist science using quantitative tools) have well understood criteria by which the rigour of application can be assessed and the limitations of the degree of generalisability of the research results established. Researcher objectivity is a key aim for these type of methods. The research results convince through their adherence to accepted standards and practices that other researchers can follow and replicate. Ethnographers deal with social systems that contain as many realities as there are members and moreover evolve over time. There is a less clear cut pathway for the steps of the research process. Researcher objectivity is inappropriate ‐ indeed it is in the ethnographer’s personal vision and individual perceptions that the strength of the work is likely to lie. How do we the readers decide that a valid, valuable version of the social world has been created and in the case of Information Systems research workable guides developed for the effective exploitation of IT artefacts?

The most commonly cited equivalent approach for evaluating ethnographic research is by Golden‐Biddle and Locke (1993). According to these authors, ethnographic writing must convince readers on two levels – that the researcher has carried out a credible piece of research (the steps of the project have been carried out well) and that the findings and their interpretation appear credible and relevant to the reader. In contrast to the traditional scientific method, ethnographers reports depend critically on our, the reader’s belief in their authority as researchers. To achieve this authority, Golden‐Biddle and Locke (1993) propose three qualities that reports should seek to achieve ‐ authenticity, plausibility and criticality. Authenticity assesses the way in which the research was carried out – the quality of the work with respect to the specific characteristics of the situation studied. Plausibility deals with the reader’s likely interpretation – what is the readers, response to, and understanding of, the research findings. Hence identifying the community of readers becomes a key issue when writing the report. Criticality is about the results of the work and its likely impact on the community of readers. This quality is determined by the ability of the report to upset this community’s existing assumptions.

Authenticity focuses on the role of the ethnographer. How well does the text establish that the ethnographer has indeed been in the field for a period of time and at a level of immersion that would enable a firsthand experience of the social world under investigation to be obtained? Does the text convince the reader that the researcher has truly grasped the key aspects of this social world – is it authentic to the member’s perceptions?

Do we the readers believe that the researcher has ‘grasped and understood the members’ world … according to the members’ constructions of it’ (Golden‐Biddle and Locke, 1993)

Plausibility focuses on the quality of story told, the findings and their interpretation. This is where the reader’s experiences, discipline background and concerns come into play. The story has to make sense to the reader. It has to be familiar, relevant and distinctive and add to the existing body of knowledge in the field (Myers, 1999). This presents a special challenge. The work must speak to the community of readers (normally assumed to be the community of researchers in the same field) in terms that they understand. Hence it needs to use the language understood by this community and connect to the current theories and accepted ‘disciplinary knowledge’ of the community. But it must also offer a distinctive research contribution in that it offers a new insight into the phenomenon under investigation.

Criticality focuses on the impact of the work on the reader. Does the text contain thought‐provoking insights which stimulate the reader to question their assumptions and consider new possibilities? According to Golden‐ Biddle and Locke (1993) a text achieves criticality ‘ by disrupting readers’ common sense and enabling them to step back and to reflect on and question the personal assumptions underlying their work’.

62

Ann Brown The importance of the individual insight of the ethnographer in managing the project, choosing analytical tools and writing up suggests an additional criterion by which to assess the work – the depth and thoroughness of the report and transparency of the research notes. The inclusion of information about the ethnographer’s prior experiences, attitudes and beliefs as well as the pathway taken through the research study and the reasons for the decisions taken would offer the chance for other researchers to revisit the research work and perhaps find other interpretations of value.

Only Shultze’s (2000) paper meets the call for self assessment. She offers an in depth evaluation of her own work using the Biddle and Locke (1993) criteria. None of the remaining papers include any form of self evaluation of the ethnographic work.

For us the readers do we have confidence in the results (authenticity)? Do the results seem plausible?

Shultze’s (2000) explains at length the steps taken in the research. For the community of researchers in Knowledge Work she offers an impressive level of detail. This inevitably means that the paper is long.

Miscione (2007) gives an excellent description of the research process and how he reaches his conclusions. The findings would seem to resonate with health workers. Pollock and Williams(2009) also give a good description of the research process. Much of the paper describes the various groups of actors involved in the process of IT procurement. The level of detail impresses us the readers as to the plausibility of the activities described.

McBride (2008) applies performance ethnography which naturally focuses on himself and his experience of software development. The research process is described well as it is interwoven with the final presentation.

The findings certainly speak to my experience of software development, and would be plausible for any IS developers. Myers and Young (1997) make little attempt to describe the research process. Most of the paper is taken up with the theory that they were applying (Habbermas’ model of societal development). But the experiences of the project group are clearly written in a form that we would all recognise. They are plausible.

Do the results reveal something new – perhaps unexpected? All five papers revealed interesting and to some extent unexpected results. McBride (2008) uses the unusual performance techniques to stunning effect to describe both the research process and present the results. The results are counter the received view about how to achieve quality software. Miscione (2007) presents his work extremely effectively leaving us with a deep understanding of how the various stakeholders react to and use the new Telemedecine system. These reactions are not what had been planned for or expected when the new system was designed and installed.

Schultze is less concise but also leaves us with a deep understanding of knowledge work and in no doubt of ‘subjectivity and objectivity as a duality in knowledge work’ (Schultze ,p36, 2000). This is in opposition to the main stream of research with respect to Knowledge Work, which aims to create a structured approach to knowledge creation and dissemination – seeking to excise subjectivity. Pollock and Williams(2009) leave us with an understanding of the problems of IT procurement (for the buyers) but a profound scepticism of the role played by industry analysts. Myers and Young (1997) demonstrate that the Habbermas model can be applied to a part of the project operation. The conclusions but support the significance of social factors for the successful exploitation of information systems whish at the time was not the main stream view of practitioners or researchers.

How far does the individual voice emerge? Clearly for four of the papers this is explicitly referred to. McBride (2008), Schultze (2000) and Miscione (2007) all write up their work referring to themselves personally and take time to explain how their individual knowledge, attitudes and approach are used by them in the research process. Pollock and Williams(2009) refer to the ethnographer as one of the authors (NP) and do explain how his personal relationships helped the research, but perhaps offer less on how it shaped the interpretation and analysis. Myers and Young (1997) offer little on the way their individual experiences affected the research and findings.

6. Conclusions All five papers are impressive pieces of work. They address important practical problems of immense significance to the IS community both practitioner and academic. The results obtained are unexpected and challenging. They show that the type of research question that IS ethnographers investigate display a number of characteristics. An IT artefact will be at the heart of such investigations and it is the response of the people affected by the artefact that is a key element. The artefact may be a new IS or a concept but it is often a trigger for social and organizational change, hence the research aim is most likely to include developing or improving

63

Ann Brown the researchers understanding of this effect. Access to suitable sites is so important that most work seems to flow from access provided through other research projects. An unexpected element is the intensive use of theory in all papers, in setting the research direction, the data collection and in particular helping to frame the way the analysis is carried out. The IS ethnographer will need to have or acquire a good knowledge of the IT artefact which is at the core of the research.

The principle weaknesses displayed by these papers is the description of the participant observer role and evaluation of the work carried out. Despite the key role played by participant observation in ethnographic work, they tended to omit much discussion of the ambiguities and problems of carrying out such work (apart from Schultze, 2000).

To meet the requirement to be able to convince an academic audience, it is likely that most papers adopting this method will be lengthy, so as to include both the details of the problem, the IT artefact, the social situation, the relevant theory, the appropriate amount of detail on the research steps and a serious evaluation of the work.

References Evered, R. and Reis Louis, M. (1981) ‘Alternative Perspectives in the Organizational Sciences: ‘Inquiry from the Inside’ And ‘Inquiry from the Outside’’ in Academy of Management Review, Vol. 6, No. 3, pp. 385‐395. rd Fetterman D (2010) ‘Ethnography step by step’ (3 edition) Applied Social Research Methods Series volume 17 Sage Publications Golden‐Biddle K and Locke K (1993) ‘Appealing Work: An Investigation of how Ethnographic texts Convince’ Organization Science vol4 no 4 Harvey L and Myers M (1995) ‘Scholarship and practice the contribution of ethnographic research methods to bridging the gap’ Information technology & people vol 8 (3) pp13‐27 Klein, H. K. and Myers M (1999) ‘A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information’, MIS Quarterly, Special Issue on Intensive Research (23:1), pp. 67‐93.

Leidner D and Kayworth T (2006) ‘A review of Culture in Information Systems Research: Towards a Theory of Information Technology Culture Conflict’ MIS Quarterly Vol 30 (2) , pp. 357‐399 McBride N (2008) ‘Using Performance ethnography to explore the human aspects of software quality’ Information Technology & People vol 21 (1) pp91‐111 Miles, M.B. and Huberman, A.M. (1994) Qualitative Data Analysis London: Sage Mingers J (2003) ‘The Paucity of multimethod research: a review of the information systems literature’ Information Systems Journal vol 13 pp233‐249 Miscione, G (2007) ‘Telemedecine in the Upper Amazon: Interplay with Local health Care Practices’ MIS Quarterly vol 31 (2) pp403‐425 Myers M (1999)’ Investigating Information Systems with ethnographic Research’ Communications of The Association for Information Systems vol 2 article 23 Myers M and Young L (1997) ‘Hidden Agendas, power and managerial assumptions in information systems development; An ethnographic study’ Information Technology & People vol 10 (3) pp224‐240 Orlokowski W J and Baroudi JJ (1991) "Studying Information Technology in Organizations: Research Approaches and Assumptions", Information Systems Research (2) 1991, pp. 1‐28 Pollock N and Williams R (2009) ‘The sociology of a market analysis tool: How industry analysts sort vendors and organize markets’ Information and Organization vol 19 p129‐151 Sanday PR (1979) ‘The Ethnographic Paradigm(s)’ Administrative Science Quarterly vol 24 pp527‐538 Schultze U (2000) ‘A confessional Account of an Ethnography about Knowledge Work’ MIS Quarterly vol 24 (1) pp3‐41 Tan M. and Hall W (2007) ‘Beyond the Theoretical and Methodological Pluralism in Interpretive IS Research: The example of Symbolic Interactionist Ethnography’ Communications of The Association for Information Systems vol 19 pp589‐ 610 Van Maanan (1988) ‘Tales of the Field: On Writing Ethnography’ University of Chicago Press Van Maanan (1979) ‘The Fact or Fiction in Organizational Ethnography’ Administrative Science Quarterly vol 24 pp 539‐550 Yin R (1994, 2003) ‘Case Study Research Design and Methods’ second and third edition Applied Social Research Methods Series, Volume 5, Sage Publications.

64

Applying the GRAMMS to a Community Engaged Mixed Methods Study of Skilled Migrants in Regional Australia Roslyn Cameron1, Trudy Dwyer2, Scott Richardson1, Ezaz Ahmed1 and Aswini Sukumaran1 1 School of Business and Law,Central Queensland University, Australia 2 School of Nursing and Midwifery,Central Queensland University, Australia r.cameron@cqu.edu.au

Abstract: The purpose of this paper is to apply the Good Reporting of a Mixed Methods Study (GRAMMS) framework to an exploratory mixed methods research study which has investigated the contributions made to regional Australia by skilled migrants and their families. The study aimed to explore the economic, social and cultural contributions made by skilled migrants and their families to Australian society and in particular to the sustainability of Australia’s regional areas. There is no one national data collection mechanism which records the whereabouts and activities of skilled migrants once they enter an Australian port. Due to this lack of data the study needed to employ innovative ways in which to build data sets that would contribute to parts of the larger picture and to thereby attempt to answer the research questions posited. The paper discusses the methodological choices made during the research through the application of the GRAMMS framework (O’Cathain, Murphy & Nicholl 2008) in a process of reflexivity to prepare for more sophisticated data analysis, integration and interpretation for future academic publication. The study employed an exploratory mixed methods research design over three sequential phases and involved a combination of both qualitative and quantitative data combinations throughout the three phases. The paper will present the research design and address the use of applying a mixed method research design by reflexively applying the GRAMMS framework for evaluating a mixed methods research study. The paper will discuss each of the six points of the GRAMMS framework as a means for presenting the rationale for the methodological choices made for using a mixed methods research design. In doing so, the paper addresses a common criticism of reported mixed methods research studies which is usually aimed at: a lack of explicitly presenting a rationale for utilising mixed methods and providing arguments for the methodological choices made throughout a mixed methods study (data collection methods, sequencing, sampling, priority of data, points of integration and data analysis techniques utilised). The paper aims to progress the study beyond the research product produced for its community based constituent audience to a more theoretical academic audience. The value of the paper lies in its explicit attention to the GRAMMS framework and offers an exemplar to other mixed methods researchers on how to best address some of the common criticisms put forward about reported mixed methods studies.

Keywords: mixed methods research, GRAMMS, exploratory mixed methods research design, skilled migrants, extended mixed methods notation system, data transformation

1. Introduction This paper applies the good reporting of a mixed methods study (GRAMMS) framework developed by O’Cathain, Murphy and Nicholl (2008) to a study which has been recently completed on the contributions made to regional Australia by skilled migrants and their families. The study was internally funded and was guided by a Community Advisory Committee (CAC) made up of representatives from employer, government and community groups. To this end the study was an exemplar of community engaged research. The study was conducted by a multidisciplinary research team in conjunction with the CAC over a nine month period in 2012.

A report was written and published at the end of the project with the report’s prime audience being members of the regional community in which the research was undertaken. The report was written specifically for this audience in mind and academic publications are now planned from the study and its findings. To this end more sophisticated analysis and data integration will be undertaken. The process of applying the GRAMMS will assist the research team in a process of reflexivity and will enable the team to plan this next stage of reporting for academic publication. Sandelowski and Barroso (2003) refer to a typology of qualitative research findings.

“Findings in journal reports of qualitative studies in the health domain can be classified on a continuum of data transformation” (Sandelowski & Barroso 2003, p. 905) which, indicates the degree of data transformation along the continuum. The degree of data transformation of the data reported in the community report for this study is much lower on the continuum of data transformation described by Sandelowski and Barroso (2003) than is anticipated for future academic publication. The process of applying the GRAMMS will assist us to progress the data analysis along this continuum of data transformation.

A major finding from the study was there is no one national data collection mechanism which records the whereabouts and activities of skilled migrants once they enter an Australian port. Due to this lack of data the study needed to employ innovative ways in which to build data sets that would contribute to parts of the

65

Roslyn Cameron et al. larger picture and to thereby attempt to answer the research questions posited. The paper discusses the methodological choices made during the research through the application of the good reporting of a mixed methods study (GRAMMS) framework (O’Cathain, Murphy & Nicholl 2008).

The paper will present an overview of the study (research aims, objectives, research questions and research design) before introducing a discussion of Haneef’s (2013) flow of empirical research products and content and Sandelowski and Barroso’s (2003) typology of qualitative findings, to position the current state of the analysis of the study’s findings within the research process and it’s products. This will be followed by the application of the GRAMMS framework to the study as a process of reflexivity for preparing the study’s data (qualitative, quantitative and integrated) for academic publication. Additional tools and frameworks will also be utilised to assist address the GRAMMS for future academic audiences. These include the Extended MMR Notation System (Cameron 2012) and Newman et al’s (2003) Typology of Research Purposes.

2. Research process and products This section of the paper introduces two conceptual frameworks (Haneef’s (2013) flow of empirical research products and content and Sandelowski and Barroso’s (2003) typology of qualitative findings) to assist us position the current status of the study within the boarder ‘progressive development of research work products and report content’ (Haneef 2013, p. 388).

Haneef (2013) refers to the breakdown of research report elements which is depicted in Figure 2, as a simplified and notional version of empirical research. Empirical content includes the data collected and the findings which have been developed from it. The pre‐empirical content includes ‘material developed prior to data collection, such as theoretical frameworks, motivation, research questions, context, and methods. Post‐ Empirical Content includes material that is based on the findings as well as the experience from the research activity, such as generated theory, implications, discussion, and recommendations’ (Haneef, 2013, p. 388).

Pre-Empirical Content (Questions, methods, theoretical frames etc)

Empirical Content Data

Findings

Post-Empirical Content (Recommendatio ns, Theory etc)

Continuum of data transformation Source: Adapted from and Haneef (2013) and Sandelowski & Barroso (2003) Figure 1: Flow of empirical research work products and report content Another conceptual framework which has been helpful in assisting us to position the current study is Sandelowski and Barroso’s (2003) typology of qualitative findings depicted in Figure 2. Sandelowski and Barroso (2003) developed this typology to explain the differences between qualitative findings specifically in health research, ‘What our typology does do is place findings on a continuum indicating degree of transformation of data, from findings that remain very close to data as given (e.g., a summary of informational contents from a manifest content analysis of data) to findings representing many transformative moves away from data (e.g., a phenomenology of self‐transcendence)’ (Sandelowski and Barroso (2003, p. 907‐8). Figure 2 shows the categories in the typology on a continuum indicating their interpretive distance from data. A reference to the continuum of data transformation from this typology is mentioned in Figure 1.

The Sandelowski and Barroso (2003) typology/continuum assists us in positioning the study at its current state of data interpretation, or more specifically at the point directly following the production of a research product aimed at a particular community based audience. This report from the study was published in November 2012 primarily for the Gladstone community (Cameron et al 2012). The study is now at a point within the research

66

Roslyn Cameron et al. process where more analytical data analysis and transformation is required to enable the research team to disseminate the results of the research to an academic audience through academic publication.

Closest to data No findings

Not research

Farthest from data Thematic Survey

Conceptual/ Thematic description

Exploratory

Descriptive Qualitative

Topical survey

Not qualitative research

Interpretive explanation

Explanatory

Borderline studies

Source: Sandelowski and Barroso (2003, p. 908).

Figure 2: Continuum of qualitative findings

Sandelowski and Barroso (2003) see a key feature of the typology as being: ... its emphasis not on what authors named their analytic and interpretive methods but rather on the contents and form of what they presented as the findings in their reports of the studies they conducted....A focus on the findings themselves reveals the kind of analytic and interpretive work actually performed, no matter what the research rhetoric.... Such factors as the presence, amount, and absence of verbal text, metaphoric language, quotes, numbers (e.g., frequencies, means, percentages), tables, figures, and other textual and visual displays revealed more about the methodological orientation of a study than any statements of method in the method sections of reports (Sandelowski and Barroso (2003, p. 909).

We position the research report presented to the Gladstone community (Cameron et al 2012) as the community based constituent audience research product from the study. The intended audience should be indicated in research reporting whether these be practitioners, policy makers, funding agencies, community groups, general public, specific research communities, academic researchers etc. The Cameron et al (2012) report is a research product intended for a particular audience with certain levels of data analysis and transformation best suited to the consumption and use of that audience. We position this report on the continuum at the exploratory stage and aim to progress the data transformation to the descriptive stage of the continuum.

3. The mixed methods study: Case study of skilled migrants in regional Australia The paper will now introduce the study.

Aim and purpose of study:

The overall aim of this study was to investigate the contributions of skilled migrants and their families to regional business and communities and to identify the factors which attract and retain skilled migrants in key workforces to regional areas. The research questions driving the study are listed below:

RQ1: What economic, social and cultural contributions do skilled migrants and their families make to regional business and communities?

RQ2: What are the issues encountered by business and communities when employing skilled migrants?

RQ3: What support does business and the community need to provide to ensure they attract and retain skilled migrants and their families in regional areas?

67

Roslyn Cameron et al.

RQ4: What support do skilled migrants and their families need to better assist them settle, become active members of communities and remain in regions?

RQ5: What are the implications of the study findings for policy and practice?

Research design:

The research design employed for this proposed research was a sequential exploratory mixed methods design utilising both qualitative and quantitative data collection methods across three sequential research phases. A sequential exploratory design utilises the first method (qualitative) to assist develop and inform the second method (quantitative) (Greene, 2007). This design is well suited to exploring a phenomenon (Plano Clark and Creswell, 2008) and is useful when the researchers need to develop an instrument because one is not available (Creswell, 2009). Creswell and Plano Clark (2007) describe this design as follows: In this design, the researcher first qualitatively explores the research topic with a few participants. The qualitative findings then guide the development of items and scales for a quantitative survey instrument. In the second data collection stage, the researcher implements and validates the instrument quantitatively. In this design the qualitative and quantitative are connected through the development of the instrument items. Researchers using this variant often emphasize the quantitative aspect of the study (Creswell and Plano Clark 2007, p. 77).

This study deviated from this description in two respects. Firstly, there was three phases in the research and two quantitative instruments were developed (Professional Workforce Survey and the Community Migrant Survey). The Professional Workforce survey was disseminated at the end of the first phase and the second survey (Community Migrant Survey) was distributed at the end of the third phase.

Phase 1 of the research involved a scan of Health Services in the region (QUAL), semi‐ structured interviews with HR managers (QUAL) and the development and distribution of a Professional Workforce Survey (quan).

Phase 2 involved the collection and analysis of an array of secondary data sets (QUAN) and a series of five case studies (QUAL) of small to medium sized enterprises that had used skilled migration. The third Phase involved interviews (QUAL) with community leaders and members of various ethnic communities (Indian, South African, Filipino and Jamacian) within the Gladstone region followed by the development and distribution of a Community Migrant Survey (QUAN). Figure 3 provides a visual depiction of the overall research design which has been adapted slightly from that presented in Cameron et al (2012).

Phase 1: QUAL + quan

Phase 2: QUAN + QUAL

Primary data:

Secondary data sets:

Health services scan HR Manager semistructured interviews (n=7) Professional Workforce Survey (n=26)

Phase 3: QUAL + QUAN

•

Census & DIAC Settlement data

Primary data:

•

GRC-citizenship

•

GAPDL-RSMS

•

DETE-public schools

Semi-structured interviews with community leaders and migrant groups (n=8)

Primary data: SME Case Studies (n=5)

Community Migrant Survey (n=73)

Source: Adapted from Cameron et al (2012, p. 37) Figure 3: Summary of research design and phases To a large degree the research was emergent and difficulties faced in achieving access to data (for example, professionals for the Professional Workforce Survey in Phase 1) meant that decisions were made during the first and second phases to attempt to gain access to other data sources not planned at the initial stages. As a means to supplement the relatively failed attempt to obtain a statistically significant number of professional

68

Roslyn Cameron et al.

Workforce Surveys, the research team in consultation with the CAC, opted for a series of case studies in Phase 2 and the development of a Community Migrant Survey in Phase 3.

Research Team and Community Advisory Committee:

The research team was made up of academics from a regional university across three regional campuses and from three different discipline areas: human resource management, nursing and midwifery and tourism. The research team worked very closely with the Community Advisory Committee (CAC) who were made up representatives from the community and who met regularly with the research team to advise, guide and provide access to secondary and primary data sources. The study was an exemplar of community engaged research. The organisations represented in the CAC were as follows:

Gladstone Economic and Industry Development Board (GEIDB)

Department of Education Training & Employment (DETE)

Gladstone Industry Leadership Group (GILG)

Gladstone Area Promotions and Development Ltd (GAPDL) ‐ Regional Certifying Body for Regional Skilled Migration Scheme

Gladstone Regional Council (GRC)

Gladstone Regional Hospital

Gladstone Multicultural Association Inc (GMAI)

Welcoming Intercultural Neighbours (WIN)

Yarralla Sports Club

As can be seen from the composition of the CAC and the community engaged research approach taken, it was important the first research product from the research be targeted to the intended community based audience.

The following section of the paper introduces the GRAMMS and the intent to behind applying the GRAMMS to the study.

4. GRAMMS framework O’Cathain, Murphy and Nicholl (2008) undertook an analysis of mixed methods studies in the health services research published between 1994 to 2004. The aim of this study was to assess the quality of these mixed methods studies. The study found that the main quality issue was a “lack of transparency of the mixed methods aspects of the studies and the individual components. The qualitative components were more likely to be poorly described than the quantitative ones” (O’Cathain, Murphy and Nicholl 2008, pp. 96‐97). In terms of attempts to integrate the qualitative and quantitative data there were few if any attempts to do so as there was a “tendency for researchers to keep the qualitative and quantitative components separate rather than attempt to integrate data or findings in reports or publications” (O’Cathain, Murphy and Nicholl 2008, pp. 97).

The study produced the Good Reporting of A Mixed Methods Study (GRAMMS) framework as a means to guide those reporting mixed methods studies and to encourage quality reporting. This six‐item guidance framework

includes prompts about the ‘success of the study, the mixed methods design, the individual qualitative and

quantitative components, the integration between methods and the inferences drawn from completed studies’ (O’Cathain et al. 2008, p. 92). The GRAMMS includes the following set of quality guidelines:

1. Describe the justification for using a mixed methods approach to the research question 2. Describe the design in terms of the purpose, priority and sequence of methods 3. Describe each method in terms of sampling, data collection and analysis 4. Describe where integration has occurred, how it has occurred and who has participated in it 5. Describe any limitation of one method associated with the presence of the other method 6. Describe any insights gained from mixing or integrating methods (O’Cathain et al. 2008, p. 97).

69

Roslyn Cameron et al.

The paper will know engage in a process of reflexivity by applying the GRAMMS as a means to position the next stage of the data analysis and integration for the purpose of producing research products for future

publication aimed at academic audiences. Each of the six steps of the GRAMMs will be systematically applied to the study.

5. The reflexive process: GRAMMS The following section of the paper reflexively addresses each of the six key prompts from the GRAMMS.

1. Describe the justification for using a mixed methods approach to the research question.

Researchers demonstrate and prove validity when they “can show the consistency among the research purposes, the questions, and the methods they use” (Newman, Ridenour, Newman and DeMarco 2003, p.167).

There needs to be methodological logic between the key aspects of the research. The justification of the use of a mixed methods research design was not fully explained or explored in the original research report to the community suffice to say that it was used to address a complex problem:

Mixed methods ‘are used when the phenomena being studied is considered complex and beyond the reach of a single method’ (Morse & Niehaus 2009, p.15). Mixed methods research provides opportunities for presenting a greater diversity of divergent views and can provide stronger inferences (Teddlie & Tashakkori 2003, p. 15) (Cameron et al 2012, p. 36).

The next iteration of the reporting of the study and its findings will contain a full justification for using the overall mixed methods research design and will also explicitly address each data collection method within each Phase. This discussion will draw upon: the lack of national, state or regional data collection mechanisms, the complex nature of the phenomenon being investigated, the lack of a substantial research base to draw from (as identified in the review of literature), the difficulties gaining access to data sources in industries experiencing high pressures related to world commodity prices, mergers and acquisitions and re‐ structuring and limited resources and timeframes.

2. Describe the design in terms of the purpose, priority and sequence of methods

The purpose was not made as explicit as it could have been in the report to the community. This will be addressed by drawing upon the literature within the MMR community which refers to purposes for utilising mixed methods research. To this end the research that will be reported for academic audiences will apply Newman et al’s (2003, p. 175) typology of research purposes which contains nine purposes:

Predict

Add to the knowledge base.

Have a personal, social, institutional, and/or organisational impact.

Understand complex phenomena.

Test new ideas.

Generate new ideas.

Inform constituencies.

Examine the past.

The research, as it was reported to the Gladstone community in late 2012, had a key purpose of informing constituents of the wider community. Newman et al (2003) have provided more specific purposes under each of these nine categories. For the category of “Inform constituencies”, a further seven more specific purposes are tabled:

Inform constituencies.

Inform the public.

Heighten awareness.

70

Roslyn Cameron et al.

Public relations.

Enlighten.

Hear from those who are affected by treatment/program.

Describe the present.

Comply with authority

(Newman et al 2003, p.177).

Of these all except the last (Comply with authority) were directly related to the purpose and intention of the research report published in November 2012. Other research purposes that will be tabled in future reporting of the research will include “Add to the knowledge base”, “Understanding a complex phenomenon” and “Measure change‐ measure the consequences of practice”. These purposes will be prioritised and discussed in much greater detail.

The priority and sequence of the methods was influenced greatly by the timeframe, funding and contextual issues (access to data sources) and to a large extent was emergent. The priority and sequence of methods was described and documented in the Methodology section of the community report with a figure similar to that of Figure 1 in this paper (see Figure 6 in Cameron et al 2012, p. 37).

3. Describe each method in terms of sampling, data collection and analysis

One of the best ways to describe the sampling, data collection and analysis in a mixed methods study is to utilise the newly developed Extended MMR Notation System developed by Cameron (2012). The explanation and visual depiction of the study for future academic publication will be enhanced through the application of this system. This will provide greater clarity in terms of the purpose, priority and sequence of the methods across the three phases. An example of this is presented in Figure 4 which depicts the Extended Notation System for Mixed Methods Research applied to Phase 1 of the research.

Phase 1: QUAL + quan

Phase 2: QUAN + QUAL

Phase 3: QUAL + QUAN

Health Services Scan HR Manager semistructured interviews Professional Workforce Survey Health Service Scan DS: Primary(1) S-SIZE: (n=150) INST: QL ANSIS: QL ÆQT Interviews DS: Primary(2) S-SIZE: (n=7) INST: QL ANSIS: QL-1 Survey DS: Primary(3) S-SIZE: (n=26) INST: qt ANSIS: qt-1

The Extended MMR Notation System will also be included for Phases 2 & 3

Figure 4: Applying the extended mixed methods notation system to phase 1

71

Roslyn Cameron et al.

The Extended MMR Notation System summarises the discussion and description that needs to be detailed in the body of the reporting (data sources/samples, sample sizes, data collection instruments and the analysis undertaken for each data collection methods). This will be detailed and expanded upon in the future reporting of the research for academic audiences.

4. Describe where integration has occurred, how it has occurred and who has participated in it

Data integration was not reported in the community report which was written and presented to the community in late 2012 (as per the project timelines). It contained basic sets of descriptive data and analysis.

For some data collection methods the samples were too small to be of statistical significance and so planned analysis could not be undertaken. More sophisticated data analysis is now occurring in preparation for academic audience publication. The writing of this paper is part of that data transformation process and will be influential in determining what data integration will be analysed and reported. The use of the Extended Notation System for Mixed Methods research mentioned above will assist in this process as the system notates data sources (primary and secondary), sample sizes, data collection instruments and data analysis techniques for each data collection method. This will be particularly useful when documenting data integration where various configurations of data integration is presented to address the research questions. An example of this will be in addressing the following aspects of RQ1:

RQ1: What economic, social and cultural contributions do skilled migrants and their families make to regional business and communities?

RQ1 Economic contributions

Social Contributions

Cultural Contributions

Data Phase 1: Health Services Scan, Interviews with HR Managers, Aspects of Professional Workers Survey ( eg occupations and educational qualifications) Phase 2: Population capital data from secondary data sources, Case studies of SMEs Phase 3: Interviews with community leaders Phase 1: Aspects of Professional Workers Survey ( eg joining of community groups and volunteering) Phase 3: Interviews with community leaders and representatives from ethnic groups, Aspects of Community Migrant Survey Phase 1: Aspects of Professional Workers Survey Phase 2: Data from secondary data sources that demonstrate levels of cultural diversity within the region Phase 3: Interviews with community leaders and representatives from ethnic groups, Aspects of Community Migrant Survey

5. Describe any limitation of one method associated with the presence of the other method

Limitations in the original report for community audiences was addressed in terms of the limitations of the secondary data collected in Phase 2 of the study. The discussion on the limitations of the study will be more comprehensive for reporting the study to academic audiences and will flow on from the application of the Extended Mixed Methods Notation System to be applied when addressing item 3 of the GRAMMS.

The limitations of using the methods chosen with the presence of the other methods will be addressed in the reporting to academic audiences through a discussion of the limitations of each data collection method and within each of the three phases of the research.

6. Describe any insights gained from mixing or integrating methods

The mixing of methods was necessitated by the data void the team experienced in attempting to obtain reliable evidence as to the presence and numbers of skilled migrants and their families in a regional area in Australia. This in itself was a significant finding. Much greater insights and an array of perspectives were gained from the use of a three phased MMR design as could not have been obtained from single (monomethod) or even multi method research designs. These methodological approaches would not have been able to address the data void or collect an array of perspectives (multiple data sources) or obtain the richness of data required to address the research questions posited. The combination of both secondary (QUAN) and primary data (QUAN and QUAL) collected from a variety of data sources within the community enabled the study to put

72

Roslyn Cameron et al. together some of the pieces of the puzzle to what is a complex phenomenon. As a result, snippets of data, from across a range of data sources within the community was collected given the limitations in respect to resources and timeframes.

6. Conclusion The purpose of this paper is to apply the Good Reporting of a Mixed Methods Study (GRAMMS) framework to an exploratory mixed methods research study which has investigated the contributions made to regional Australia by skilled migrants and their families. There is no one national data collection mechanism which records the whereabouts and activities of skilled migrants once they enter an Australian port. Due to this lack of data the study needed to employ innovative ways in which to build data sets that would contribute to parts of the larger picture and to thereby attempt to answer the research questions posited. The paper presented an overview of the study (research aims, objectives, research questions and research design) before introducing a discussion of Haneef’s (2013) flow of empirical research products and content and Sandelowski and Barroso’s (2003) typology of qualitative findings, to position the current state of the analysis of the study’s findings within the research process and it’s products. This was followed by the application of the GRAMMS framework to the study as a process of reflexivity for preparing the study’s data (qualitative, quantitative and integrated) for academic publication as opposed to community based constituent audience. Additional tools and frameworks were used to assist apply the GRAMMS in order to aid further data transformation in preparation for future reporting of the study and its findings to academic audiences. These include the Extended MMR Notation System (Cameron 2012) and the Typology of Research Purposes (Newman et al 2003).

The value of the paper lies in its explicit attention to the GRAMMS framework and offers an exemplar to other mixed methods researchers on how to best address some of the common criticisms put forward about reported mixed methods study and for dealing with appropriate reporting of mixed methods research for an array of audiences

References Cameron, R. (2012) “Applying the newly developed Extended Mixed Methods Research (MMR) Notation System”, British Academy of Management 2012 Conference, 11‐13th September, Cardiff, Wales, 2012.

Cameron, R, Dwyer, T, Richardson, S, Ahmed, E and Sukumaran, A. (2012) Skilled migrants and their families in regional Australia : A Gladstone case study, e‐book, CQUniversity Australia, Gladstone, Qld. Creswell, J.W. and Plano Clark, VL. (2007) Designing and Conducting Mixed Methods Research, Sage, Thousand Oaks, CA. Haneef, N. (2013) “Empirical research consolidation: a generic overview and a classification scheme for methods”, Quality and Quantity, Vol. 47, pp. 383‐410.

Newman, I., Ridenour, CS., Newman, C. and DeMarco, GM. (2003) “A typology of research purposes and its relationship to mixed methods”, in Tashakkori, Abbas. & Teddlie, Charles. Handbook of Mixed Methods in Social and Behavioral Research. Thousand Oaks, CA: SAGE, pp. 167‐ 188.

O’Cathain, A., Murphy, E., and Nicholl, J. (2008) “The quality of mixed methods studies in health services research”, Journal of Health Services Research and Policy, Vol. 13, No. 2, pp. 92‐98.

Sandelowski, M. and Barroso, J. (2003) “Classifying the findings in qualitative studies”, Qualitative Health Research, Vol. 13, No. 7, pp. 905‐923.

Tashakkori, A. and Teddlie, C. (Eds.). (2003) Handbook of Mixed Methods in Social & Behavioral Research, Sage, Thousand Oakes, CA. nd Tashakkori, A. and Teddlie, C. (Eds.) (2010) Handbook of Mixed Methods in Social & Behavioral Research, 2 Edition, Sage, Thousand Oakes, CA. Teddlie, C. and Tashakkori, A. (2003) “Major issues and controversies in the use of mixed methods in the social and behavioural sciences”, in Tashakkori, Abbas. & Teddlie, Charles. Handbook of Mixed Methods in Social and Behavioral Research. Thousand Oaks, CA: SAGE, pp. 3‐ 50.

Teddlie, C. and Tashakkori, A. (2010) “Overview of contemporary issues in mixed methods research”, in Tashakkori A and Teddlie C (Eds.) (2010) Handbook of Mixed Methods in Social & Behavioral Research, 2nd Edn, Sage, California, pp.1‐44.

73

A Methodological Scan of a National Industry Based Research Program for the Rail Industry 2007‐2014 Roslyn Cameron School of Business and Law, Central Queensland University, Gladstone Campus, Australia r.cameron@cqu.edu.au

Abstract: The purpose of this paper is to present the findings from a methodological scan of a national industry‐based research program which began in 2007 and will end in 2014. The program is a cooperative partnership between the Australian government and the Australasian rail industry. The Cooperative Research Centre (CRC) for Rail Innovation has expended $100 million AUD on 106 research projects across four program areas: Economic, Social and Environmental Sustainability, Operations and Safety, Engineering and Safety and, Workforce Development. The CRC for Rail Innovation involves the strategic participation of various key companies in the rail industry in Australasia as well as Australian universities. A key characteristic of CRCs is the close collaboration between industry and researchers: ‘The CRC Program links researchers with industry and government with a focus towards research application. The close interaction between researchers and the end users is the defining characteristic of the Program. Moreover, it allows end users to help plan the direction of the research as well as to monitor its progress’ (CRCA, n.d). The study utilised content analysis to analyse levels of methodological variety across the research projects and levels of industry and university collaboration throughout the life of the CRC for Rail Innovation research program. The methodological scan categorised each research project in terms of the number of data collection methods used (one method, two methods, three methods etc). As a result of this analysis the most frequent category was that of two data collection methods (30%) followed by those studies using 3 data collection methods (27%) and 4 data collection methods (12%). Literature reviews were common at 36.6 per cent and reflect the number of smaller scoping studies which were primarily reviews of the literature and the central role of literature reviews in pre‐empirical research. The study also categorised each research project as either one of the following: monomethod qualitative; monomethod quantitative; multimethod qualitative; multimethod quantitative or; mixed methods. The most dominant type of study was multimethod qualitative (42%), followed by an equal second for monomethod qualitative and mixed methods (28% each). There were no multimethod quantitative studies and one monomethod quantitative study. Other data collected looked at the composition of project steering committees to gauge the level of industry and university collaboration and engagement. The research findings will provide great value in evaluating the level of success with the current CRC model and provide insights into future models that could be adopted.

This is of particular importance to the CRC for Rail Innovation board as they are considering future models for collaborative research within the Australasian rail sector.

Keywords: methodological scan, rail industry, mixed methods, multimethods, monomethods, methodological diversity, applied industry research

1. Introduction The Cooperative Research Centres (CRC) program is an Australian Government Initiative administered by the Department of Industry, Innovation, Science, Research and Tertiary Education (DIISRTE). The CRC program supports end user driven research collaborations to address major challenges facing Australia. CRCs pursue solutions to these challenges that are innovative, of high impact and capable of being effectively deployed by the end users (CRC, n.d.). The CRC program began in 1991 and since that time there have been 14 CRC selection rounds. The selection rounds of the latest round closed in June 2012. As at August 2011 there were 38 current CRCs in operation. These are categorised across four Australian and New Zealand Standard Industrial Classification (ANZSIC) sectors: Agriculture, Forestry and Fishing (n=11); Services (n=24); Manufacturing (n=5); and Mining (n=4). The CRC for Rail Innovation is one of the 24 CRCs classified in the Services sector. At 30 June 2012, the Australian Government has committed more than $3.4 billion (AUD) to the CRC program.

The aims of the CRC program is to support ‘end user driven research partnerships between publicly funded researchers and end users to address clearly articulated, major challenges that require medium to long term collaborative efforts’ (Allen Consulting Group 2012, p. 1). In 2012 the Department of Industry, Innovation, Science, Research and Tertiary Education (DIISRTE) commissioned the Allen Consulting Group to evaluate the impacts of the CRC program. The study estimated the economic, environmental and social impacts of the CRC program from 1991 and projected out to 2017. This involved analysis of approximately 120 past and present CRCs. The study gathered data through surveys of current CRCs which were cross checked against reports and economic studies. The study estimated the following:

74

Roslyn Cameron Between 1991 and 2017 almost $14.5 billion of direct economic impacts are estimated to have accrued from CRC produced technologies, products and processes. This includes $8.6 billion of impacts already materialised from 1991 to 2012 and a further $5.9 billion of imminent impacts estimated to occur over the next five years....this study has identified significant:

environmental benefits including impacts on land, ecosystems, pollutants, natural

resources, plants, animals and biodiversity; and

social benefits that affect the Australian community, the health and well‐being of

individuals and any other social implications.

The unique structure of the program has had a significant influence on the program’s ability to produce high quality research and link researchers with industry...The CRC program has proven to be highly important to the Australia R&D scene. By linking researchers with domestic and international end users, the program has delivered significant economic, environmental and social impacts (Allen Consulting Group 2012, p. vi).

A central feature of CRCs is the core criticality of collaboration between industry and public funded academic researchers.

Aim and purpose of study

The purpose of this paper is to present data from the methodological scan of CRC for Rail Innovation research from 2007 to 2014. The scan aims to identify the level of cross institutional and industry collaboration and the level of methodological diversity and innovation. These indicators of research culture will be investigated further with input from Program Leaders, CRC Research staff and CRC Scholarship holders to explore the research culture which has developed within the CRC for Rail Innovation Research Program. Data from the methodological scan of research projects (n=103) across the four research programs will be presented in this paper.

Research Questions

As a result the following research questions were posited for the first stage of the study (content analysis of the CRC for Rail Innovation research projects 2007‐2014):

RQ1: What is the level of methodological diversity and innovation in the funded CRC for Rail Innovation research projects from 2007 to 2014?

RQ2: What have been the levels of cross institutional collaboration and industry engagement with the CRC for Rail Innovation research projects funded from 2007‐2014?

The research project will provide a great amount of value not only to the CRC for Rail Innovation board but also the wider Australasian rail sector and the CRC Association (CRCA). Through conducting this study and determining the level of collaboration and methodological diversity the findings can assist the CRC for Rail Innovation as they are considering future research partnerships and collaborative research entities.

2. The CRC for rail innovation The Cooperative Research Centre (CRC) for Rail Innovation was established in 2007 and is due to be concluded on 30 June 2014 (7 year program). The CRC for Rail Innovation was originally founded in an effort to provide funding and guidance regarding the conduct of research projects into issues that were present in the Australian rail sector. This involves the strategic participation of various key companies in the rail industry in the Australasia region as well as seven universities across Australia (Cooperative Research Centre for Rail Innovation, n.d.). The research program has invested $100m (AUD) in rail research across these six themes.

The current rail CRC has a much broader scope than engineering and technical research as compared to its predecessor. The current participants of the CRC for Rail Innovation are outlined in Table 1. These include the seven participating Australian universities, core industry participants (n=12), supporting participants (n=14) and other participants (n=8). These include the seven participating Australian universities, core industry participants (n=12), supporting participants (n=14) and other participants (n=8).

75

Roslyn Cameron Table 1: CRC for rail innovation participants Central Queensland University Queensland University of Technology

Core‐University Core‐University

The University of Queensland

Core‐University

Southern Cross University The University of Wollongong

Core‐University Core‐University

Monash University

Core‐University

University of South Australia

Core‐University

Aurizon Operations Limited

Core

Queensland Rail Limited

Core

Rail Corporation of New South Wales

Core

Fortescue Metals Group Australian Rail Track Corporation Ltd

Core Core

Rio Tinto Australasian Railway Association Inc Public Transport Victoria

Core Core Core (Consortium)

‐ Metro Trains Melbourne Pty Ltd ‐ V/Line Passenger Pty Ltd ‐ Victorian Rail Track ‐ Yarra Trams (KDR) NZ ‐ Kiwirail NZ ‐ New Zealand Transport Agency

Core (Consortium) Core (Consortium) Core (Consortium) Core (Consortium) Supporting (Consortium) Supporting (Consortium)

NZ ‐ Veolia Transport Auckland Ltd

Supporting (Consortium)

NZ ‐ Auckland Regional Transport Authority

Supporting (Consortium)

Canada ‐ Canadian Pacific Canada ‐ Canadian National Railway Company

Supporting (Consortium) Supporting (Consortium)

Canada ‐ Via Rail Canada Inc. Canada ‐ Transport Canada

Supporting (Consortium) Supporting (Consortium)

Canada ‐ Railway Association of Canada

Supporting (Consortium)

Transport for NSW Department of Transport Planning and Infrastructure Public Transport Authority of Western Australia Department of Transport and Main Roads Genesee & Wyoming Australia Pty Ltd

Supporting Supporting Supporting Supporting Theme (size based)

Brookfield Rail Pty Ltd

Theme (size based)

ARUP Pty Ltd

Theme (size based)

Tasmanian Railway Pty Ltd

Theme (size based)

Source: CRC for Rail Innovation

Research programs

There are four major research programs within the CRC for Rail Innovation and six research themes across these programs. The four programs are: R1:

Economics, Social and Environmental Sustainability

R2:

Operations and Safety

R3:

Engineering and Safety

P4:

Education and Training Programs

The CRC for Rail Innovation has continued to focus on six research themes that have helped the rail industry solve the major challenges it is facing, namely:Research Theme 1: Safety and Security

76

Roslyn Cameron Research Theme 2: Urban Rail Access Research Theme 3: Performance Research Theme 4: Smart Technologies Research Theme 5: Climate Change and the Environment Research Theme 6: Workforce Development.

These six research themes can cut across the four research programs (see Table 2 for more details).

3. Aspects of the methodological scan The methodological scan covers issues related to levels of collaboration between CRC participants and levels of methodological diversity. The theoretical foundations of each of these aspects of the methodological scan will be briefly introduced.

Collaboration

Collaboration is a central feature to the CRC initiative and the CRC for Rail Innovation has a policy which requires at least two university and two industry partners on each research project. This is seen as mandated collaboration however it ensures a minimum level of collaboration between universities and industry participants and the distribution of research activity across geographic areas as well as across CRC participants.

Industry and participant engagement in the CRC programs is vital to its success. Each research project has a Project Steering Committee (PSC) made up of a Project Leader (university academic), a Project Chair (industry representative), and participating university researchers and industry representatives. These PSCs can meet anywhere from monthly to ever quarter and must undertake several reports to the CRC during the life of the research project in addition to the research report(s).

Collaboration is increasingly being investigated in relation to research and research partnerships however it has been a relatively unknown factor in terms of its influence. Shulha and Wilson (2003, p. 639) assert that collaboration in research remains relatively unstudied despite the fact that ‘...the increasing appearance of collaboration as a criterion in the adjudicating of human and social science research proposals suggests that collaboration is gaining credibility as a feature of investigation’.

Shulha and Wilson (2003) built on the work of Baldwin and Austin (1995) to develop a tool to assist in practical activities for forming collaborative teams and monitoring them. The tool is a rubric for understanding collaboration. They identified factors and dimensions from various literatures including cognitive development, decision making, and sociocultural studies. From this they developed five elements to explain the variability in learning during collaborative research. These five elements are: motivation for involvement; depth of participation; quality of the dialogue; authority in decision making and; joint meaning making (Shulha and Wilson 2003, p. 663).

For the CRC for Rail Innovation additional contextual factors are at play and include the following for all participating parties: governance structures, funding models, performance indicators/ KPIs, industry and sector structures, policy structures, fluctuations in economies, etc. All these can be analysed through stakeholder theory and analysis.

Methodological diversity

One of the stated aims of the methodological scan of the CRC for Rail Innovation program was to analyse the methodological diversity across the four research programs and six research themes. The framework used to measure and categorise the CRC research projects in terms of methodological diversity is based on definitions of mono, multiple and mixed method research designs as developed by Teddlie and Tashakkori (2003).

Monomethod designs are defined as those designs which ‘use a single research method or data collection technique (QUAL or QUAN) and corresponding data analysis procedures to answer research questions. They are also know as single‐phase designs’ (Tashakkori and Teddlie 2003, p. 711). Multimethod research occurs when ‘the research questions are answered by using two data collection procedures (e.g., participant observation and oral histories) or two research methods (e.g., ethnography and case study), each of which is from the same QUAL or QUAN tradition’ (Teddlie and Tashakkori 2003, p. 11). ‘Mixed methods designs (use

77

Roslyn Cameron QUAL and QUAN data collection procedures or research methods) 1. Mixed method research (occurs in the methods stage of a study) 2. Mixed model research (can occur in several stages of a study)‘ (Teddlie and Tashakkori 2003, p. 11).

Figure 1 provides a visual depiction of the definitional framework for the mixed and multi method research designs.

Figure 1: Definitions of multiple method research designs

4. Methodology The overall research design for the full project is an exploratory mixed methods research design. Wu (2012) argues that through the use of mixed methods approaches to research that there can be an increased level of understanding of the data collected. In addition to this, Klingner and Boardman (2011) propose that the use of mixed methods research approaches are important as qualitative data is essential to validating the quantitative data that has been collected. This is of particular importance to the Methodological Scan CRC for Rail Innovation 2007 to 2014 research project as a mixed methods approach has allowed for the triangulation of the data. The study analysed secondary data sets which involved the quantitizing of qualitative data.

Quantitizing refers to ‘converting qualitative data into numerical codes that can be statistically analyzed’ (Tashakkori and Teddlie 2003, p. 714). This was followed by primary data collection via surveys (quantitative) and semi‐structured interviews (qualitative). The research design employed is an exploratory sequential mixed methods research design which is a two phased approach starting with qualitative data to explore a phenomenon and then this is used to develop the second quantitative phase (Creswell and Plano Clark 2007).

The study deviated from this design by including both a quantitative and qualitative component in the second phase due to the complexity of issues being explored.

The first stage involved a content analysis of the research projects across all the Research Programs through the analysis of research reports and associated documentation supplied by the Rail CRC. The analysis focused upon the composition of the Project Steering Committees (PSCs) including industry representatives and the research teams within these and the research designs and methodological approaches undertaken in each research project. The second stage of the study involved an online survey administered to past (completed) and current CRC scholarship students. The survey analysis is currently underway along with the data collection and analysis for the second stage qualitative data. This will involve a qualitative inquiry with key stakeholders utilising semi‐structured interviews. The theoretical framework that has guided this aspect of the research study has been stakeholder theory.

78

Roslyn Cameron Due to the timeframes and limitations of this paper the data being presented in this paper derives from the extensive content analysis of 103 research projects from the CRC for Rail Innovation from 2007 to 2014.

Figure 2 depicts the overall research design for the full project. The data being reported in this paper is derived from the content analysis of Stage 1.

Stage 2: QUAN + qual CRC Scholarship Student Surveys: Past and Current students Semi-structured interviews with CRC research staff and Program Leaders (n=8)

Stage 1: QUAL Content Analysis of CRC Research Projects 2007-2014 (n=103)

QUAL Æ QUAN + qual

Figure 2: Mixed methods research design The Methodological Scan CRC for Rail Innovation 2007 to 2014 research project commenced in June 2012 and is due for completion at the end of January 2013. The preliminary secondary data collection and analysis began in June. Secondary information and data on each research project was sourced for the CRC for Rail Innovation and access was provided to documents and reports. From this data a series of content analyses was undertaken. This involved the analysis of the composition of PSCs from research projects from across research programs and research themes, categorising each research project in terms of methodological approach (mono, multi, mixed methods) and the number and type of data collection methods utilised in each project.

5. Findings and discussion Data and findings from the methodological scan is presented in this section. Table 2 provides a summary of the research projects across the research programs and research themes. Three of the 106 research projects from 2007 to 2014 were cancelled. Analysis and data presented has focused upon the 103 projects which proceeded.

Table 2: CRC for rail innovation research programs and themes Program

Theme

R1: Economics, Social and Environmental Sustainability

Urban Rail Access Climate Change & the Environment Workforce Development Performance Smart Technologies Safety & Security Smart Technology Urban Rail Access Workforce Development Performance Safety & Security Urban Rail Access Workforce Development

R2: Operations and Safety

R3: Engineering and Safety

P4: Education and Training Programs TOTAL

No. of Projects 20 9

Total Approved 37

Total**

22

21 (20.3%)

37 (36%)

4 3 1 16 3 2 1 18 4 3 22

25

24 (23.3%)

22

106 **

21 (20.3%) 103 (100%)

**3 projects were cancelled (P4.118, R2.108 & R3.125).

The R1 program had the most number of funded research projects (36%) followed by R3 (23.3%), and R2 and P4 with 20.3% each of the research projects. The R1 program had the greatest mix of projects across the five research themes, with the Urban Rail Access research theme with the most projects in the R1 program (n=20).

The R2 program had projects across four research themes with the majority in the Safe & Security research

79

Roslyn Cameron theme (n=16). The R3 program had projects across three of the research themes with the majority being in the Performance research theme area (n=18). The P4 program focused entirely upon one of the research themes: Workforce Development (n=21). If the number of projects are analysed in terms of research themes only, the top funded research themes were: Workforce Development (n=27), Urban Rail Access (n=25), Performance (n=21) and Safety and Security (n=20).

Table 3 provides a summary of the number of data collection methods utilised in each project across the four research programs.

Table 3: Data collection methods across the four research programs

R1 R2

1 Method 14 5

2 Methods 9 5

3 Methods 9 6

4 Methods 3 4

5 Methods 2 0

TOTAL 37 20

R3 P4 TOTAL

8 2 29 (28%)

10 7 31 (30%)

6 7 28 (27%)

1 4 12 (12%)

0 1 3 (3%)

25 21 103 (100%)

The most common use of methods is at least two data collection methods (30%) followed by one method (28%) and then 3 methods (27%). Twelve per cent of projects had four data collection methods and 3% had 5 data collection methods. It must be noted that of the one method projects (n=29) a majority of them were a literature review (n=23). This reflects the number of smaller scoping studies which were funded which may or may not have progressed to a larger study and were generally literature reviews. Haneef (2013) refers to this as pre–empirical content.

Figure 3 provides a visual depiction of the data collection methods across the four research programs.

16 14 12 1 Method

10

2 Methods

8

3 Methods 4 Methods

6

5 Methods

4 2 0 R1

R2

R3

P4

Figure 3: Data collection methods across the four research programs As can be seen from this figure the R1 program has a predominance of one method projects with the most two method projects in the R3 program. The R2 program has a relatively even distribution of data collection methods (1‐4 data collection methods) across its program and the R2 and P4 programs have the highest rates of 4 data collection method projects.

Table 4 summarises the number of methodological approaches utilised in each project across the four research programs. As is demonstrated by Table 4 the most common approach is that of qualitative multimethods research (42%) followed by qualitative monomethods (28.5%) and mixed methods (28.5%). There were no quantitative multimethod research approaches employed and only one quantitative monomethod research approach utilised.

80

Roslyn Cameron Table 4: Categories of methodological approaches across the four research programs methods

Multi QUAL 14 12 8 9 43 (42%)

R1 R2 R3 P4 TOTAL

Mono QUAL 14 3 8 4 29 (28.5%)

Mixed Methods 8 4 9 8 29 (28.5%)

Mono QUAN 0 1 0 0 1 (1%)

Multi QUAN

TOTAL

0 0 0 0 0

36 20 25 21 102 (28.5%)

The findings indicate a strong preference for qualitative methods and for using qualitative methods in multiple and mixed data collection configurations. Quantitative methods have been use predominantly in mixed methods projects on 29 occasions (28.5%), with only one mono quantitative and no multi quantitative approaches present.

Figure 4 provides a visual depiction of the research approaches across the four research programs (R1, R2, R3 and P4).

16 14 12 Mono QUAL

10

Mono QUAN

8

Multi QUAL Multi QUAN

6

Mixed

4 2 0 R1

R2

R3

P4

Figure 4: Categories of methodological approaches across the four research programs methods The highest rates of qualitative multimethods are in the R1 and R2 programs, closely followed by the P4 and R3 programs. The highest rates of qualitative monomethods are in the R1 program followed by the R3 program. The highest rates of mixed methods approaches are found in the R3 followed closely by R1 and P4 programs.

Table 5 summarises the data collection methods utilised across the four research programs. Nineteen different types of data collection methods were identified as being used. This demonstrates a variety of methodological variation across the projects. The most frequently utilised method was literature reviews (36.6%) followed by interviews (16.3%), focus groups (8.4%), surveys (7.1%), workshops (6.2%) and case studies (4.8%) and simulations (4.8%).

Table 5: Frequencies for data collection methods DATA COLLECTION METHOD Literature review

R1

R2

R3

P4

TOTAL

27

13

23

20

Interviews

14

6

1

16

Focus groups

8

7

0

4

Survey

4

1

6

5

83 (36.6%) 37 (16.3%) 19 (8.4%) 16 (7.1%)

81

Roslyn Cameron DATA COLLECTION METHOD Workshop

R1

R2

R3

P4

TOTAL

8

5

1

0

Case studies

6

1

0

4

Simulations

2

4

5

0

Field test

1

0

6

0

Questionnaire

3

1

0

2

Observation

1

4

0

1

Modelling

2

0

3

0

Lab test

1

0

3

0

Human Factor analysis Other

0

1

1

0

Pilot Study (1)

Pref. Choice Experiment (1)

Indy Forum (1)

Trials (1) Desk Audit (1) Delphi Technique (1)

14 (6.2%) 11 (4.8%) 11 (4.8%) 7 (3.1%) 6 (2.6%) 6 (2.6%) 5 (2.2%) 4 (1.8%) 2 (0.9%) 6 (2.6%)

227 (100%)

The literature reviews dominate but as mentioned earlier this is mainly due to the number of smaller scoping studies which were funded.

Tables 6 and 7 summarise the university and industry participant involvement within the research projects across the research programs. Table 6 indicates the number of times academics from each of the seven universities who were represented on Project Steering Committees across the four research programs. This summary is presented in Table 6 in descending order with the highest frequency for each university in bold.

Table 6: Summary of university involvement in programs Program R1

CQU 7

UniSA 10

QUT 13

SCU 26

UQ 13

UoW 4

Monash 2

R2 R3 P4

9 17 6

14 1 13

8 11 5

0 0 4

4 4 3

0 9 3

7 6 0

TOTAL

39

38

37

30

24

16

15

CQUniversity (CQU) had the highest level of representation followed closely by the University of South Australia (UniSA) and Queensland University of Technology (QUT). The research programs with the highest participation rates for each university are bolded in the table. Southern Cross University (SCU), QUT and the University of Queensland (UQ) had their highest representation in the R1 program. Monash University had its highest representation in the R2 and R3 programs. CQUniversity and the University of Wollongong’s (UoW) highest representation was in the R3 program. The highest university representation in P4 was with UniSA. Four universities had involvement across all four of the research programs (CQU, UniSA, QUT and UQ). Two universities had involvement in 3 of the 4 research programs (UoW and Monash) and SCU was involved in 2 of the 4 research programs which can be explained by the fact SCU is the only university participant which does not have an engineering school.

Table 7 indicates the number of industry participants (IP) who were represented on research projects across the four research programs.

82

Roslyn Cameron Table 7: Summary of industry participants involvement in programs Program R1 R2

1 IP 1 2

2 IP 2 4

3 IP 7 2

4 IP 12 3

5 IP 13 2

6 IP 0 1

7 IP 2 4

8 IP 0 1

9 IP 0 0

10 IP 0 2

TOTAL 37 21*

R3 P4 TOTALS

2 0 5

8 2 16

7 7 23

3 3 21

3 1 19

0 3 4

0 1 7

1 2 4

0 1 1

0 1 3

24* 21* 103*

* 3 projects cancelled (P4.118, R2.108 & R3.125).

The highest frequency for industry participant representation of Project Steering Committees was 3 followed by 4 and 5. Only 5 projects (4.9%) had only one industry participant. A total of 19 projects (18.4%) had over five industry representatives with 76.7% of the research projects with 2‐5 industry representatives. The data points to the fact that mandatory policy of a minimum of two industry partners was reached and exceeded with the exception of the five 1 industry partner projects.

6. Conclusion This paper has reported the findings from the first stage of an exploratory sequential mixed methods study which has presented data from a methodological scan of the CRC for Rail Innovation from 2007 to 2014. The scan aimed to identify the level of cross institutional and industry collaboration and the level of methodological diversity and innovation. The study found the most frequent data collection configuration for research projects was that of two data collection methods (30%) followed by those studies using 3 data collection methods (27%) and 4 data collection methods (12%). The most common methodological approach was qualitative multimethods research (42%) followed by qualitative monomethods (28.5%) and mixed methods (28.5%). There were no quantitative multimethod research approaches employed and only one quantitative monomethod research approach utilised. The findings indicate a strong preference for qualitative methods and for using qualitative methods in multiple and mixed data collection configurations. Quantitative methods have been use predominantly in mixed methods projects. Nineteen different types of data collection methods were identified which demonstrates methodological diversity. In terms of university involvement four universities had representation on all four research programs, two had representation on 3 and one had representation on 2 of the 4 research programs however this university was the only university of the seven without an engineering school. In terms of industry engagement the data demonstrated high rates of. The CRCs policy of a minimum of 2 industry and university partners on research projects seems to have achieved above minimum levels of collaboration and involvement across the majority of projects. Those research projects with over the minimum requirement of 2 industry participants (3 to 10 industry participants) represented 79.6% (n=82) of the projects.

The study will provide great value in evaluating the level of success with the current CRC model and provide insights into future approaches. The findings of this study will form a valuable evidence base for recommendations regarding future research models that could be used to promote an effective and sustainable research culture within the Australasian rail sector after the current CRC for Rail Innovation ceases in July 2014. This is of particular importance to the CRC Rail Innovation Board as they are considering future models for collaborative research within the Australasian rail sector.

Acknowledgements The author are grateful to the CRC for Rail Innovation (established and supported under the Australian Government’s Cooperative Research Centres program) for the funding of this research, A Methodological Scan CRC for Rail Innovation 2007‐2014.

The author is also grateful and wishes to acknowledge the contribution made by Max Halpin, a student of CQUniversity in 2012, who assisted in the research project through a Business Project Course BUSN19021, in his Bachelor of Business program.

83

Roslyn Cameron

References Allen Consulting Group. (2012) The Economic, Social and Environmental Impacts of the Cooperative Research Centres Program, report to the Department of Industry, Innovation, Science Research and Tertiary Education, July 2012, Canberra, ACT. Baldwin, R.G. and Austin, A.E. (1995) "Toward Greater Understanding of Faculty Research Collaboration," Review of Higher Education, Vol. 19, No. 1, pp 45‐70.

Cameron, R. (2011) “A methodological scan of NCVER funded research 1998‐2010”, 14th Annual Conference, AVETRA, Melbourne, 27‐29 April.

Cooperative Research Centres Association. (CRCA) (n.d). About the CRCA, accessed 2 December 2013, http://crca.asn.au/about‐the‐crc‐association/about‐crcs/ Cooperative Research Centre. (n.d.,) Welcome to the CRC for rail innovation, [online], viewed 20 September 2012, http://www.railcrc.net.au/ Cooperative Research Centre. (n.d.,) Core participants, [online], viewed 21 September 2012, http://www.railcrc.net.au/participants/core CRC for Rail Innovation. (2012) Annual Report 2011‐2012, Brisbane, Australia, [online], viewed 10 January 2013, http://www.railcrc.net.au/sites/default/files/managed/files/crc_for_rail_innovation_annual_report_11.12.pdf

Creswell, J.W. (2010) “Mapping the developing landscape of mixed methods research” in Tashakkori, A. and C. Teddlie (2010). Handbook of Mixed Methods in Social & Behavioral Research, Sage, Thousand Oakes, CA, pp 45‐68.

Creswell, J.W. and Plano Clark, V. (2007) Designing and Conducting Mixed Methods Research, Sage, Thousand Oaks, California.

Department of Innovation, Industry, Science and Research. (DIISR) (2001) Cooperative Research Centres (CRC) Program CRCs over time, DIISR, Canberra.

Haneef, N. (2013) “Empirical research consolidation: a generic overview and a classification scheme for methods”, Quality and Quantity, Vol. 47, pp 383‐410.

John‐Steiner, V. (2000) Creative Collaboration, New York, Oxford University Press.

Klingner, J. and Boardman, A. (2011) “Addressing the “research gap” in special education through mixed methods”, Learning Disability Quarterly, Vol. 34, No. 3, pp 208‐218.

Plano Clark, V. (2010) “The adoption and practice of mixed methods in: U.S Trends in federally funded health‐related research”, Qualitative Inquiry, Vol. 16, pp 428‐440.

Shulha, L.M. and Wilson, R.J. (2003) “Collaborative mixed methods research”, in Tashakkori, Abbas. & Teddlie, Charles.

Handbook of mixed methods in social and behavioral research. Sage, Thousand Oaks, CA, pp 639‐669.

Tashakkori, A. and Teddlie, C. (2003) Handbook of mixed methods in social and behavioral research. Sage, Thousand Oaks, CA. Teddlie, C. and Tashakkori, A. (2003) “Major issues and controversies in the use of mixed methods in the social and behavioural sciences”, in Tashakkori, Abbas. & Teddlie, Charles. Handbook of mixed methods in social and behavioral research. Sage, Thousand Oaks, CA, pp 3‐ 50.

Wu, P. (2012) “A mixed methods approach to technology acceptance research”, Journal of the Association for Information Systems, Vol. 13, No. 3, pp 172‐187.

84

Conflicting Conceptions of Business Ethics Methodology Nicholas Capaldi Loyola University, New Orleans, USA capaldi@gmail.com

Abstract: This paper begins by outlining the initial difference between Philosophical and Management methodological approaches to business ethics. It then identifies two competing conceptions of Philosophical methodology: normative versus explicative. Normative methodology answers the question “what should business ethics practice be? It then explains how and why normative methodology was adopted by Management and marketing specialists. Explicative methodology, on the other hand, addresses the question “what are the norms inherent in current business practice?” After noting cultural differences (US vs. EU vs. Asia, etc.), explication raises three issues: Are the norms internally consistent? Are the norms consistent with other (larger?) societal norms? Will there be consensus, convergence, or inevitably conflicting normative frameworks? A brief case is made for the superiority of explication.

Keywords: normative, explicative, narrative, philosophical, management

1. The quest for ethical expertise (the initial difference between philosophical and management methodological approaches to business ethics) Prior to 1960, deliberation about ethical matters and initiation into ethical decision making had largely been left to centers of cultural authority that operated quite independently of the academy: religion, the family, the professions, and other intermediate institutions. For a number of reasons, these traditional centers of authority were discredited and institutional leaders were persuaded that there were ‘ethical experts’. The question arises where does one find ethical experts?

In 1959 both the Ford Foundation (Gordon and Howell, 1959) and the Carnegie Foundation (Pearson, 1959) completed studies urging that(a) business education in general become more academic and less tied to practitioners and (b) that there be the implementation of a “management science” based on the application of theory to practice (Khurana, 2007).

The field of Management is the primary locus of business ethics in American business schools. The faculties who teach business ethics largely come from two backgrounds: Philosophy and Management. Many of the PhDs in Management reflect a social science methodology. The social sciences have encouraged empirical studies, and to engage in such a study with regard to ethics would involve such things as an analysis of how business‐people make moral decisions much the same way as marketers study consumer preferences. This approach does not challenge the status quo of current ethical decision making; nor does it by itself reflect a normative management science – something that professionals could authoritatively teach business practitioners. It was felt that something more was needed – what that is will reflect developments in philosophy. It was the evolution of intellectual developments in philosophy that made the idea of normative management science tenable.

Prior to 1970, philosophy had treated ethical issues in an unpromising fashion: either philosophers construed their task as meta‐ethical (clarification of ethical discourse) or they were content to point out the difference between facts and values in a way that precluded a normative science of ethics. All of that changed with the publication of John Rawls, A Theory of Justice in 1971. Subsequently, philosophers felt themselves empowered to engage in normative work. Medical ethics and business ethics became the major growth subfields in philosophy. Social scientists in general and management specialists in particular focused on methodological concerns and were quick to borrow from the discipline of philosophy the idea that they too could engage in normative work.

What accounted for this change? The answer is the domination of intellectual culture by the belief in the existence of an authoritative and viable social technology. This belief is called the Enlightenment Project (Capaldi, 1998). The Enlightenment (Becker, 1962) Project (EP) is the attempt to define, explain, and control the human predicament through the use of scientific technology. Since Galileo and Newton among others physical science has been successful in explaining, predicting, and controlling the physical universe, there must be an analogous social science that will enable us to explain, predict, and control the human and social world, that is

85

Nicholas Capaldi produce a social technology. What the EP promises is to explain both how there can be management science and how social scientists and philosophers can engage in normative theorizing and education. We can even, plausibly, educate students to do the “right” thing.

2. How does the EP accomplish this?

There are two kinds of explanation in physical science: elimination and exploration. Elimination substitutes new ideas for older ones (e.g., the replacement of Ptolemy's geocentric view of the universe with Copernicus' heliocentric view of the universe). This is not useful in the social sciences because social theorists claim, initially, to be explaining our values not immediately replacing our values.

Exploration, we begin with our ordinary understanding of how things work and then ‘explains’ what is behind those workings. This leads to a change in our ordinary understanding. The new understanding does not elaborate the old understanding, but replaces it by following out the implications of some hypothetical model.

The replacement ‘explanation’ appeals to an underlying structure (e.g., the atomic theory explains chemical behavior and the behavior of gases). Exploratory explanations seek the formal elements underlying the everyday world rather than accepting that the everyday world can constitute its own level of understanding.

For example, ordinary moral judgments are epi‐phenomena that do not exhibit law‐like behavior; the underlying sub‐structure allegedly permits scientific exploration.

Exploration was already a philosophical model of explanation in social science. By analogy with physical science, social scientists seek out the hidden structure behind the everyday understanding of social activities.

Marx and Freud are examples of social scientists who sought to reveal a hidden structure behind everyday activities.

3. Normative methodology The resurgence of exploration in philosophy is best exemplified by the influence of Rawls. The influence of Rawls on business ethics can be seen prominently in the work of Thomas Donaldson (1982, 1989) and Christopher McMahon (1994).

Rawls offered an exploration in his book A Theory of Justice. He does not start with an explication of existing practice but with an imaginary original position. He is not interested in where we are and how we got there but where we ought to be. In the hypothetical ‘original position’ we choose principles of justice ‘behind a veil of ignorance,’ without knowledge of “[one’s] place in society, his class position or social status; nor does he know his fortune in the distribution of natural assets and abilities, his intelligence and strength,” even one’s sex (Rawls, 1971). If this is not persuasive, one can always appeal to Nozick (1975), Rawls’ Harvard rival.

Nozick, in Anarchy, State, and Utopia (1975) also offers an exploration, postulating a Lockean state of nature without a contract. Primarily focused on the insecurity of individual rights in a free market, human beings are led by an invisible hand to join in protective associations with the state emerging as the dominant protective association. The major obligation of the state is to provide protection against force, theft, and fraud and to enforce contracts.

Under the guise of ethical “theorizing” many contemporary philosophers employed concepts from classic famous philosophers such as Aristotle, natural law theory, Kant or Mill. This mode of generic theorizing begins with very abstract principles and then seeks to apply them to specific ethical issues. Its liability is that in the application the theorist must make additional assumptions so that the original abstract principle can be used to arrive at any and all conclusions.

4. Shortcomings of normative methodology as exploration The major shortcoming of exploration is that there is no way to confirm or disconfirm an exploration in the social world, that is, we never reach a structural level that is observable with advanced technology. There are no formal criteria and no consensus to which we can appeal in choosing among competing explorations. Social theorists invariably appeal to informal criteria or intuitions about which hidden structure hypothesis "best" captures our ordinary understanding. There is no independent way of establishing this. The prominence of Richard Rorty (1989) and his view that philosophy is an interminable conversation of incommensurable voices reflects this situation. Nevertheless, exploration is the preeminent mode of explanation in contemporary philosophy, and in business ethics. The next move was to offer a ‘higher’ level exploratory hypothesis about

86

Nicholas Capaldi other people’s exploratory hypotheses. But choosing among competing ‘higher’ level explorations involves us in an infinite regress. To an outsider it appears as if a hidden structure hypothesis is no more than a rationalization for a private agenda. Faced with this impasse, Foucault (2002) was led to argue that the process is simply a power struggle. Thus, both analytic philosophy and most versions of continental philosophy engage in the same type of thinking and make comparable errors. A similar development can be seen in jurisprudence.

The positivism of H.L.A. Hart gave way to the explorations of Ronald Dworkin (1986) and the Critical Legal Studies movement among many others.

A recent development is the appearance of an anti‐market approach drawn from Continental European philosophical sources. There is now an International PhD program at Lund University, Sweden, called Critical management studies. Many researchers in management and organization studies assume that companies and other organizations work for ‘the common good’, that their outputs make things better for customers, employees, owners and the general public. Organizational structures and practices are thought to accomplish organizational objectives and serve stakeholders. In short, organizational and institutional arrangements are seen as fair and unproblematic.

In contrast, critical management studies problematize the status quo. It focuses on the ’darker’ side of business and organizations. The word ‘critical’ has, of course, a number of meanings. All research is critical in the sense that the researcher is observant and intolerant of weak argumentation, speculative statements, erroneous conclusions etc. In critical management studies, ‘critical’ is understood as the stimulation of a more extensive reflection upon established ideas, ideologies and institutions in order to encourage liberation or at least reduce repression. Critical theory is referred to as a tradition of social science, including the Frankfurt School and related authors and lines of thought such as Foucault, critical poststructuralism, certain versions of feminism and so on. Critical management studies is a large and expanding research orientation, prominent in organization studies but also to some extent in accounting, marketing and strategy.

Not only are there significant moral disagreements, but most of these controversies cannot be resolved through sound rational argument. We either have different foundational commitments or we have different rankings of the good. Resolution is achievable only by begging the question, arguing in a circle, or engaging in infinite regress. We cannot appeal to consequences without knowing how to rank the impact of different approaches with regard to different moral interests. We cannot appeal to preference satisfaction unless we already grant how we shall correct preferences and compare rational versus impassioned preferences, as well as calculate the discount rate for preferences over time. An appeal to disinterested observers, hypothetical choosers, or hypothetical contractors fails as well. Disinterested deciders choose nothing. Intuitions are always met with contrary intuitions (Engelhardt, 2000). There is no reason to believe that these debates can be brought to closure in a principled fashion. As MacIntyre (1981) put it, “There seems to be no rational way of securing moral agreement in our culture.” Partisans of any position find themselves embedded within their own discourse, unable to step outside of their own respective hermeneutic circles without embracing new and divergent premises and rules of inferences.

5. Explicative methodology There is an alternative, namely the form of reflection (not theorizing) championed by Wittgenstein, namely, “explication.” Reflection is ultimately reflection on primordial practices that existed prior to our theorizing about them. Explication as a mode of understanding social practices presupposes that all practices function with implicit norms. To explicate practice is to make explicit the implicit norms. The analogue to explication is case law in Anglo‐American jurisprudence. In explication we clarify that which is routinely taken for granted in the hope of extracting from our previous practice a set of norms that can be used reflectively to guide future practice. We do not change our ordinary understanding but rather come to know it in a new and better way.

Explication is a way of arriving at a kind of practical knowledge that takes human communal agency as primary; it mediates practice from within practice itself.

What meaning can be given to the practice of business ethics? A business ethicist can be either a scholar/teacher or a business ethics practitioner.

The point of business education is to understand and explain market activity in the broadest possible sense. It is a philosophical activity in part because it seeks to reveal the role of business activity on the map of our total

87

Nicholas Capaldi experience. It is not a purely theoretical endeavor. It involves knowledge of our tradition(s) of business behavior. Comparative studies can be valuable in getting us to look more carefully at our traditions. Historical studies can show what people have said and thought ‐ manner of thinking. Its immediate purpose is not to expose errors but to understand its prejudices. Business ethics education is an explanatory not a practical activity; we cannot infer practical consequences from the understanding or explanation. To be a scholar of business ethics is not linked directly to being a business ethics practitioner.

Business ethics as a practice is immanent: it is either identifying the traditional norm(s) relevant to a particular situation or amending existing arrangements by explication of the norms inherent in previous practice. It is similar to the practice of law in the common law tradition. Norms are not accessed as a permanent sub‐ structure; they can never be definitively explicated but are fertile sources of adaptation; they are an inheritance that does not entail its own future development.

Explications are narratives that may contain arguments within them, but the overall explication is not itself an argument. You do not refute an explication, but you can offer an alternative one. An explication presupposes background agreement on what we are trying to achieve; it starts with a diagnosis of the problem at issue; it proposes a response; it recommends this proposal by considering the consequences likely to follow from acting upon it; and it balances these against those of at least one other proposal.

6. Challenges to exploration What happens if we find that there are incommensurable ethical communities? The answer is that you move to the political level and recognize that once you are on the political‐legal level all that you will obtain is agreement on procedural norms that leave different ethical sub‐communities or the members thereof to practice as they see fit within the larger procedural framework. Might there be specific communities in which there is serious conflict and confusion both about what the right thing is to do and about who the mentors are or whether the mentors could be misguided on a specific issue? The answer is yes. Depending upon how charitable one wants to be we can call these communities (a) dysfunctional, or (b) communities in crisis, or (c) disintegrating communities. This is a story for some other occasion.

Explicative methodology in business ethics aims to answer the following question: What are the norms inherent in current business practice? This does not preclude critique of such practices, but the critique of any one would have to appeal to another practice, and in that special sense explication is an inherently conservative intellectual process.

This raises the following challenges:

Can the explication of business practice be carried out wholly from within the practice of business (which seems unlikely) or are we required to consider the larger cultural/social context within which business takes place?

In what sense is commerce itself a microcosm for understanding norms?

Are there not interesting and important differences across cultures that call for comparative analysis: for example, the major market societies of the US/UK, Continental Europe, Japan (or even China, India, Brazil)? (Michel, 1993)

In what sense is globalization influencing business practice?

Will there be consensus, convergence, or inevitably conflicting normative frameworks?

7. Conclusion Given the inevitably partisan nature of exploration, we suggest that some version of explication is a more fruitful methodology for business ethics research. Explication does not provide ready answers to our ethical problems but it suggests a clearer and non‐partisan way of approaching complex issues that provides an understanding of how normative concerns might be included in such research.

References Becker, C. (1962) The Heavenly City of the 18th‐Century Philosophers, Yale University Press, New Haven.

Capaldi, N. (1998) The Enlightenment Project in the Analytic Conversation, Kluwer, Boston.

88

Nicholas Capaldi Donaldson, T. 1982. Corporations and Morality. Prentice‐Hall, Upper Saddle River, NJ. Donaldson, T. 1989. The Ethics of International Business, Oxford University Press, New York .

Dworkin, R. (1986) Law’s Empire, Harvard University Press, Cambridge, MA. Engelhardt, T. (2000) Foundations of Christian Bioethics, Scrivener, Salem, MA. Foucault, M. (2002) Archeology of Knowledge, Routledge, London.

Gordon, Robert Aaron and Howell, James Edwin Howell (1959) Higher Education for Business. Columbia University Press, New York.

Khurana, R. (2007) From Higher Aims to Hired Hands, Princeton University Press, Princeton, NJ., pp. 268‐73.

MacIntyre, A. (1981) After Virtue University of Notre Dame Press, Notre Dame, IN., p.6.

McMahon, C. (1994) Authority and Democracy, Princeton University Press, Princeton.

Michel, A. (1993) Capitalism Against Capitalism, Whurr, London.

Nozick, R. (1975) Anarchy, State, and Utopia, Harvard University Press, Cambridge, MA. Pierson, Frank Cook (1959) The Education of American Businessmen: A Study of University‐College Programs in Business Administration. McGraw‐Hill, New York.

Rawls, J. (1971) A Theory of Justice, Harvard, Cambridge, MA. p. 137.

Rorty, R. (1989) Contingency, Irony, and Solidarity, Cambridge University Press, Cambridge.

89

And we Shall Spend the First Million Together: Narrative as Basis for a Case Study Research Process Margarida Cardoso and Isabel Ramos Universidade do Minho, Guimarães, Portugal mscardoso301@gmail.com iramos@dsi.uminho.pt

Abstract: This work depicts the methodological rationale of a research about open innovation in SMEs. The general objective of the study is to understand how identity strategies determine the adoption and success of interactions among innovation practitioners, enabling open innovation to succeed among small firms and becoming an essential part of their business and informational strategy. The article aims at describing the choice of a specific qualitative approach, case study, linked to actor‐network‐theory epistemological background, as the study involves a series of innovation actors, and how narrative inquiry is chosen to be the most adequate approach to retrieve the information a case study needs to set its methodological pillars.

Keywords: open innovation, case study. actor‐network theory, methods, narrative, triangulation

1. Introduction In Laboratory Life, Latour and Wolgar ask a significant question related to causal relationships: “(…) does the formation of social groupings give rise to the pursuit by scientists of certain intellectual lines of enquiry, or does the existence of intellectual problems lead to the creation of social networks of scientists?” (Latour and Woolgar,1986: 25). We can bring in the same issue related to open innovation (OI) in SMEs: do certain social groups enable the development of OI processes; or instead, are those processes and their objectives the reason why certain networks of open collaboration come up and work? And how do these networks structure themselves, which identity issues allow for an OI ‐ collaboration process. This is the general frame for this study, and the present work is a debate on the methodological structure chosen to approach the issue. In 1, a brief review in OI is presented; in 2, we describe the qualitative settings chosen, introducing actor‐network theory (ANT) and its adequacy to methodological needs, related to case study approach; in 3, a short review is presented about narrative and the way it is useful to that proposed case study approach; in 4, basis both to case study development and narrative, an unstructured interviewing’ debate is issued; in 5 we discuss triangulation and analysis implying a set of different approaches to the same discourse; 6 presents two exemplary stories retrieved through one single case, selected as foundational narratives for certain identity issues; in 7 discourse analysis is proposed for those stories; 8 presents conclusions.

2. A brief review Open innovation (OI) is originally an open‐system’ approach defined as “the use of purposive inflows and outflows of knowledge to accelerate internal innovation and to expand the markets for external use of innovation, respectively” (Chesbrough, 2003: 2). It seems framed by the Zeitgeist 1 , though probably it has always been indispensable to industrial SMEs. While building a proposition for OI research pathways, West, Vanhaverbeke and Chesbrough say that “Open innovation is both a set of practices for profiting from innovation, and also a cognitive model for creating, interpreting and researching those practices. Some of these practices are not new.” (2006: 3). OI implies the need to face a global economy, competition and alternative markets (Dahlander and Gann, 2010; Herzog, 2011), technological complexity (Herzog, 2011), but it also means facilitating standards and collaborative technologies (Dahlander and Gann, 2010) allowing for information flows; If closed innovation implies control (Chesbrough, 2003a; Herzog, 2011: 19), OI accepts interesting human resources might be outside the firm, in a reference to work changing patterns (Dahlander and Gann, 2010), knowledgeable to create complex outputs (Herzog, 2011). Pénin reinforces the concept based in knowledge flows, and calls for a voluntary release of knowledge by firms, open and available to all those participating, collaboration which is continuously reproduced through interactions among stakeholders (Pénin, 2008).

External innovation must be followed by an inside process, so ideas can be best used (Herzog, 2011); Intellectual property rights (IPR) call for an adequate management among all parts involved (Herzog, 2011)

1

Spirit of the time.

90

Margarida Cardoso and Isabel Ramos explored as well as venture capital (Dahlander and Gann 2010). Strategy and business model have to be enhanced prior to reaching the market (Chesbrough and Schwartz, 2007; Chesbrough and Appleyard, 2007; Herzog, 2011). While seeking for a more precise definition of openness, Dahlander and Gann also conclude that innovation itself has always been somewhat open, and that there’s a need to know openness better through qualitative and quantitative methods, to know how decision processes work, exploring product/ technology life‐cycles and their relation to an OI framework ‐ and, finally, how different forms of openness combine.

Saguy tells us about a needed change not only in strategy but also mindsets, if we want to understand OI issues related to the approach done to academia‐industry relationships, IP and other entities involved (Saguy, 2011). This implies considering key elements such as the close academia participation in industrial development, which networks set the stage, support for fundamental and applied research, unravelling intellectual property (IP) issues, managing human resources. The present work focus is on these value generating networks (Hall; 2001) or interorganizational value networks (West, Vanhaverbeke and Chesbrough, 2006: 15), forms “of open innovation (...) in several configurations or typologies” (Skandon, 2011: 85), integrating public and private sector “across fields, sectors and technologies” (OECD, 2008: 5). Preceding Pénin and is knowledge perspective, Hall says that “If the network is to replace the firm as the unit of analysis, perhaps we should be concerned with a knowledge based view of the network” (Hall, 2001: 150).

Skandon states that “trust‐based relationships” in innovation networks reduce risks and costs and can prolong relationships (Skandon, 2011: 85; see also Lee et al., 2010). In SMEs collaboration, “(…) personal relationships play a crucial role, collaboration rules are usually informal, and trust oils the cooperation.” (Vanhaverbeke, Vermeersch and De Zutter, 2012: 11) in which the intermediaries participate. An intermediary is “an organization or a body that acts as an agent or broker in any aspect of the open innovation process” (Howells, 2006: 720). Though Hossain refers to OI intermediary platforms (Hossain, 2012), ours depart from human actors in each network, but not only, as in OI “a wide range of heterogeneous actors interact through formal and informal alliances” (Pénin, 2008: 2). The whole approach relates to SME and micro‐firms’ networks, the relevant economic tissue in Portugal (considering OI in SMEs an urgent issue, as already described by other 2 authors ).

3. A way to a case study: Each case a network While Yin describes case study as a research strategy, more adequate to hows and whys (Yin, 2003), authors like Schrank (2006) consider it a research design; different data sources become implicated, as well as different strategies for information retrieval and analysis are gathered. This means triangulation, to “confirm and deepen understanding by using multiple sources all focusing on the same process/ event” (Woodside, 2010: 1).

This means it can be difficult to circumscribe units of analysis: as Yin states, case study approaches contexts, so “the boundaries between phenomenon and context are not clearly evident” (Yin, 2003: 13).

Case study is defined in many different ways, but methodological principles are indispensable to frame cases through concept formation (it is based on theoretical sampling) and causal mechanisms underlying the case.

The epistemological need to establish a distinction among qualitative and quantitative approaches, no matter both can be used (Schrank, 2006a), doesn’t avoid misunderstandings when regarding quantitatively to qualitative approaches (see also Woodside, 2010 for this matter): "[quantitative] fundamentalists tend to assume (...) that [qualitative] researchers can adopt ‐ or at least mimic ‐ their preferred mode of analysis by examining ever larger samples of cases.(...)”

(Schrank, 2006: 22).

This derives from a fundamental misinterpretation of case‐based research objectives, which in fact relates to a deeper understanding, an inside perspective of processes (Schrank, 2006), a number of factors influencing contexts and actions, such as identity‐related ones, as in the present study.

2

See references to this urgency in research report by Vanhaverbeke, Vermeersch and De Zutter (2012); for a work about SMEs in other context, see for instances Jahja, H. and Wirawan, C. (2012) “Open innovation implementation to sustain Indonesian SMEs”, International conference on small and medium enterprises development with a theme “Innovation and sustainability in SME development” (ISCMED 2012), Procedia Economics and Finance, 4: 223‐233.

91

Margarida Cardoso and Isabel Ramos

Case study can focus on a specific setting, such as interorganizational networks: that is the case. An actor‐ network theory approach is chosen (Latour, 2003), as a coherent ontology to join network‐based cases, with narratives that allow for their structuring and identity as a central issue. ANT seems particularly adequate to give significance to OI processes, as essential network actors. Elements can become actors and also withdraw from the network: actants can become actors, but actors can be “reduced to the role of an actant (an object of the network’s action)” (Czarniawska, 2009: 429). Discourse becomes, then, the narrative basis for these networks’ evidence.

4. Clues for reassembling Boje (2001) describes a set of narrative methods, retrieving information of antenarrative as “the fragmented, non‐linear, incoherent, collective, unplotted, and pre‐narrative speculation, a bet, [on which] a proper narrative can be constituted” (Boje, 2001: 1). Author refers to Aristotle’s definition of the so‐called proper stories as linear, coherent and based on one narrator (Boje, 2008a). But he chooses his perspective according to Czarniawska’s non‐linearity, fragmentation, distribution and collectiveness. In ‘antenarrative’ (Boje, 2001), storytelling becomes a promise, something Latour (1996: 119) shapes as a ‘whirlwind’ (maybe thinking of a spiral‐shaped narrative?), including a chronological and causal sense, often implying uchronic beneficial reviews of the protagonist’s path (Woodside, 2010). While building one “scientifiction”, Latour states the importance of contradictions (Latour, 1996a: 7). Retroaction and translation are relevant to understand the social context for technology, reassembling social meanings over different contexts (Latour, 1996a) and using a method assemblage which selects what will be kept in‐here and what will be absent out‐there ‐ as a complexity‐survival kit (Law, 2004: 13 and following).

Boje signals a set of analytical tools to understand a story, such as deconstruction analysis; the search for a “regime of truth” (Brown, 1991: 192‐3) and the logic of specific social contexts; and a network analysis, where the author states “it’s not about mapmaking, it’s about the living relations” (Boje, 2001: 62). All stories imply a certain network analysis, when trying to translate links behind discourse. But Boje also explores d) intertextuality, antenarrative understood as fragments referring to each other and to other texts; e) causality, search for links between thoughts, understanding results but also looking for the "why"; f) plot analysis, seeing the underlying pattern hidden in a set of events or in a network of relations, which altogether defines a narrative structure and theme analysis, as "theme analysis without antenarrative would divest story of time, place, plurality and connectivity." (Boje, 2001: 122).

If we depict discourse in a spiral way, from categories to their significance, from significance to a frame for our study as some authors propose for the method process itself (see Gobo, 2008, on ethnographic method) ‐ it allows us to unravel juxtapositions of evidence portraying complexity. So we propose to assemble methods on the same discourse basis, digging its significance from categories to context. This implies that discourse produced has the minimum boundaries possible, so the “hinterland of indefinite, necessary but hidden Otherness” (Law, 2004: 14) can be explored. Narrative is expected to allow for the outlining of OI actor‐ networks, as well as their determining identity features.

5. Some good reasons for long unstructured interviews Should discourse analysis be an “umbrella term resulting from a multidisciplinary set of theoretical‐ methodological approaches” (Tracy and Mirivel, 2009: 153), it implies in any case an interview, a conversation among the research participant and the researcher. We can find a triangulation process, related not exactly to ways of information retrieval, or sources of information, but related to the built up of the epistemological nature of the object of study. It is in fact based on assumptions; a discourse related to them and finally analysis, which supposedly brings together two previously parted information chunks: that in our head and the one laying in our subject’s head and life path. “Discourse (…) refers to actual instances of talk (or written texts)” (Tracy and Mirivel, 2009: 154) expected to expose how subjects think about the social.

What is coded and analyzed are cultural and social practices; episodes related to stories told; encounters, social roles, relationships, contexts and groups, etc. (Saldaña, 2009). In our study, due to the choice of a non‐ structured interview approach, coding becomes a more relevant issue to drive the research case as thematically needed.

92

Margarida Cardoso and Isabel Ramos

6. Triangulation, assemblage, translation About fact‐production status in a narrative, Czarniawska states “It is (…) up to the researcher to check the production certificate (by many well‐known method like comparing stories, checking written documents, doing source analysis, etc.) or else take part in the production – that is, making this checking process part of the research results.”

(Czarniawska, 2004: 132).

Triangulation is indispensable to case study methodology: can be that of data (different sources), of researchers (different interviewers), of theories and Denzin’s method triangulation, within or between methods (Flick, 2004). The intrinsically relational condition of ANT (Saldanha, 2003) also adds for the need of triangulation, as a condition to perceive relations among actors (humans and nonhumans) of a network. All these are applied in this work, including source triangulation ‐ interviews, documents available online, accessory information given by people; method triangulation ‐ worked in several different ways, providing either a narrative of events, stories, or the basis for a different comprehension of processes, through discourse analysis.

Besides the ANT definition for translation, as previously mentioned, we can figuratively associate translation to coding (Saldaña, 2009) and interpretation, or (more) literally associate it with languages, implicit and explicit as in the current work, considering authors are Portuguese, and most interviewed people are Portuguese or at least interacting in Portuguese during interviews. Anyway, translation reinforces its centrality in this work, and strengthens the epistemological principle that researchers consider themselves network actors.

7. DaFRM, an open innovation practitioner 3 ?

We have two research questions for the whole study: How does an SME adheres to open innovation practices to improve its business activities? And how relevant and complementary are different identity‐determined processes aiming at open innovation practices? Both were tested in a first exploratory case about an industrial SME, working in waste management industry: Rvolta 4 , a case published in 2012 (Cardoso and Ramos, 2012).

Case shows a network of SMEs and includes research institutions, PT and US labs; associations and customers, besides other elements determining the OI process itself. Issues raised include details about business management, context, work processes and IT; also interaction, where we found specific references about trust, business friendship for R&D, a reference to a “business‐customer relation that allows for a R&D user‐ driven test over an innovative issue” (Cardoso and Ramos, 2012: 98).

5 This case is about another industrial SME, DaFRM , working with lightweight engineering structures and having recently won a prize at one of the most important Portuguese innovation contests. DaFRM was established in 2006 to make R&D on lightweight structures based on a proprietary sandwich panel technology. The company is established at the same time its owner invents a specific lightweight panel. Among its partners we can find an important aircraft manufacturer, or a European OEM 6 of global impact. From information retrieved through interviews two stories were produced, based on two interviews to BC, about DaFRM, together with other information gathered, having a specific value to our analytical basis. Both episodes are temporally and causally connected (Czarniawska, 2004).

7.1 Stories told and the network establishment 7.1.1 Total risk In the first story, we can observe the unfolding of a specific translation event, as risks are a requisite for the SME to become an actor in the center of a macro actor, the network (Latour,1996). This story substantiates an alternative definition for OI, already proposed in the Rvolta case: the “development of risk‐taking knowledge share among firms, so dynamic interactions based on mutual trust allow for shared innovation” (Cardoso and Ramos, 2012: 98).

3

Key definition adopted: “A player who introduces ideas, shares knowledge, risk, transfers contexts and occasionally earns business profits in open innovation grounds” (Cardoso and Ramos, 2012: 98).

4 Rvolta case research participants authorized disclosure and reviewed all information chunks resulting from interviews.

5 The research participant asked for complete anonymity; names are aliases and all information chunks retrieved from the interviews were previously validated by DaFRM research participants.

6 Original Equipment Manufacturer.

93

Margarida Cardoso and Isabel Ramos

Well, I have two possibilities of finding a partner with whom to develop this technology… either he is a raw material producer, or instead he is what we call an integrator, meaning a technology user… so I decided to approach the raw material producers, why? because that way I would have a larger range for potential application… if I chose a train manufacturer as a partner, I would only be able to fabricate trains… on the contrary, raw material can be converted into a lot of things… but then I hesitated, what would be the best cultural context to approach, in an open way, at this rather delicate stage? Cultural context meaning country, business… because for instances… their country is quite aware of open innovation, there is respect and business ethics… and companies are reluctant of being unethical there… so I decided, there are huge enterprises for raw material, that would be very interesting, but what if I got there not having previously my IPR, my patents, my contracts, my lawyers… and then one day I got frightened because I understood the idea was of an extraordinary value, a prospective impossible to determine, it is a platform technology with a lot of connections, because companies I’m telling you about are 7 billion’ and 10 billion’ companies… suddenly we’re not talking about 300 million’ or 500 million’ companies anymore… they’re global companies… and at the heart of this whole mess, while deciding if I would talk to aluminum or steel’s OEM, because it could be either of them… well, I made some contacts but people aren’t always easy to get to, and I told myself “I have a month to decide this issue, I cannot take six months”… I didn’t even have a patent… so it was a total risk, when I decided to share the information, well… it was at my own risk, I talked to people I knew, companies I already had relations with, some awarded me, and so there was common respect, confidentiality, I didn’t 7 even know what a NDA was at that time … I started as an entrepreneur without the faintest training, and what I have nowadays is experience… but I’ve been an entrepreneur all my life, so… inside the companies I worked in … I’ve been an innovator all my life, and everybody listened to me… and all my life I co‐created with costumers, so I found two or three names related to KTK 8 and sandwich technology and I took an enormous risk, I sent a jpeg with my idea, something you can understand in three minutes, any guy knowing anything about this looks at it and says “Jesus”… and I sent an email saying, “Dear friends, I believe you should want this, tell me about it”… and I just knew I chose the right partner when I read the answer: “the last ten emails we got… we were lucky enough to read yours, the idea is so evident, of course we had to answer you… you should come here, we should meet, we want to talk to you”.

7.1.2 Belonging The second story shows us another translation moment, where thanks to previous risk‐taking event and therefore the increase of trust as an essential element, DaFRM becomes a pilot enterprise, meaning a “partner firm working as R&D platform” (Cardoso and Ramos, 2012: 98), from this moment on, to another company, KTK. After taking a total risk contacting them, they called me. I took a plane and went there, people were expecting me: the ones who work with these structures, the company vice‐president ‐ and R&D chief ‐, we had a meeting whit an official record minute, and so they just tell me: “We want this”… and I answered, “well, its yours”… we shook hands and then talked details, and afterwards we started an intellectual protection deal, made the invention assignment… anyway, we had some issues to solve, because I wasn’t their employee, I wasn’t a national, so neither ruled by their company in IPR and invention matters, nor by their laws like one of their citizens (innovation is an issue quite detailed in their work laws, as a matter of fact)… so we had some difficulties to bend, they applied the employee process without me being one, I didn’t give a damn about financials, they paid me a small amount as they would to one of their own, and so I told them: “this becomes your invention, you invest in intellectual property, basic technology, so you’ll finance it… so you’re going to finance it and we shall spend the first million together, I’ll share it with you… and in return I want a royalty free license, for my own future companies, for all materials”.

7

Non‐disclosure agreement: “is a legal contract [between partners], not to disclose information [shared] for a specific purpose, allowing businesses to approach potential partners, suppliers, customers, etc., knowing that information cannot be legally passed on. This can be used to share intellectual property, commercial information, business plans, etc.””, in http://europa.eu/youreurope/business/competing‐ through‐innovation/protecting‐intellectual‐property/united‐kingdom/index_en.htm, [14.2.2013] 8 Another alias, this time a company in DaFRM network.

94

Margarida Cardoso and Isabel Ramos

8. Info generation and analysis Broom, Cheshire and Emmison stress that qualitative data is generated instead of collected, “being co‐ produced by the researcher and research participants” (Moore, 2007; Broom, Cheshire and Emmison, 2009: 1165). So being, data generation is also produced through the researcher participation in this network, if we understand it as an actor‐network where every actant can be implied (Latour, 2003). It is a relevant epistemological standing point to sanction a certain degree of freedom in discourse structuring and narrative, as a contribution for translation. Each network generation process is associated to Latour’s translation as transformation, mediation, the actor‐network formation, the way by which strategies intertwine so to adapt to a network where some degree of common sharing gets established: the object of study. Referred by Latour (1996), and implying this ANT approach, translation becomes a rather important issue for this case, as narratives set up translation moments for an interorganizational OI network, in its most crucial aspects ‐ including actors, risks, intellectual property, trust and the whole network itself, as previously mentioned.

The primary coding method used (Saldaña, 2009) was a simple word cloud (through IBM word cloud), and the second coding was also based on theme information from the first coding, working the transcription discourse 9 through Cmap and NVivo .The process of generating (in the course of an interview, for instances) and afterwards analyzing discourse, mediated through researchers, is evident in Figure 1, where Portuguese words aren’t yet language‐translated.

Figure 1: A word cloud in Portuguese, prepared to anticipate themes for discourse analysis over the first interview (before the second one). We can observe words such as “fazer” (to do), “inovação” (innovation), “empresa” (business company), “tecnologia” (technology), among others we dare suggest the reader to google‐up Discourse analysis can be based on ethnographical methods (Atkinson, Okada and Talmy, 2011). It shows adequacy to this specific object of study, but also to the relational perspective proposed by ANT. The fact that either ethnography and ANT look, at least partially, for an emic‐based approach meaning “as understood by [actors] themselves” (Atkinson, Okada and Talmy, 2012: 86) makes it particularly adequate. Even if that means

9

Cmap is a concept mapping freeware, created by the Florida Institute for Human & Machine Cognition, at http://ftp.ihmc.us/. NVivo is a qualitative and mix‐methods research software from QSR international, at http://www.qsrinternational.com. IBM Word Cloud generator works offline, what is quite adequate for confidentiality and information safety purposes.

95

Margarida Cardoso and Isabel Ramos

a reification exercise... every ethnographer spends a share of his/ her courses dismantling reification tricks, so reification as an ontological proposition (related to the essence of actants, that can be nonhumans as well as humans), will only be a daring exercise.

After the first analysis shown in the word cloud, pointing some of the issues to approach in a second interview, discourse categorization shows other relevant issues. One is the voluntary release of knowledge mentioned by Pénin (2011), mixed with trust (Skandon, 2011; Cardoso and Ramos, 2012) through the partnership choice ‐ I have two possibilities of finding a partner (BC1: 1) and confirmed –“Dear friends, I believe you should want this, tell me about it”… and I just knew I chose the right partner when I read the answer (BC1: 32). Participants state the importance of having a strategy (Chesbrough and Schwartz, 2007; Chesbrough and Appleyard, 2007), as in [If] I chose a train manufacturer as a partner, I would only be able to fabricate trains (BC1: 5); perceived relevance of cultural factors is also apparent (Herzog, 2011): what would be the best cultural context to approach, in an open way, at this rather delicate stage? (BC1: 7), including some specific cultural issues such as time ‐ I have a month to decide this issue, I cannot take six months… (BC1: 18).

The value of the idea is evident to our eyes: I got frightened because I understood the idea was of an extraordinary value (BC1: 12) and the need to safeguard it, including through IPR such as in what if I got there not having previously my IPR, my patents, my contracts, my lawyers (BC1: 11), or I didn’t even have a patent…

10 so it was a total risk, when I decided to share the information (BC1: 19) and I didn’t even know what a NDA was at that time (BC1: 22) (all OI authors normally debate IPR), associated with the risk perception and inherently with trust (Skandon, 2011; Cardoso and Ramos, 2012) as in I took an enormous risk, I sent a jpeg with my idea, something you can understand in three minutes (BC1: 28); the entrepreneurial capacity ‐ I started as an entrepreneur without the faintest training (BC1: 23) and associated with credibility, the innovative capacity, I’ve been an innovator all my life, and everybody listened to me (BC1: 25) ‐ also as co‐ creator all my life I co‐created with costumers (BC1: 26).

The second story reinforces the strategic approach together with investment issues, in this becomes your invention, you invest in intellectual property, basic technology (…) and in return I want a royalty free license, for my own future companies, for all materials (BC2: 12). But it also reinforces trust issues, through the meeting episode: we had a meeting whit an official record minute (…) we shook hands and then talked details, and afterwards we started an intellectual protection deal, made the invention assignment (BC2: 3), again clearly mixed with IPR issues. This story comprises a curious cultural process, related to the way innovation can be framed (hardly): I wasn’t their employee, I wasn’t a nationa l (…) so we had some difficulties to bend, they applied the employee process without me being one (BC2: 7).

9. Concluding on case‐based grounds Both stories retrieved seem almost aristotelic in their linearity; they introduce two relevant moments for DaFRM case, translation moments, meaning those which constitute and give structure to an innovative macro network, which essentials are presented. But the DaFRM case also demonstrates matters previously approached by Rvolta case, including issues such as trust (Lee et al. 2010, Skandon, 2011; Vanhaverbeke, Vermeersch and De Zutter, 2012; Cardoso and Ramos, 2012) and strategy (Chesbrough and Schwartz, 2007; Chesbrough and Appleyard, 2007). The following analysis depicts category replication between the exploratory case and the present. Czarniawska states (when analyzing the emergence of the London School of Economics) that a network as a macro actor can comprise “entrepreneurs, a name, rented rooms, a prospectus, money, staff, students, media testimony: a set of actants in place” (2009: 429). So like in a spiral process, these analytical levels also seek to display the set of actants in place.

10

Non‐disclosure agreement: “is a legal contract [between partners], not to disclose information [shared] for a specific purpose, allowing businesses to approach potential partners, suppliers, customers, etc., knowing that information cannot be legally passed on. This can be used to share intellectual property, commercial information, business plans, etc.””, in http://europa.eu/youreurope/business/competing‐ through‐innovation/protecting‐intellectual‐property/united‐kingdom/index_en.htm, [14.2.2013]

96

Margarida Cardoso and Isabel Ramos

Figure 2: Some shared discourse categories between Rvolta (green) and DaFRM (blue) (Cmap) Though not all exploratory in nature, all case studies end up including some sort of exploratory input, because different issues are at stake concerning methods and sources for triangulation. As each case brings up a core of concepts needed to understand its narrative, these imply the whole case structure. The first case was really exploratory for factors’ and methods’ sake, concerning Rvolta network, even testing the adequacy to a case study methodology for this work. This second case is not exploratory anymore, but includes renewed conceptual and data issues (DaFRM network), implying narrative relevance and interviewing strategy adequacy as well as a specific ontological affiliation to ANT. Finally a third case yet to be worked should confirm previous information, but new actants are expected, substantiating the approach affiliation adequacy to ANT.

Acknowledgements Authors wish to thank interviewees for cases included in the present work, who either becoming public or remaining anonymous had the spirits to endure long interviews, participated in the current research and overtly shared information for the understanding of open innovation processes.

This research has been funded by FEDER funds through Programa Operacional Fatores de Competitividade – COMPETE and Portuguese national funds through FCT – Fundação para a Ciência e Tecnologia under the Project: FCOMP‐01‐0124‐FEDER‐022674”.

References Atkinson, D., Okada, H. And Talmy, S. (2011) Ethnography and discourse analysis, In K. Hyland and B. Paltridge (Eds.), The continuum companion to discourse analysis, London, New York: Continuum International Publishing Group, 85‐100.

Boje, D. M. (2008) Antenarrative, in R. Thorpe and R. Holt, The Sage dictionnary of qualitative manangement research, London, Thousand Oaks, New Delhi, Singapore: Sage, 28‐30 Boje, D. M. (2008a) Storytelling, prepublication version of Storytelling, In S. R. Clegg and J. R. Bailey (Eds.), International Encyclopedia of Organization Studies, Vol. 4, London: Sage, 1454‐1458.

Boje, D. M. (2001) Narrative methods for organizational and communication research. London, Thousand Oaks, New Delhi: Sage.

97

Margarida Cardoso and Isabel Ramos

Broom, A., Cheshire, L. and Emmison, M. (2009) “Qualitative researchers’ understandings of their practice and the implications for data archiving and sharing”, Sociology, Vol. 43, Nr. 6, 1163 ‐ 1180 [online] http://soc.sagepub.com/cgi/content/abstract/43/6/1163.

Brown, R.H. (1991) “Rhetoric, textuality, and the postmodern turn in sociological theory”, Sociological Theory, 187‐197.

Buzelin, H. (2007) Translations “in the making”, In M. Wolf And A, Fukari, Constructing a sociology of translation, Amsterdam, Philadelphia: John Benjamins Publishing Company, 141‐177.

Cardoso, M. and Ramos, I. (2012) Rvolta, a case for open innovation, in H. Rahman and I. Ramos, Cases on SMEs and open innovation: applications and investigations, Hershey: IGI Global, 82‐99.

Chesbrough, Henry (2003) Open innovation: The new imperative for creating and profiting from technology. Boston: Harvard Business School Press.

Chesbrough, H. W. (2003a) “The era of open innovation”. MIT Sloan Management Review, 44(3), [online] http://sloanreview.mit.edu/article/the‐era‐of‐open‐innovation/.

Chesbrough, H.W. (2006) Open innovation: a new paradigm for understanding industrial innovation, In H.W. Chesbrough, W. Vanhaverbeke, J. West, Open innovation: Researching a new paradigm, Oxford: Oxford University Press: 1‐14.

Chesbrough, H. and Schwartz, K. (2007) “Innovating business models with co‐development partnerships”, Research Technology Management, Jan.‐Feb. 2007, 55‐59.

Chesbrough, H. and Appleyard, M. (2007) “Open innovation and strategy”, California Management Review, 50 (1), Fall 2007, 57‐76.

Cozby, P.C. and Bates, S.C. (2011) Methods in behavioral research, 11th edition, New York: McGraw‐Hill.

Czarniawska, B. (2009) “Emerging institutions: pyramids or anthills?”, Organization Studies, 30: 423‐441, [online] http://oss.sagepub.com/cgi/content/abstract/30/4/423.

Czarniawska, B. (2004) Writing social science, In B. Czarniawska, Narratives in social science research, London, Thousand Oaks, New Delhi: Sage, 102‐116.

Dahlander, L. and Gann, D.M. (2010) “How open is innovation”, Research Policy 39, 699‐709.

Flick, U. (2004) Triangulation in qualitative research, In U. Flick, E. von Kardoff, I. Steinke (Eds), A companion to qualitative research, London, Thousand Oaks, New Delhi: Sage, 178‐183.

Gobo, G. (2008) Doing ethnography, LA, London, New Delhi, Singapore: Sage.

Hall, R. (2001) A post industrial network organisation, In Pierpaolo Andriani and Giuseppina Passiante (Eds.), Complexity theory and the management of networks, Proceedings of the workshop on organisational networks as distributed systems of knowledge, University of Leece, Italy 2001, London: Imperial College Press, 149‐154.

Herzog, P. (2011) Open and closed innovation: different cultures for different strategies. S.l.: Gabler Verlag. th Hossain, M. (2012), “Performance and potential of open innovation intermediaries”, 8 International Strategic Management Conference, Procedia – Social and behavioral sciences, 58, 754‐764.

Howells, J. (2006), “Intermediation and the role of intermediaries in innovation”, Research policy, 35, 715‐728.

Latour, B. and Woolgar, S. (1986) Laboratory life: the construction of scientific facts, Princeton, New Jersey: Princeton University Press.

Latour, B. (1996a) Aramis, or the Love of Technology. Trans C. Porter. Cambridge, MA: Harvard University Press.

Latour, B. (1996) “Lettre à mon ami Pierre sur l' anthropologie symétrique“, Ethnologie française, XXVI, 1996, 32‐37.

Latour, Bruno (2005), Reassembling the social. An introduction to actor‐network‐theory, Oxford: Oxford University Press.

Law, J. (2004) After method: mess in social science research, London, New York: Routledge.

Lee, S. Et al. (2010), “Open innovation in SMEs – an intermediate network model”, Research policy, 39, 290‐300.

McKenzie, B. (2008) Techniques for collecting verbal histories, in H. Neergard and J. P. Ulhoi (Eds.) Handbook of qualitative research methods in entrepreneurship, Cheltenham, UK, Northampton, MA, USA: Edward Elgar.

Moore, N. (2007) “(Re)using qualitative data”, Sociological Research Online, Vol. 12 (3) [online]: http://scresonline.org.uk/12/3/1.htm.

OECD (2008), Business Symposium on open innovation in global networks, Policy issues paper, Copenhagen; OECD, 25‐26 February.

Pénin, J. (2008), More open than open innovation? Rethinking the concept of openness in innovation studies, Document de travail nº 2008‐18, Juillet 2008, Bureau d’economie théorique et appliqué (BETA), UMR 7522, Strasbourg: Faculté des sciences économiques et de gestion. th Saguy, S. I.(2011) “Academia‐industry innovation interaction: paradigm shifts and avenues for the future”, 11 International Congress on Engineering and Food (ICEF11), Procedia Food Science 1, 1875‐1882.

Saldaña, J. (2009) The coding manual for qualitative researchers, London, Thousand Oaks, New Delhi, Singapore: Sage.

Saldanha, A. (2003) “Review essay. Actor‐network theory and critical sociology”, Critical Sociology, vol. 29 (3), [online] www.brill.nl.

Schrank, A. (2006) Case studies: case‐based research, in E. Perecman, and S. R. Curran, (Eds.) A handbook for social science field research. Essays & bibliographic sources on research design and methods. Thousand Oaks, London, New Delhi: Sage, 21‐38.

Schrank, A. (2006a) Essentials for the case study method: the case study and causal inference, in E. Perecman, and S. R. Curran, (Eds.) A handbook for social science field research. Essays & bibliographic sources on research design and methods. Thousand Oaks, London, New Delhi: Sage, 169‐174.

Shkedi, A. (2005) Multiple case narrative. A qualitative approach to studying multiple populations, Amsterdam, Philadelphia: John Benjamins Publishing Company.

98

Margarida Cardoso and Isabel Ramos Skandon, J. (2011) “The role of trust in innovation networks”, Procedia – Social and behavioral sciences, 26, 85‐93.

Tracy, K. and Mirivel, J. C. (2009) Discourse analysis: the practice and practical value of taping, transcribing and analyzing talk, in L. R. Frey and K. N. Cissna (Eds.) Routledge handbook of applied communication research, New York and London: Routledge, Taylor and Francis Group.

Woodside, Arch (2010) Case study research: theory, methods, practice, Bingley, UK: Emerald Group Publishing Limited.

Vanhaverke, W., Vermeersch, I. And De Zutter, S. (2012) Open innovation in SMEs: how can small companies and start‐ups benefit from open innovation strategies? Research report, S.l.: Flanders district of creativity.

West, J., Vanhaverbeke, W. And Chesbrough, H. (2006) Open innovation, a research agenda, In H.W. Chesbrough, W. Vanhaverbeke, J. West, Open innovation: Researching a new paradigm, Oxford: Oxford University Press: 285‐308.

Yin, R. K. (2003) Case study research: design and methods, 3rd edition, Thousand Oaks, London, New Delhi: Sage.

99

Crisis Management Simulations – Narrative Inquiry Into Transformative Learning David Clemson and Khalid Samara Department of Business Studies, London South Bank University, UK d.clemson@lsbu.ac.uk

Abstract: Crisis Management Simulations are identified as enhancing transformative learning behaviour patterns as observed in large groups. This paper reviews a study of over 200 crisis management simulation participants over 4 years who have represented their experiences through reflective narrative accounts. The specific focus is on co‐locating the deeper level narratives of transformation, including the development of a coding schema based on Mezirow's Transformative Learning Theory. A series of approaches developed and applied by the authors is illustrated, also noting that this shows some of the highest reported incidence of deeper‐level transformative learning amongst large groups, building on the research of the lead author, in the region of 80%. The relevant literature within transformative learning, sense‐making, rhizomic narrative inquiry, simulations, sustainability, complexity and crisis management is used to provide context for this research into the phenomena of transformative learning. Analysis of guided inquiry, co‐operative inquiry (Dionysian and Apollonian) and reflective narratives creates a differentiation between ‘straightforward’ transformation and ‘deeper level’ transformation which we highlight and use rhizomic narrative inquiry to explore aspects of performativity and authenticity. Some of the emerging themes from this research which we will discuss focus on anxiety and leadership style within crisis management simulations, the evidence of very rapid mimesis in situations of high uncertainty and high anxiety, transpersonal transformative psychological perspectives, particularly those informed by the work of Jung and Hillman. Ethical conduct within business focused crisis management simulations – learner‐centred approaches to generating codes of conduct and ethical standards, levels of self‐discipline and adherence to group‐generated participatory codes compared to external regulation Through a cycle of addressing the issues of ethics, sustainability and risk management in a dynamic learner‐centred way enhancing their abilities to handle complex situations which they are likely to encounter in real‐world business contexts, we are able to conclude that crisis management simulations are a highly effective method of creating a transformative learning environment with larger groups, especially when used in conjunction with reflective processing and narrative inquiry.

Keywords: risk management, transformative learning, management education, complexity narratives, rhizomes

1. Introduction – a crisis in management education?

Melanie Newman (2010, p.17) reports on work by Gerard Hanlon, Director of the School of Business and Management at Queen Mary, University of London, who said that: schools were producing "…students ill‐prepared to cope with the complexity of problems they are likely to face in business – and in life."

"Business schools see the education of managers, rather than the study of management, as their key role…teaching is focused on management technique and is done on the assumption that management is an unqualified good…"

He (Hanlon) added that most of his students aspire to be chief executives of major corporations.

"But if you’re a senior manager at Royal Dutch Shell you’re going to have to deal with geopolitics, terrorism and environmental protesters – and textbook management techniques aren’t going to help much."

And: "…ethics, sustainability and risk management should be key features of MBA courses."

Hence, the question is how to create a more interactive transformative learning experience for students. The use of Crisis Management Simulations (CMS) as a dynamic tool helps with inclusion, enabling a highly effective means of creating a transformative learning environment. Considering real‐life modelling in a teaching and learning (T&L) environment the process of developing T&L experiences designed with CMS are viewed as tools to enhance the group‐based assignments and individual reflections that facilitate an integrative, inter‐ disciplinary learning environment in which transformative learning can flourish.

This paper begins by giving further consideration to CMS as an assessment and learning approach, with specific details as to how this is applied. Setting this in the wider context of transformative learning, particularly Mezirow’s Transformative Learning Theory, and the ways in which transformative learning is evidenced using

100

David Clemson and Khalid Samara guided narrative inquiry and analysis of rhizomic narratives is discussed before presenting an analysis over 4 years of data and emergent findings.

Through the use of interactive simulation tools as in CMS the picture outlined above by Hanlon introduces the three phases of transformative learning that are able to be achieved in an industrially linked business environment, in view of the transferable skills students are able to achieve; critical reflection, reflective discourse, and action. With a CMS learners critically understand that learning involves deep, powerful emotions or beliefs, and is evidenced in action, seen through the role play as outlined above and throughout this paper.

Considering business simulations as a means of a T&L tool, and in the example set out in the paper as an assessment model it is easy to view Clark’s (1991) philosophy of 'perspective transformation', in practice by learners and assessors as CMS cover the three dimensions: psychological (enhancing change in understanding one’s self), convictional (enhancing revision of belief systems), and behavioural (as simulations enhance changes in lifestyle). As is discussed, each year that the CMS has run, refinements relating to the CMS have been put in place to expand the consciousness of learners through the transformation of basic worldview and specific capacities of ones self; as stated by Elias (1997, p.3). CMS sessions are specifically mentioned in student formal feedback as being "the best skill building techniques we’ve taken in the whole course", yet there is on‐going scope to refine the CMS and, with particular reference to transformative learning, CMS tools are able to facilitate experiences through consciously directing processes such as appreciative access and receiving the symbolic contents of the unconscious while critically analysing underlying premises as indicated

by Elias (1997, p.3). CMS tools examine a learner’s experience in co‐creating an environment of transformative learning. It is on this particular facet that this paper concentrates through the use of reflective thinking and narrative inquiry.

2. Educational context – reflective processing and crisis management simulations The research takes a practical approach and covers an investigation taken place within normal classroom activities and is consistent with the British Educational Research Association guidelines. However, the heightened state of anxiety on the run‐up to and during the CMS must be acknowledged – this being an integral and experiential factor of undertaking study in business modules. This feature is something students are made aware of both before starting the module and at the beginning of the module.

At an early stage, students are asked in seminars to reflectively link the learning content with the forms of assessment and consider how they relate to the specific learning outcomes for the module. This early engagement with reflective discussion aims to give students experience, which will be later used in the assessments of the module and, ultimately, become a natural and integral part of their practice. Assessment for this module is made up of a group‐based assignment involving researching and preparing a risk management strategy and business continuity plan for a major international business. This is handed out first and handed in at the end of the module including group‐based reflections. Next is the preparation for the CMS and the associated Individual Reflective Report (IRR). This creates a sense of intensity (DeLanda, 2002) in the learning environment by spiralling inwards towards the CMS as an event. This ‘intensification’ is akin to what Schlamm (2001) associates with Rawlinson’s hot‐unstructured traditions, the period after the CMS being associated with ‘slowing down’ or de‐intensification, akin to a movement through hot‐structured towards cool‐structured traditions (Schlamm, 2001) through reflective processing and analysis.

Students begin to voice concerns (anxiety) about the CMS after the handout of the Group Assignment and the necessary groups were formed in each delivery of the module. Regular discussion takes place in lectures and seminars about simulations and what might/might not happen in situations of crisis, ways of intervening and the importance of preparation.

Students are invited to consider a number of academic journal articles during lectures and seminars and these articles were made available to them through the Virtual Learning Environment (VLE). These articles were supported by brief commentary as to key points, to provide a perspective of the infrequency of disorienting dilemmas as Mezirow (1995, p.50) states these are triggered by a life crisis or major life transition that is considered to be the accumulation of transformations in meaning from sudden interaction with skill building and confrontation within a CMS. The consideration of journal articles like; Cloudman and Hallahan (2006);

101

David Clemson and Khalid Samara DeVries and Fitzpatrick (2006); Dyl (2009); Harvey, J. (1988); Greyser (2009), Pecora (2002); and Van Fleet and Van Fleet (2006), enable less dramatic predicaments in the CMS and hence promote transformation, as individuals are promoted to change their frames of reference by critically reflecting on their assumptions and beliefs. This belief in a transformative learning environment can bring about new ways of defining students’ worlds. For instance, in the 3 most recent deliveries, students were shown Naomi Klein’s (2007) documentary ‘The Shock Doctrine’ which received many examples of thoughtful and positive feedback as students considered the impact that rational and analytical analysis has in a transformative learning environment.

Guidelines for the IRR on the CMS were circulated 4 weeks before the CMS were due to be held, the intention initially having been to offer a sense of structure and calmness – although this seemed, in many cases, to create higher levels of anxiety. With tension rising on the days before the CMS, the overall guidelines, groupings and the roles in the differing scenarios were released through the VLE the preceding day. This did generate a number of exploratory questions and there were students who appeared to have given thoughtful consideration to what might happen and what roles they and others in their team might fill.

On the mornings of the CMS, many students arrive early, discussed possible crisis scenarios, how they might handle them while waiting close to the briefing room. To facilitate reflection, the scenarios were filmed and photographed and made available to participants through the VLE.

For illustration, one of the scenarios (there are four used, students transitioning through the varying roles) used in all 4 deliveries from this research is situated is:

Galaxon Pharmaceuticals is a UK‐based company that researches, develops and manufactures a variety of prescription based drug products used primarily in hospitals, although some are also distributed through GPs. One of the company’s specialist areas is in pain relief for advanced stage cancer treatment and it has recently completed a full scale trial and subsequent licensing of a new pain relief drug Galazeme at St Giles Hospital, a major London teaching hospital.

Galaxon has, for many years, operated a graduate entry scheme which it highly regarded in the industry for its use of mentors for recent entrants and provides a reward scheme for those who introduce new employees.

Concern has been raised with the Director of Risk Management that some relationships between Mentors and Employees could lead to potential breaches of security. Rumours about this have been circulating on Twitter…

Group 4 – approx 12 people Roles: CEO Chairman Research Director Head of Legal & Patents Unit Director of Risk Management Director Human Resources Employee (1) Employee (2) Employee (3) (if necessary) Mentor (1) Mentor (2) Mentor (3) (if necessary) Marketing Director

Group 1 – approx 12 people Roles: Health Correspondent – The Independent Freelance Investigative Journalist Secretary of State for Health Government Chief Medical Officer Medical Ethics Correspondent – The Daily Telegraph Consultant Oncologist (Cancer) – St Giles Hospital Registrar (Oncology) – St Giles Hospital Director of Clinical Trials – St Giles Hospital

102

David Clemson and Khalid Samara Chief Executive & General Manager – St Giles Hospital Presenter – BBC Newsnight General Practitioner (local family doctor) Hospital Patient Next‐of‐kin of Patient

Group 2 as Observers – approx 16 people

Group 3 as Recorders – approx 15 people

There are a number of ‘triggers’ that can be used to precipitate a crisis from this scenario and allowed to play their way out until a conclusions is reached.

The critical role of reflective feedback sessions was facilitated through using a variation of Brookfield’s Classroom Critical Incident Questionnaire (CIQ) (Keefer, 2009), through which successful evidence of transformative learning was drawn as taking place when evaluated against Mezirow’s Ten Phases of Transformative Learning (Kitchenham, 2008).

This variation of Brookfield’s CIQ, specifically relating to the CMS was prepared for the class and distributed one week after the CMS. Following a brief explanation of the Crisis Management CIQ, all students present filled it in. The creation of a calm, quiet and reflective atmosphere in the classroom ensued. Although electronic circulation of the CIQ would be an option, it would not enable the atmosphere of group reflection that was created in the classroom.

The subsequent discussions in class after having filled in the CIQs have, in all cases, been fascinating, intense and, at times, uncomfortable as students began to realise the, as examples, aspects of gender‐based viewpoints and ethno‐specific judgements they had made during the simulations and were also inclined towards in everyday life. This is an example of "surfacing undiscussables" (Thomas, 2008). The IRRs provide a rich source of sense‐making narratives in this regard and the subsequent transformative perspective shifts evidence of deeper contemplative work.

3. Nature of inquiry and related literature Transformative Learning

Kezar (2000) explores the reflective circulation offered by the hermeneutic circle. While Dewey’s praxis theory considers the emergence of methodological approaches, such as action research, participatory research and collaborative research. These approaches inform the means of sense‐making in regards to "Ready‐at‐hand" or Participation; “Unready‐to‐hand” or Circumspect; and “Present‐at‐hand” or Contemplation, as ways of working‐out towards understanding and comprehension and cycling of increasing detachment and increasing involvement as discussed by Kezar. Having begun as a naturalistic inquiry (Thomas, 2008 and Braud & Anderson 1998), this inquiry was 'shaped' (Reason & Bradbury, 2001, p.427) and emerged into the domain of action research, the research cycle leading towards continuous improvement both in the sense of research process and the impact of CMS as a means of achieving transformative learning for business students, content reflecting process (Thomas, 2008).

The focus of this research centres on the usefulness of the CMS to facilitate an integrative, inter‐disciplinary learning environment in which transformative learning can occur, this is linked towards Mezirow’s Transformative Learning Theory (TLT). The first phase is a ‘disorientating dilemma’, the focus here is on the distinct aspects of this as part of the CMS, and therefore taking place in a situation characterised by high intensity, high anxiety and unpredictability – the time factor is also short, without time or space for disinterested reflection within the situation.

The focus is the nature of knowledge and power within participatory research (Reason & Bradbury, 2001, p.74‐ 76), particularly ‘awareness building’, social change and empowerment and consciousness – production of knowledge changing the awareness or worldview of those involved, linking to Freire’s notion of ‘conscientization’ being ‘learning to perceive social, political and economic contradictions – developing a

103

David Clemson and Khalid Samara critical awareness – so that individuals can take action against the oppressive elements of reality’ (Kitchenham, 2008) which underlies Mezirow’s work.

Drawing also on co‐operative inquiry, particularly Dionysian and Apollonian inquiry (Heron, 1996, p.95‐96 and Reason & Bradbury, 2001, p.183), the circuit of fourfold knowing – experiential, presentational, propositional and practical (Heron, 1996, p.169) facilitates a dynamic experience of emergence and pre‐planning both as a researcher‐participant and from the student‐participant perspective.

Within the assessment, particularly the CMS, we are also participants and observers, needing to maintain an overview of the scenario in progress and issue ‘trigger event’ instructions during the scenario. In this sense both the Dionysian and Apollonian inquiry approaches discussed by Heron (1996, p.95‐96) are present and informing of this co‐operative form of inquiry.

Crisis Situations and Sense‐making

Stein (2004) notes that research shows cases where sense‐making is essential for survival during the critical period of a disaster however he also considers the possibilities where sense‐making can, when informed by psychoanalytic theory, lead to increased problems. He investigates the capacity for anxiety toleration as a variable within sense‐making in a crisis situation. He introduces the concept of the ‘critical period’ being the period of the unfolding of the disaster. In this sense the sequence is then: ‘incubation period’, ‘triggering event’ leading to the ‘critical period’ and the ‘aftermath’.

People’s routines – as well as their customary ways of thinking – are interrupted, leading to them having little idea how to proceed. Stein cites Weick (1993) and refers to the sudden loss of meaning as a "cosmology episode" during which people 'suddenly and deeply feel that the universe is no longer a rational, orderly system'. Weick continues "…uncertainty and collapse of meaning… trying to place stimuli into some sort of framework so structure the unknown and account for discrepancies." This is an example of a form of attachment presenting itself – in this case ‘orthodox’ management being the anchor of attachment.

Real‐time, highly charged, confusing and potentially life‐threatening circumstances of the critical period: this marks sense‐making during the critical period out as different from the relative clarity of decision‐making.

Sense‐making, Weick (1993) argues, 'is built out of vague questions, muddy answers, and negotiated agreements that attempt to reduce confusion'. Sense making is therefore borne of uncertainty and is the on‐ going attempt to address it through establishing a familiar framework with which to understand it (Stein, 2004). Anxiety in Stein’s paper is explored through what is essentially a Freudian approach and the transpersonal, archetypal and depth psychologies offered by Jung and Hillman are not used.

Langley (1999) describes seven strategies for sense‐making and we anticipate that, in the context of these simulations, narrative, alternate template, visual mapping and possibly synthetic strategies would be most likely to present. The distinctions drawn are essentially guided by the attitudes to rapid processing of complex and incomplete information. Occurrence of narrative sense‐making is likely to emerge a little more slowly as a consequence of reflection particularly given that this is a suggested form for the IRR that students are required to submit.

Nature of Simulations

Seaton & Boyd (2008) review attitudes towards the use of simulations in business and management courses and draw attention to the "Simulation – No Simulation" argument and research reasons why some individuals choose to use simulations and other do not, suggesting that "a business course should be designed, not with the needs and preferences of the professor in mind, but instead, focused on the needs and preferences of the students". They note that there are learning opportunities that seem to occur only within simulation methodologies when used as assignments in addition to those within other methodologies. Specifically they note that simulation methodologies offer academic/professional environment linkage, action learning and autonomy of learning process. Simulations also offer authenticity, critical thinking, and integration of functional areas and learning of team dynamics. They highlight that one of the most common elements missing from simulations is ethical training. Commentary on the possible weakness of simulation only

104

David Clemson and Khalid Samara methodologies is given, with the suggestion that whilst they can be a very important feature of learning process, they should, in many situations, be complemented by other methodologies.

The role of CMS as a creative management education 'triggers' Low‐probability High Consequence events (Lp‐ HC) as given by Meszaros (1999). These triggers precipitate differing forms of crisis and the behavioural patterns that are likely to emerge. Management skills in environments, which are highly unpredictable due to factors such as sudden shifts in meaning, lack of information, deception and uncertainty, are examined in Stein (2004), Hunter, Bedell‐Avers & Mumford (2009) and Langley (1999).

Leadership in Crisis Situations

Hunter, et al. (2009) examine, using simulations, the impact of situational framing and complexity on three leadership types – charismatic, ideological and pragmatic. They note the observation made by a number of leadership scholars that "…outstanding leaders often emerge during times of crisis." In their summary of theorised mental‐model differences they note the pragmatic leader has a present time‐frame orientation, has an interactive locus of causation, an external focus in model construction, is malleable in terms of the nature of the nature of outcomes sought, also variable in the number of outcomes sought. In contrast to charismatic and ideological leadership types they find that pragmatic leadership types exhibit higher performance under situations of higher complexity. Therefore in crisis management situations, and in the CMS being looked at in this paper as situational frames, pragmatic leadership can be expected to show through as most effective.

Narrative Inquiry

Individual Reflective Reports are a required part of the assessment for this module and some guidance is offered in writing narratives and reflective processing. Prescriptive guidance however is avoided, highlighting the individual legitimacy of personal experience and sense‐making. Inquiry into these IRRs is approached by way of rhizomic narrative analysis (Sermijn et al, 2010 and forthcoming, Kang, 2007, and Yougblood Jackson, 2003). This naturally inclusive approach allows multiple entry points, many beginnings/endings, multiplicities (‘lines of flight’) to be examined. The work of Mallett & Wappshott (2011) on narrative identity and Beabout (2012) looking at the role of unintentional and intentional turbulence, perturbance and complexity offers further aspects of inquiry into the accounts given by participants.

4. Analysis of crisis management CIQs and individual reflective reports In the sense that 'a disorientating dilemma' is indicative of Phase 1 of Mezirow’s TLT – every student who answered the Crisis Management CIQ reached the 'edge of knowing' and thereby the possibility of beginning a transformative learning experience. This is based on 105 completed CIQs from a total of 207 students over 4 years. Subsequent analyses of IRRs show this in 192 cases.

What is less clear from CIQs is whether transformative learning experiences have occurred and whether they are 'straightforward' or 'profound'. Analysis of IRRs specifically focusing on the level of reflective thinking does however elicit this, based on the method and coding scheme developed by Kember et al. (1999).

That said, 71 CIQs = 68% illustrate self‐reflective responses indicating the learning of new meaning schemes and the learning through meaning transformation. The lasting effects of perspective shift are not captured by the CIQ and this again links to the IRR to study evidence of contemplative lasting changes in perspective that can be expected to persist over a longer time horizon. Over 4 years an incidence rate of 81% has been observed through analysis of IRRs which we would categorise as ‘profound’ rather than just ‘straightforward’.

Factors affecting this within a group setting have also been seen to emerge through group based reflections within the group assignment for this unit. This is based on identifying ‘premise reflection – learning through meaning transformation’ rather than ‘content reflection’ or ‘process reflection’ (Kitchenham, citing Mezirow, 1995). Although one aim of this research cycle has been to continuously improve CMS, the incidence rate of premise reflection has been very steady over the 4 separate cycles with 80%, 79%, 82% and 81%.

Berger (2004) notes "being on the edge is a variable experience" and, in consequence of this, student experience from participating in the simulations should be variable and not necessarily pleasant. From the CIQs and IRRs this is supported, with a 71% incidence rate over 4 years, reporting unpleasant experiences with

105

David Clemson and Khalid Samara reframing through reflection. Coincident with this were strong experiences relating to power and knowledge – both positive and negative and uncertainty and anxiety caused by not knowing what was going on.

Situations where ‘orthodox’ management methods do not work ("cosmology episodes" and the collapse of meaning (Stein, 2004)) tended to be more visible at a distance – the sense of being 'lost in the situation' – was certainly evident. Deeper reflections emerged from the student perspective in their IRRs with an incidence rate of 74%. Examples were reported of wild oscillations in scenarios, conflicting themes causing breakdowns in effective communications.

5. Conclusions and key findings As Berger (2004) notes "in the busy world of teaching, there is rarely time to listen deeply to the sense students are making – and to the ways they are not making sense, even to themselves. Charting the path of transformation – and paying attention to its most fertile ground – allows us to be better company for students who are working through their own transformative experiences." This series of assessments are one way of enabling students to begin their journey. As Berger suggests, "…slow down and honour transformative spaces…" and this is a principal aim of the IRR. This is where the circumspection shifting to contemplation and subsequent informing of participation shows through in premise reflection.

With incidence rates of deeper‐level transformative learning of 81% within a population of N=207, this is higher than those reported in Snyder (2008).

As this research is ongoing, the authors would like to suggest that these emergent research themes from their work as outcomes for further investigation:

Do pragmatic leadership skills emerge through crisis management simulations as being the most effective leadership style?

Is there any evidence of rapid mimesis (the imitation of actions of others) which can be an efficient response to uncertainty and a means of ensuring performance is no worse than others? This is an interesting phenomenon to explore given the situation is characterised by high stress and high anxiety and compared to lower stress/lower anxiety environments.

Anxiety and Leadership in CMS – an exploration of the dynamics from a transpersonal, transformative psychological standpoint informed by the work of Jung and Hillman (Hillman, 1995).

Ethical conduct within business CMS – student‐centred approaches to generating codes of conduct and ethical standards. Investigating levels of self‐discipline and adherence to group‐generated participatory codes compared to externally imposed regulation. Within the settings of UN PRME and UNGC these are particularly relevant and offer governance frameworks for content, process and premise reflection.

Through a cycle of addressing the issues of ethics, sustainability and risk management in a dynamic student‐ centred way enhancing their ability to handle complex situations which they could face in real‐world business it can be concluded that CMS is an effective method of creating transformative learning environments with larger groups, especially when used in conjunction with reflective processing. Of longer term interest is how these collections of narratives of transformation are carried forwards, on an individual level and as a collective ensemble, their permeating into further academic study and into professional engagement and individual careers. The enthusiasm with which participants in earlier CMS want to return (often with employer support) to play parts in these CMS suggests that the deeper level transformative learning continues.

References Beabout, B. (2012); Turbulence, Perturbance, and Educational Change. Complicity: An International Journal of Complexity and Education, 9(2), 15‐29 Berger, J. (2004). Dancing on the Threshold of Meaning: Recognizing and Understanding the Growing Edge. Journal of Transformative Education, 2(4), 336‐351.

Braud, W. & Anderson, R. (Eds). (1998). Transpersonal Research Methods for the Social Sciences: Honouring Human Experience. London: Sage.

Clark, M.C. (1991). Restructuring of meaning: An analysis of the impact of context on transformative learning. Georgia: University of Georgia.

Cloudman, R. & Hallahan, K. (2006). Crisis Communications Preparedness among US Organizations: activities and assessments by public relations practitioners. Public Relations Review, 32(4), 367‐376.

106

David Clemson and Khalid Samara DeLanda, M (2002); Intensive Science and Virtual Philosophy; London, Continuum Books DeVries, D. & Fitzpatrick, K. (2006). Defining the Characteristics of a Lingering Crisis: Lessons from the National Zoo. Public Relations Review, 32(2), 160‐167.

Dyl, J. (2009). Lessons from History: coastal cities and natural disasters. Management of Environmental Quality: an International Journal, 20(4), 460‐473.

Elias, D. (1997). It’s time to change our minds: An introduction to transformative learning. ReVision, 20(1), 2‐5.

Greyser, S. (2009); Corporate Brand Reputation and Brand Crisis Management. Management Decision, 47(4), 590‐602 Harvey, J. (1988). The Abilene Paradox: the management of agreement. Organizational Dynamics, 17(1), 17‐43.

Heron, J. (1996). Co‐operative Inquiry: Research into the Human Condition. London: Sage.

Hillman, J. (1995); Kinds of Power; New York, Currency Doubleday Hunter, S., Bedell‐Avers, K. & Mumford, M. (2009). Impact of Situational Framing and Complexity on Charismatic, Ideological and Pragmatic Leaders: Investigation using a computer simulation. The Leadership Quarterly, 20(3), 383‐ 404.

Kang, D J (2007); Rhizoactivity: Towards a Postmodern Theory of Lifelong Learning. Adult Education Quarterly, 57(3), 205‐ 220 Keefer, J. (2009). The Critical Incident Questionnaire (CIQ): From Research to Practice and Back Again. Adult Education Research Conference Proceedings, January 2009, 177‐182 Kember, D., Jones, A., Loke, A., McKay, J., Sinclair, K., Tse, H., Webb, C., Wong, F., Wong, M. & Yeung, E. (1999).

Determining the Level of Reflective Thinking from Students’ Written Journals using a Coding Scheme based on the work of Mezirow. International Journal of Lifelong Education, 18(1), 18‐30.

Kezar, A. (2000). The Importance of Pilot Studies: Beginning the Hermeneutic Circle. Research in Higher Education, 41(3), 385‐400.

Kitchenham, A. (2008). The Evolution of John Mezirow’s Transformative Learning Theory. Journal of Transformative Education, 6(2), 104‐123.

Klein, N. (2007). The Shock Doctrine: The Rise of Disaster Capitalism. Canada: Knopf.

Langley, A. (1999). Strategies for Theorizing from Process Data. Academy of Management Review, 24(4), 691‐710.

Mallett, O. & Wapshott, R. (2011); The Challenges of Identity Work: Developing Ricoeurian naarative identity in organisations. ephemera – theory and politics in organisation, 11(3), 271‐288 Meszaros, J.D. (1999). Preventative Choices: Organizations’ Heuristics, Decision Processes and Catastrophic Risks. Journal of Management Studies, 36(7), 977‐998.

Mezirow, J. (1995). Transformation Theory of Adult Learning. In: In Defense of the Lifeworld. (Eds). by Welton, M.R. p.39– 70. New York: SUNY Press.

Mezirow, J. (2000). Learning as Transformation: Critical Perspectives on a Theory in Progress. San Francisco: Jossey‐Bass.

Newmann, M. (2010). The bottom line is not enough: how business schools fail their students. Times Higher Education, 7 January, 17.

Pecora, V. (2002). Culture of Surveillance. Qualitative Sociology, 25(3), 345‐358.

Reason, P. & Bradbury, H. (Eds). (2001). Handbook of Action Research: Participative Inquiry & Practice. London: Sage.

Seaton, J. & Boyd, M. (2008). The Effective Use of Simulations in Business Courses. Academy of Educational Leadership Journal, 12(1), 107‐118.

Schlamm, L. (2001); Ken Wilber’s Spectrum Model: Identifying Alternative Soteriological Perspectives; Religion, 31, 19– 39 Sermijn, J., Loots, G. & Devlieger, P. (2010); Wolves in Sheep’s Clothing or Sheep in Wolf’s Clothing? Creative Approaches to Research, 3(2), 39‐51 Sermijn, J., Loots, G. & Devlieger, P. (forthcoming); In Dialogue About the Self: from a Modern Cartesian Self to Alternative Views on Selfhood, Human Studies Snyder, C. (2008). Grabbing Hold of a Moving Target: Identifying and Measuring the Transformative Learning Process.

Journal of Transformative Education, 6(3), 159‐181.

Stein, M. (2004). The Critical Period of Disasters: Insights from sense‐making and psychoanalytic theory. Human Relations, 57(10), 1243‐1261.

Thomas, G. (2008). When Research Content Reflects Research Process: unexpected but welcomes symmetries. Qualitative Research Journal, 8(2), 2‐13.

Van Fleet, D. & Van Fleet, E. (2006). Internal Terrorists: the terrorists inside organizations. Journal of Managerial Psychology, 21(8), 763‐774.

Weick, K.E. (1993). The collapse of sense making in organizations: The Mann Gulch disaster. Administrative Science Quarterly, 38(4), 628–652 Youngblood Jackson, A. (2003); Rhizovocality. Qualitative Studies in Education, 16(5), 693‐710.

107

Developing a General Analytical Inductive Qualitative Research Strategy to Investigate Business School ‐ Industry Collaboration Fariba Darabi and Murray Clark Sheffield Business School, Sheffield Hallam University, UK f.darabi@shu.ac.uk

Abstract: This paper examines the development of a general analytical inductive approach to qualitative research. It critically assesses the research design and analytical processes that facilitated the development of a framework for understanding the initiation stages of collaborative partnerships between Business Schools and Industry. An interpretivist approach was adopted to understand (verstehen) how people make sense of their world (Gill and Johnson, 2010). The research design followed Thomas (2006), and the outcomes of this paper demonstrate how an analytical inductive analysis, one that refers to detailed readings and interpretations of raw data, can be rigorously used to drive the identification of concepts, themes, and models. In developing the analytical approach, data from both the Business School and SMEs was gathered and the 24 interview transcripts were analysed producing a provisional list of common features. Similarities between categories were established and deviant features accommodated either by linking them with other common features or by generating a new category with unique features. Eventually, cross case analysis within the groups and between groups at the business school and SMEs occurred and a number of themes emerged which form the basis of a proposed model for initiating collaboration, one linked with Vangen and Huxham’s (2003) ‘trust building loop’. Through a practical application of the research design, data collection and analytical approach, the paper demonstrates the credibility of a general analytical inductive research strategy as a qualitative research methodology; a strategy that promotes the linking of theory and practice. The benefits from the research are seen as facilitating the development of effective business relationships between local universities and SMEs in their regions.

Keywords: qualitative research methods, collaborative relationships & business development, general analytical induction

1. Introduction This paper examines the development of a general analytical inductive approach to qualitative research. It critically assesses the research design and analytical processes that facilitated the development of a framework for understanding the initiation stages of collaborative partnerships between Business Schools and Industry.

Corporate demands are increasingly leading organizations to engage in ‘partnerships’ for the advancement of ‘collaborative advantage' and the higher education sector is no exception in seeing this as a key focus for its business activities (Salter, A., Tartari, V., D’Este, P. and Neely, A. 2010). The continuing need to develop practice–oriented theory into the management of inter‐organizational collaboration has led many researchers to focus on exploratory research and this paper assess the contribution that an interpretive research methodology, based on analytical inductive principles, can make to praxis. The methodological approach embodied Analytic Induction, the principles of which focus on the generation of theory from the observation of the empirical world of the participants (Johnson, 2004). It facilitated the research need to develop a practice‐based theory concerning the role of universities in building knowledge transfer between the sectors.

2. Interpretive approach in developing business relationship Creswell (1994, 2003) and Guba (1990) argue that most of the research paradigms share three fundamental elements: ontology, epistemology, and methodology. Ontology concerns the nature and form of knowledge in the physical world, whilst, epistemology concerns the source of knowledge, or the ways of knowing knowledge. The actors in the two sectors were the source of knowledge for this study. The way in which the researcher gained access to the knowledge or social reality was through interviewing the actors (Alvesson, 2011). The third fundamental element is methodology; that concerns the rationales behind the procedures used to research what it is believed it is possible to be known (Creswell, 1994; Guba and Lincoln, 1994). Since the researcher philosophical position can shape the research design, therefore, knowing how the researcher explores the reality of the relationship between the Business school and SMEs was important in order to adopt the appropriate method to study the phenomenon.

Research can be undertaken through number of research paradigms that shape the research design. It was possible to take a positivist, post positivist, interpretive, critical theory, constructionist or a postmodernist

108

Fariba Darabi and Murray Clark approach (Lincoln and Guba, 2000) to study a business relationship between UBS and regional SMEs in this research. The inductive/interpretive approach was undertaken because the researcher believes that the way that people behave is based on their interpretation of a phenomenon. The detailed rational behind this selection is now discussed by comparing the positivist and post‐positivist paradigms.

The positivist paradigm follows an objectivist epistemology (Crotty, 1998), i.e. the way of knowing the knowledge is in an objective manner by measuring or scaling a phenomenon, therefore, it looks at a phenomenon quantitatively and also places an emphasis on the explanation of human behaviour (Bryman and Bell, 2011) as necessary responses to the action of empirically observable, measurable stimuli, causal variables or antecedent conditions (Gill, Johnson and Clark, 2010). This kind of explanation is erklaren (Outhwaite, 1975). In contrast, neo‐positivists would argue that the actors’ subjective realm is not only important to our theoretical explanation of their behaviour but it is also possible to access it, describe it, and hence theoretically use it to explain aspects of human behaviour, in an objective manner. Despite the continuing dominance of quantitative determinist approaches, the qualitative methods used by neo‐positivists to pursue their interpretive agenda enable them to increasingly assert its relevance to business and management research (Johnson & Clark, 2006).

We agree, therefore, with Gill & Johnson (2010) that the actors have subjective abilities, both emotional and cognitive, which influence how we consciously make choices about how to behave, where and when. How people behave, therefore, is based on their perceptions and interpretations. This research is a social research, the central question for the researchers was to know 'what is happening' between the business school and SMEs in terms of their relationships. We aimed to get a deeper understanding of the actors in their relationships and focused on the perception of the stakeholders about their relationships, therefore, the focus is on an understanding of human behaviour and the need to follow a neo‐positivist approach and the adoption of qualitative methods.

Of key importance here is the neo‐positivist commitment to verstehen which is premised upon the idea that to follow the positivist approaches in the study of the social world is an error because human action relies upon an internal subjective logic which must be understood in order to make sense of what happens in practice (Johnson & Clark, 2006). So as researchers we must access the individuals' internal culturally derived logics in order to explain their behaviours whilst believing that as researchers we can objectively understand how others make sense of the social world. The term neo‐positivist is thus used for management research where the collection of qualitative empirical data is used for the inductive generation of theory ‘grounded’ in observation. The result is a separation of the knower‐researcher from their inductive descriptions of other actors’ inter‐subjective cultural experience which awaits discovery (Johnson & Clark, 2006, p.xxxii).

Thus, the social world cannot be understood in terms of causal relationships that do not consider a situation that human actions are based upon the actor’s interpretation of events, social meanings, intentions, beliefs; i.e. human action is understandable by knowing these subjective dimensions and their operations in specific social context (Gill and Johnson, 2010). Therefore, in this research, we endeavoured to interpret actor's interpretation on responses. For example, actors’ at both the university and SMEs' interpret the relationship differently because they respond based on their own cultures; beliefs and values which led them to behave in a certain way. As a result of this it was taken that a rational, positivist approach would not match with the phenomenon of study and the purpose of this research, whilst, an interpretivist approach would. Again because this study, as an interpretive study, assumes that people create and associate their own subjective and inter‐subjective meanings as they interact with the world around them. Interpretive researchers thus attempt to understand phenomena through accessing the meanings participants assign to them (Orlikowski and Baroudi, 1991).

The researchers here were trying to make sense of how actors make sense of their relationships. Gill and Johnson (2010) argue that the aim of interpretivist approaches is to understand (verstehen) how people make sense of their worlds. Interpretivists (Blaikie, 2007) argue that the study of social phenomena requires an understanding of the social world that people have constructed and which they reproduce through their continuing activities. People are constantly involved in interpreting and reinterpreting their world ‐ social situations, other people's actions, their own actions, and natural and humanly created objects (Blaikie, 2007).

Hence Individuals in SMEs and Universities were thought likely to view the phenomenon of collaboration differently and hence an interpretivist analysis (Denzin and Lincoln, 2000) was adopted. In this study the

109

Fariba Darabi and Murray Clark position taken is that the notion of relationship management in collaborations could be understood through an understanding or the meaning of the concept from those involved in this form of social action. In order, therefore, to develop an understanding of the interpretations deployed by the actors who were being studied (c.f. Shotter, 1975; Giddens, 1976; Denzin and Lincoln, 2000) an interpretivist approach was adopted for this study because understanding of human behaviour is concerned with the perceived understanding of human action rather than with the forces that are supposed to act on it (Bryman and Bell, 2011).

From the researchers’ point of view the subject of this study is socially constructed by individuals and it needs to be understood from the point of view of the minds of the individuals who are directly involved in it. As Mead (1934) argues human have a sense of self that they develop through interactions with others‐ 'through senses of self that we construct and the actions that we take towards objects in our world". Blaikie (1993, p.96) states that interpretivism 'entails an ontology in which social reality is regarded as the product of processes by which social actors together negotiate the meanings for actions and situations'. This argument is in line with Walsham (1993), in that, that interpretive methods of research start from the position that our knowledge of reality, including the domain of human action, is a social construction by human actors and that this applies equally to researchers. Here interpretive ontology, therefore, is internal realism where reality is seen an inter‐subjective construction of the shared human cognitive apparatus (Walsham, 1995). According to Blumer (1969) meaning arises from social interactions. Therefore, the reality is different from each actors point of view in this research i.e. interpretation of the participants at Universities and SMEs is different from one another because they have different experiences of their relationship and they perceive the reality based on their understanding. Thus, the reality about the relationship between these two sectors is out there, the researcher gaining access to actors' reality by exploring their views and then giving her personal understanding to participants' thinking by interpreting their views. Following Bulmer's (1969) notion of symbolic intercationism, Gill & Johnson (2010) argue that meaning is handled and modified through an ongoing interpretive process, i.e. meanings are not fixed, and it is a process.

In interpretivist research, in understanding the world, researchers must engage and participate in it (Hatch, 1997). In practicing this paper as an interpretivist inductive research, we were in a privilege position to stand back and listen to the actors' perception about their relationship during the data collection therefore the knower and the known (Johnson and Duberley, 2000) were separate thus we were not bias and didn't influenced the participants' views and later at data analysis stage (data interpretation) we engaged with data and vigorously interpret them. Hence what we call our data are really our own constructions of other people's construction of what they and their compatriot are up to (Geertz, 1973).

The central principle of interpretivism is that there is fundamental different between the subject matters of the natural and the social sciences which makes interpretive method different, because nature has to be studied from the 'outside', whereas social phenomena have to be studied from the 'inside' (Blaikie, 2007). This is in the line with Gill & Johnson (2010) that natural scientists impose an external causal logic for explaining a behaviour which is inappropriate in explaining human behaviour. For example, in collaboration between SMEs and the University it needed to be studied from inside, why academic behave in a certain way in the collaboration with businesses i.e. looking for an internal logic for the way that academic behave. Therefore qualitative research is fundamentally interpretive; this means that the researcher makes an interpretation of the data. This includes developing a description of an individual or setting, analyzing data for themes or categories, and finally making an interpretation or drawing conclusions. Therefore, the qualitative researcher views social phenomena holistically and systematically reflects on who s/he is in the inquiry (Creswell, 2003).

3. Qualitative research approach It is common to talk about two approaches to research design ‐ quantitative and qualitative methods.

Polarising research as either qualitative or quantitative may be seen as a useful 'heuristic device' or an aid to learning, however, this differentiation is at risk of oversimplifying and obscuring the subtleties of research methodological design. Notwithstanding this, in this paper we intend to use the term qualitative research to representing the inductively driven interpretivist approaches characterised by neo‐positivism

The Quantitative analytical approach has its basis in positivism, and having an objectivist conception of social reality it has a distinctive objective epistemological position which stresses the importance of measuring and quantifying phenomena (Bryman and Bell, 2011). If researchers are interested in finding the cause and effect

110

Fariba Darabi and Murray Clark relationship in a phenomenon the quantitative approach may be appropriate (Bryman, 2004). However, it was not the aim of this research to establish a cause‐effect relationship between the stakeholders' views in the business school and SMES collaboration. Neither did we want to quantify the actors’ attitude and values with respect to developing their relationships.

The qualitative approach tends to be concerned with words rather than numbers. For example, in a qualitative interpretivist approach the stress is on the understanding of the social world through an examination of the interpretation of that world by its participants (Bryman and Bell, 2011). Through Verstehen, qualitative methods aim (See Van Maanen 1998; Alvesson and Deetz, 2000; Denzin and Lincoln, 2000) for an understanding of other’s experience by inductively accessing the actual meanings and interpretations they subjectively and inter‐subjectively deploy in making sense of their worlds and which influence their ongoing social construction and accomplishment of meaningful action (Gill & Johnson, 2010: 62). The qualitative approach allows researchers to capture data on ‘the perception of respondents in the context of their setting, through a process of attentiveness and empathetic understanding’ (Miles and Huberman, 1994:6). In this study qualitative research allowed the researcher to get a deep understanding of the personal views of strategic and operational managers at University and SMEs as decision makers in their organisations regarding to their relationships; why do they need the collaboration, what could trigger them to approach each other and getting involve in a collaboration etc. Therefore, that will help us to get a rich picture of the stories behind each relationship thus very qualitative detailed data will be collected.

4. Research method As explained above, a qualitative research method was adopted for this study. According to Creswell (2003) the qualitative researcher views social phenomena holistically. This explains why qualitative research studies appear as broad views rather than micro‐analysis. It can give researchers a holistic view of the whole phenomena under investigation. There are different ways of collecting qualitative data, such as observation, interviews, documents, audio‐visual materials (Creswell, 2003). Of these the interview is the most widely employed method in qualitative research and is attractive to researchers mainly because of its flexibility (Bryman and Bell, 2011).

4.1 Interview technique The Interview technique was the main method used for collecting data in this research. The type of interview was semi‐structured, in‐depth and face‐to‐face: It was our concern and interest to get a detailed and deep picture of the interviewee's perceptions and understanding about the interactive process of the collaboration.

Semi‐structured interviews are widely recognized as being able to facilitate this aim and as Silverman (2005) stresses 'qualitative interviews provide us with a means to explore the points of view of our research subjects'.

Through the interview session, the focus was on finding the history of SME relationships with the University.

The advantages of this method are participants can provide historical information and it, also, allows researcher 'control' over the line of questioning (Creswell, 2003). It allows us to fully explore the topic from the respondent's perspective (Doole, 2000). As Denzin and Lincoln (2005) argue the interview is a conversation, it is not a neutral tool, for at least two people create the reality of the interview situation. We gained historical and in depth information about the successful and unsuccessful examples of working between the two sectors by asking actors at the two sectors about the stories of their current and past relationships.

4.2 Data collection method 4.2.1 Purposive sampling technique A purposive sampling procedure was followed in the selection of the organizations and the participants in the research. Creswell (2003) argues that the idea behind interpretive research is to purposefully select participants that are viewed as most likely to help the researcher understand the problem and the research question. Thus, in this research, purposive sampling demanded that the researchers thought critically about the parameters of the population under study and chose the sample cases carefully on this basis (Silverman, 2005). In this manner, a number of SMEs, already interacting with the Business School, were identified. The organization sample was then selected to represent a range of businesses based on the type of the services that they receive from the business school.

111

Fariba Darabi and Murray Clark As a result of purposive sampling approach, four more interviewees were identified as potential participants.

For example, Sales managers interviewed in SME 'A' and 'D' because they recommended by their managing directors to participate in the research because they were involved in interacting with the business school. The following table 1 illustrates data collection samples from SMEs and University sector.

Table 1: Data collection samples SME

Business

Participants

A

B

Service Manufacturing

C

Energy Consultancy

D

Retail

Managing Director Sales Manager Managing Director Operational Director Managing Director People Solution Director Managing Director/Sales Manager

E

Consultancy

Managing Director

F

Electronic and Engineering

G

Manufacturing

H

Manufacturing

University

Business School

Role Executive Management Senior Management

Knowledge Transfer Projects

Consultancy Projects

Managing Director Technical Manager Operational Director HR Advisor Project Manager

Participants Knowledge Transfer Pro‐vice Chancellor Assistant Dean, Employer Engagement Business Development Manager Business Development Officer Head of Food and Innovation Centre Knowledge Transfer Champion KT Relationship Manager KT Project Manager Consultancy project Module Leader Project Manager International Project Manager

In total twenty‐four semi structured interviews were conducted with thirteen managers in the SMEs and with eleven academic managers in the Business School.

The participants were chosen from strategic and operational levels, for example, managing directors and operational directors in the SMEs and senior management and project managers at the university and business school. As decision makers, it was important to gain the views of senior strategic management at the business school and within the businesses with respect to the development of effective relationships. In a similar way, it was important for the researcher to gain access to the perceptions of project managers and operational directors as these could give key pictures of the relationships.

D'Este and Patel (2007) and Bruneel, D'Este, and Salter (2010) examined the broad range of channels of interaction such as 'joint research projects', 'contract research', 'consultancy', 'training of firm employees', 'postgraduate training in the company', 'recruitment of recent graduates or postgraduates', and 'student placements'. Building on D'Este and Patel's research, this study considered 'knowledge transfer projects' and 'consultancy projects' as a channel of interaction between UBS and SMEs.

Drawing on the themes derived from the literature (for example, the organizational processes as barriers to developing collaboration, and the need to create a knowledge sharing culture based on trust; Marzo‐Navarro et al. 2009; Vangen & Huxham, 2003 ) a list of interview questions were designed based on two main themes; Relationship Management and Collaboration in business relationships, to explore how individuals make sense of their business relationships in order to give the researcher a deep insight of each theme. The participants were asked to state their role, the history of their involvement, and some of their experiences working within the partnership. Therefore, the questioning attempted to develop a picture of the story behind the relationships, for example; ‘How did you start the relationship with _____?’, ‘Do you have any successful or

112

Fariba Darabi and Murray Clark unsuccessful experience of working with ____’?, ‘Why do you want to develop the relationship?’, ‘What are the perceived advantages of working with ____ ?’,’How can things be improved?

The sequence was not rigidly followed instead the sequence was varied in order to pick up on specific, however, it was ensured that all the questions were answered by the interviewees. The same questions were asked with all the interviewees. Through this type of semi‐structured interview, interviewees were encouraged to talk about what they were experiencing and what they thought were key issues. It gave a chance for the researcher to clarify the questions and answers and also ask new questions followed up interviewee's replies through the interaction between the researcher and participants to get a rich picture of each interviewee.

Whyte (1953:22) notes that no single interview stands alone, it has meaning to the researcher only in terms of other interviews and observations.

5. Data analysis approach and procedure As social science research, there is a need to focus the analysis on which explanations of human action are generated inductively during data collection in order to develop an understanding of the interpretations deployed by the actors who are being studies (see Shotter, 1975; Giddens, 1979; Denzin and Lincoln, 2000).

The aim of the research was to contribute to the understanding of how university business schools initiate, develop and manage their inter‐organisational relations with SMEs; and what, if any, are their mutual advantages for business schools and SMEs to work collaboratively. As the research progressed, it soon became clear that the data suggested the key issue was how to initially start the relationship and the trust this became the central phenomenon of interest and it was decided to adopt a general inductive analytical process for the interpretation of the data.

5.1 General analytic induction The method of induction is the process of proceeding from particulars to the general‐ universals (Locke, 2007).

Its process starts with an observation or something that is puzzle and needs exploration, for example, a general question, and ends up with a new theoretical conceptualisation of the issues. Taking this approach in social research means that we are attempting to generate theory at the end of the research. Generalization is questionable in this approach because according to Bryman (2008), the scope of the findings of qualitative investigation is restricted, it is impossible to know how the findings can be generalized to other settings. Can just one or two cases be representative of all cases? The answer is no, but the findings of qualitative research are to generalize to theory rather than the population. It is the quality of the theoretical inferences that are made out of the qualitative data that is crucial to the assessment of generalization.

However, Gill and Johnson (2010) argue that human beings are able to attach meaning to the events and phenomena that surrounded them. Therefore, examining people at SMEs and UBS could have different understanding and different views of their relationships because they are from different organisational contexts and have different experiences of relationship with each other. It also provided good contrasts and comparison and thereby confronts the emergent theory with the patterning of social events under different circumstances (Johnson, 2004). McCracken (1998) believe that the object of analysing qualitative data is to determine the categories, relationships and assumptions that inform the respondent's view of the world in general, and the topic in particular.

As Johnson (2004) claims analytic induction is a set of methodological procedures that tries to generate theory grounded in the observation. This approach shaped the researcher's thoughts in applying an Analytic Induction approach for the data analysis. The following figure illustrates the processes.

In developing the analytical approach, data from both sectors was gathered and the interview transcripts were analysed, producing a provisional list of some common features and deviant cases identified. Then similarities between categories were established. Deviant features were accommodated either by linking them with other common features or by generating a new category with unique features. Eventually, cross case analysis within the groups and between groups at the business school and SMEs occurred and a number of themes emerged from the data and a tentative model of initiating collaboration, and also initiating trust building that is linked with Vangen and Huxham's (2003) ‘trust building loop’ were suggested.

113

Fariba Darabi and Murray Clark

Data Senior Managers such as University KT Pro‐Vice Chancellor

SMEs in Relationship with UBS

Departments with established relationship with SMEs

Business School

Pre‐planned Themes

pre‐planned themes

pre‐planned themes

Pre‐planned Themes

Provisional list of features

Provisional list of features Deviant

Common Features

Common Features

Create New Themes/Categories

Create New Themes/Categories

Generate theory/ies grounded in data

Deviant

Cross‐Case Analysis within and between groups at SMEs and UBS

Associate shared features with existing literature

Figure 1: Data analysis approach; analytic induction (Johnson, 1998 adapted from Bloomer, 1997)

5.2 Data analysis procedure Thomas (2006) argues that many of the underlying assumptions and procedures associated with qualitative data analysis are related to specific approaches or traditions, such as grounded theory (Strauss & Cobin, 1998), Phenomenology (e.g. Van Maanen, 1990), discourse analysis (e.g. Potter and Wetherell, 1994) and narrative analysis (e.g. Leiblich, 1998). However, a much used strategy in qualitative data analysis is the 'general inductive approach' (Bryman and Burgess, 1994; Dey, 1993). Thomas (2006) states that the inductive approach is a systematic procedure for analysing qualitative data and is one in which the analysis is guided by specific evaluation objectives. It refers to detailed readings of the raw data, and this drives the identification of concepts, themes, or the identification of a model through interpretations made from the data by the researcher (Thomas, 2006). The researcher begins with an area of study and allows the theory to emerge from the data, thus building an understanding of data analysis and theory in a manner that is consistent with Strauss and Corbin’s (1998) grounded theory methodology. This approach was adopted in this study. We were following a systematic procedure and the aim was to reduce the mass of raw data, through coding and categorizing it, such that clear links between the research objectives and the findings could be derived, whilst ensuring that these links were both transparent and defensible.

Audio file Interviews from the both sectors, that is, the University and SMEs, were transcribed into Microsoft word files and data analysis followed an iterative, constant comparative approach until data saturation occurred, or according to Glaser and Strauss (1967:61) 'theoretical saturation' happened, i.e. 'where no additional data was found whereby the [researcher] could develop properties of the category'. In other words we were faced with repetition in the answers to the interview questions and no new relevant data was presenting by the participants. Lincoln and Guba (1985) state that the process of coding can be finalised when the categories are saturated, incidents can be readily classified, and sufficient repetition occurs in the data.

Thus, new data did not add anything to the development of the categories and the created model. For example, when participants at the business school responded to 'why do you need this type of relationship with SMEs the answers were almost the same and there was no need to alter the categories which were already created. Therefore, data collection ceased and the last interview had been conducted. Silverman (2005) argues that using transcription is a form of data analysis. Therefore audio files were transcribed word

114

Fariba Darabi and Murray Clark for word and typed into Microsoft word files. The process of transcription was time consuming; however, it enabled the researcher to get very close and familiar to the content of data. Each transcription was read through several times. Through reading the raw data the researcher had two options for managing and analysing the data, either manually or electronically (i.e. using specialist software).

The first trial was manual, that is, the researcher read through the first transcription a few times to understand what the sentence, phrase or even a paragraph was about and then wrote each of them on a ‘post it note’ and gave each of them a code based on the researcher's interpretation of the quote. These were then stuck to 'flipcharts', and the ‘post its’ moved around the flipcharts as necessary in order to form constructs, where 'post its' with similar context were clustered together. As the nature of qualitative data is bulky it soon became hard to find enough space to hang the flipcharts and it was also too messy. Therefore, the researcher decided to use NVivo software to help manage data. All of the transcriptions imported to NVivo software simultaneously while data collection was still on process. The following two figures illustrate the data analysis procedure and emergent themes and sub‐categories from the data.

Figure 2: Data Analysis procedure; followed by Thomas (2006) Emerged Themes and Categories

Relationship Management - Characteristics of a successful relationship - Communication Channel - Building Relationship - Relationship Management Approaches and Characteristics

Conclusions

- The awareness and understanding of relationship management is critical to IOR management.

- Relational approach with the focus of interpersonal relationship is the best approach in managing UBS/SMEs collaboration.

- Intensive communication is required in developing collaboration because it can start to initiate trust.

Collaborative Opportunities and Challenges - Structural and Mindset Differences - Cultural Disparity - Engagement - Resources - Managing Resources - Awareness and Understanding - Purposes and Advantages of Involvements - Motivations

The Role of Trust - Characteristics of Trust - Definition and Attitude to Trust - Levels of Trust - Trust Building Approaches

- The awareness and understanding of collaborative opportunities and challenges is required to define the best strategy in initiating collaboration.

- Trust plays a crucial role in such collaboration, as it is mechanism to overcome to some of the collaborative - Competent-based trust with personal level of trust is the best kind of trust in developing collaboration between University/SMEs - The best practice in initiating trust is through managing SMEs’ expectation through an understanding of their needs, then customize programmes which suits their industry.

- the other way of initiating trust is through integrity i.e. delivering the promises. Networking also is a huge part of initiating and building practicebased trust with SMEs

Figure 3: Emerged themes and categories, and conclusions

115

Fariba Darabi and Murray Clark

6. Conclusions Specifically, this paper focused on the design and critique of a general qualitative inductive research approach.

The conceptual justification for the qualitative approach adopted was based on a neo‐positivist theoretical perspective premised on a commitment to verstehen. The term neo‐positivist was used here to represent approaches to management research where it is believed we can objectively understand how others make sense of the social world and where the collection of qualitative empirical data is used for the inductive generation of theory ‘grounded’ in observation. The emphasis on the possibility of being able cognitively accessing knowledge (truth about) of the social world is central to such qualitative grounded methods and this objectivity is often discounted by researchers who 'mistakenly' refer to general inductive qualitative research as engaging with a subjective epistemological assumption.

Through a practical application of the research design, data collection and analytical approach, the paper demonstrates the credibility of a general analytical inductive research strategy as a qualitative research methodology; a strategy that promotes the linking of theory and practice.

The benefits from the research are seen as facilitating the development of effective business relationships between local universities and SMEs in their regions. In this manner, key benefits for companies from collaborative partnerships will be gained through enhancing management development activities in order to change business behaviours.

References Alvesson, M. (2011) Interpreting Interviews. London: Sage Publications.

Alvesson, M. and Deetz, S. (2000). Doing Critical Management Research. London: Sage Publications.

Blaikie, N. (1993) 'Approaches to Social Enquiry'. Cambridge: Polity Press.

Blaikie, N. (2007), 'Approaches to Social Enquiry', Sage Publications, Second Edition Blumer, H. (1969) Symbolic interactionism. Englewood Cliffs, NJ: Prentice‐Hall.

Bruneel, J., D'Este, P. and Salter, A. (2010) "Investigating the factors that diminish the barriers to University‐Industry collaboration". Research Policy Vol. 39, Issue 7, pp. 858‐868.

Bryman, A. (2004) Social Research Methods. Oxford: Oxford University Press. nd Bryman, A. (2008) Social Research Methods. 2 ed., Oxford University Press. rd Bryman, A. and Bell, E. (2011) Business Research Methods. 3 ed., Oxford University Press Bryman, A. and Burgess, R. (1999) Qualitative Research. Thousand Oaks, CA: Sage Publications.

Creswell, J.W. (1994). Research Design: Qualitative and Quantitative approaches. London: Sage Publications.

Creswell W.J. (2003), 'Research Design; Qualitative, Quantitative, and mixed Methods Approaches', Sage Publications Ltd. Second Edition.

Crosby, L., Evans, K., and Cowells, D. (1990), 'Relationship quality in services selling: an interpersonal influence perspective', Journal of Marketing, Vol. 54No. 3, pp.68‐81 Crotty, M. (1998), The foundations of social research: meaning and perspective in the research process. London: Sage Publications.

Denzin N. and Lincoln Y. (2000), 'The sage Handbook of Qualitative Research', Sage Publications, Third Edition, rd Denzin, N.K. and Lincoln, Y. (2005) The Sage Handbook of Qualitative Research. 3 ed., Thousand Oaks, CA: Sage Publications.

D'Este, P. and Patel, P. (2007) "University‐Industry linkages in the UK: what are the factors underlying the variety of interactions with industry?" Research Policy Vol. 36, Issue 9, pp. 1295‐1313.

Dey, I. (1993) Qualitative data analysis: A user‐friendly guide for social scientists. London: Routledge.

Doole, I. (2000). How small and medium sized enterprises learn to compete effectively on international markets. PhD Thesis, Sheffield Hallam University.

Geertz, C. (1973). The interpretation of Cultures. New York: Basic Books.

Giddens, A. (1976) New Rules of sociological method. London: Routledge.

Gill J., and Johnson Ph., and Clark, M.C. (2010), 'Research Method for Managers', Fourth Edition, Sage.

Glaser, B and Strauss, A. (1967), 'The Discovery of Grounded Theory', Aldine Publishing Company, Hawthorne, NY. Goh, C. S., (2002), 'Managing effective knowledge transfer: An integrative framework and some practice implications. Journal of Knowledge Management, Vol. 6, No. 1, PP 23‐30 Guba, E. (1990) The Paradigm Dialog. Beverly Hills: Sage Publications.

Guba, E. and Lincoln, Y. (1994) “Competing Paradigms in Qualitative Research”. In: Denzin, N. and Lincoln, Y. (eds.), nd Handbook of Qualitative Research (2 ed.), Thousand Oaks: Sage Publications, pp. 105‐117.

Hatch, M.J. (1997) Organization theory: Modern, symbolic, and postmodern perspectives. Oxford: Oxford University Press.

Johnson, P. (2004) 'Analytical Induction'. In Cassell, C. & Symon, G. Essential Guide to Qualitative Methods in Organizational Research. Sage: London

116

Fariba Darabi and Murray Clark Johnson, P. and Clark M. (2006) Business and Management Research Methodologies: Sage Library in Business and Management.

Johnson, P. and Duberley (2000) 'Understanding Management Research: An Introduction to Epistemology'. London: Sage.

Lieblich, A. (1998) Narrative research: Reading, analysis and interpretation. Thousand Oaks, CA: Sage Publications.

Lincoln, Y.S. and Guba, E. (1985) 'Naturalistic Enquiry'. Beverley‐Hills: Sage Lincoln, Y.S. and Guba, E.G. (2000) “Paradigmatic controversies, contradictions and emerging confluences”. In: Denzin, N.K. nd and Lincoln, Y.S. (eds.), Handbook of Qualitative Research. 2 ed.,Thousand Oaks, CA: Sage Publications, pp. 163‐188.

Marzo‐Navarro, M., Pedraja‐Iglesias, M. and Rivera‐Torres, P. (2009), “The marketing approach in relationship between universities and firms”, Journal of Relationship Marketing, Vol. 8 No. 2, pp. 127‐47.

McCracken, G. (1998) The Long Interview. Newbury Park, CA: Sage Publications.

Mead, G.H. (1934) 'Mind, Self & Society'. Chicago: Chicago University Press.

Miles, M.B. and Huberman, A.M. (1994) Qualitative Data Analysis: An expanded course book. 2nd ed., London: Sage.

Outhwaite, W. (1975) 'Understanding Social Life: The Method Called Verstehen'. London: George Allen Unwin.

Orlikowski, W.J. and Baroudi, J.J. (1991) “Studying information technology in organizations: Research approaches and assumptions”. Information Systems Research, Vol. 2, No. 1, pp. 1‐28.

Potter, J. and Wetherell, M. (1994) “Analysing discourse”. In: Bryman, A. and Burgess, R. (eds.), Analysing Qualitative Data, London: Routledge, pp. 47‐68.

Salter, A., Tartari, V., D’Este, P. and Neely, A. (2010), 'The Republic of Engagement; Exploring UK Academic Attitudes to Collaborating with Industry and Entrepreneurship'. The Advance Institute of Management Research, London, available at: www.aimresearch.org Shotter, J. (1975) Image of Man in Psychological Research. London: Methuen.

Silverman, D. (2005), 'Doing Qualitative Research', Sage Publication, 2nd edition Strauss, A., & Corbin, J. (1998), 'Basics of qualitative research: Grounded theory procedures and techniques'. London: Sage.

Symon, G., and Cassell, C., (1998), 'Qualitative methods and analysis in organisational research‐ A practical guide', Sage Publications.

Thomas R. D. (2006), 'A General Inductive Approach for Analysing Qualitative Evaluation Data', American Journal of Evaluation 2006; 27; 237 Van Maanen, J. (1998) “Different strokes: qualitative research in the Administrative Science Quarterly from 1956 to 1996”.

In: Van Maanen, J. (ed.), Qualitative Studies of Organisations. London: Sage Publications.

Vangen, S. and Huxham, C. (2003), “Nurturing collaborative relations, building trust in inter‐organizational collaboration”, The Journal of Applied Behavioural Science, Vol. 39 No. 1, pp. 5‐31.

Walsham, G. (1993) Interpreting Information Systems in Organisations. Chichester: John Wiley and Sons.

Walsham, G. (1995) “The emergence of interpretivism in IS research”. Information Systems Research, Vol. 6 No. 4, pp. 376‐ 394.

Willis, W J., (2007), 'Foundations of qualitative research‐ Interpretive and Critical theory Approaches', Sage Publications.

117

Mixed Methods Research: Creating Fusion From the QUAL and QUAN Data Mosaic Sally Eaves Faculty of Arts, Computing, Engineering, and Sciences (ACES), Sheffield Hallam University, UK research@sallyeaves.co.uk

Abstract: There is no single recommended means of discourse for presenting and discussing mixed methods research, with lack of data synthesis and process transparency a frequently cited criticism. This paper addresses the deficiency and explores inventive means of data collection alongside innovative approaches to integrating, analysing and articulating qualitative and quantitative sources. A pragmatic philosophy, supported by theoretical and methodological bricolage is advocated and justified. A panoptic empirical study to elucidate the knowledge sharing influences of middle management in leading UK communication sector operators provides context. A sequential‐exploratory and equally weighted QUAL QUAN design was selected, incorporating emergent evaluation and integration. Innovative qualitative techniques were adopted, namely STRIKE ‐ STRuctured Interpretation of the Knowledge Environment, photographic analysis and word cloud visualisation, alongside cultural‐web focus groups. This facilitated rich, nuanced and multi‐textured data capture to aid the instrument fidelity of a quantitative cross‐operator survey. Triangulation was undertaken across all sources to assess areas of corroboration, elaboration or dissonance. It is demonstrated that this approach enables a multiplicity of perspectives to build successive deepening of understanding; supports transparency, traceability and synthesis; benefits credibility and validation and provides evidence of methodological robustness. This dynamic approach towards the design, conduct, fusion and presentation of mixed methods research therefore addresses a challenging lacuna: to combine rigour with responsiveness, texture with breadth and communicability with complexity. This can foster reflexivity and sensemaking for the researcher and further, can facilitate understanding, engagement and connection for the audience.

Keywords: mixed methods, pragmatism, bricolage, method integration, knowledge sharing

1. Introduction: Mixed methods in business and management research A content analysis of papers presented at the European Conference on Research Methodology for Business and Management Studies across 2010 and 2011 (Marais 2012) indicated that qualitative studies are predominant and that only 17% of accepted submissions are classified as multi‐methodological. Mixed method research is incorporated within this category: work which employs more than one approach within a single study’s design, data collection or analysis, encapsulating philosophical assumptions and actual modes of enquiry (Bryman 2009). Given the complex and dynamic nature of post‐industrial organisational contexts; increased application of mixed methods research is considered both timely and germane.

The primary goal of this work was to examine and advance knowledge regarding the design and conduct of mixed methods research to optimise richness, sensemaking, integration and triangulation across qualitative and quantitative approaches, addressing core criticisms and complexities. Responding to an identified gap in the knowledge management literature, contextual attention was directed towards empirical exploration of the influences which impact individual knowledge sharing behaviour at middle management level, affording a holistic, cross‐disciplinary perspective within a single study. It is posited that developing understanding which spans technological, human and social factors can be best acquired through the approach elucidated. A researcher‐as‐bricoleur position is fostered through empirical evaluation of a textured, panoptic and innovative range of data collection, analysis and bridging techniques.

1.1 Paradigm components, perspectives and implications A paradigm may be described as a worldview: the underlying beliefs, values and assumptions which act as a frame of reference and guide researcher inquiry. Beliefs may be ontological regarding perceptions/assumptions of reality, epistemological concerning the certainty and legitimacy of knowledge, axiological including ethics and rhetorical in respect to researcher voice, creating a “set of interlocking philosophical assumptions and stances” (Greene and Caracelli 1997, p6). Paradigm beliefs are normative in nature, impacting on perspectives regarding the credibility of findings. Purists explicitly or implicitly assert paradigm superiority and posit that quantitative and qualitative research methods cannot be mixed within a study or related set due to the epistemological disparity between their theoretical assumptions (Patton 2002).

118

Sally Eaves It is opined that placing approaches in opposition negates the potential to capitalise on respective strengths “including what each can contribute to the other" (Wolcott 2002, p99). Complementary dimensions include opportunities for convergent and discriminant validation as well as triangulation, applicability to cross‐ disciplinary research, and development of broad as opposed to narrow, paradigm lapidific researcher skill‐sets (Marais 2012).

Critically, complex evaluation of “real‐world” (Rallis and Rossman 2003, p493) scenarios frequently necessitates crossing paradigmatic boundaries to seek nuanced answers to the diversity of questions posed, aligned with flexibility in approach. This is pertinent to the high‐technology UK communications sector: a knowledge intensive, dynamic environment with challenges including strong market competition, pricing pressures and converging data architecture across mobile, social media and the cloud (Accenture 2013). A single paradigm approach of pragmatism is adopted to address this, as advocated by Morgan (2007).

1.2 Pragmatism and bricolage A pragmatic approach (Feilzer 2010) combines the inductive logic of qualitative exploration, the deductive reasoning of quantitative confirmation and further, the creative problem‐solving emphasis of abduction (Johnson and Onwuegbuzie 2004) to generate useful and reliable knowledge for both theoretical and practical application. This is pertinent to the Business and Management domain which embraces increasingly pragmatic disciplines such as Information Systems (Ågerfalk 2010) that combine theory with practical research and implications.

It is important to justify this choice which the author regards as a means to bridge philosophy and methodology (Cameron 2011). Epistemological and technical concerns are combined; the nature of the knowledge that is produced and the methods adopted to generate it (Morgan 2007). Problem‐driven but theoretically cognisant, practical judgement is employed to make methodological choices which optimally address research questions and enable successful outcomes. Applying critical reflexivity, research is conducted in consideration of, but not wholly constrained by, underlying philosophical debate. Pragmatism supports methodological pluralism, the case for which has been described as “irresistible” (Hirschheim 1992, p30).

Bricolage can guide pragmatic practice with a position of "researcher‐as‐bricoleur" (Denzin and Lincoln 2011, p4) considered congruous to navigate the contextual complexity which drives “the need for multiple ways of seeing” (Kincheloe 2005, p327), interdisciplinary awareness and the adoption of inventive, imaginative and resourceful research choices. Both methodological and theoretical perspectives are relevant and allow eclecticism where appropriate, promote surfacing of new meanings and insight (Deuze 2006) and further, can aid the removal of “blinders” to facilitate cross‐discipline development (Denzin and Lincoln 2011, p168).

It is recognised that this approach increases demands on the researcher, including the need to appreciate a variety of discourses and knowledge of multiple methods: their application, combination and evaluation. It is argued that this is both a valuable skill‐set to acquire and perspective to understand, particularly relevant to Business and Management research which is frequently underpinned by related themes of sensemaking and improvisation.

2. Methodology and method selection 2.1 Research design To support the reflective, pragmatic methodology elucidated, an equally weighted, sequential‐exploratory mixed methods design with instrument development was selected (Milton et al. 2003), summarised as QUAL QUAN. Following reflection on researcher skills, resources, pre‐knowledge and phenomenon character (Cronholm and Hjalmarsson 2011), this configuration was considered optimal to provide alignment with, and attention to, stated research goals (Onwuegbuzie and Teddlie 2003). It also supports the underlying rationale of benefiting instrument fidelity, significance enhancement (Collins, Onwuegbuzie and Sutton 2006) and researcher flexibility.

Application of an emergent point of interface (Morse and Niehaus 2009) aims to facilitate integration of developing findings as opposed to only separate analysis, promoting connections to be forged between the qualitative and quantitative components. This process can be complex and necessitates transparency, an

119

Sally Eaves aspect insufficiently addressed in mixed methods research (Bryman 2007, 2009). As predicated by Rosenberg (2008, p245), researcher choices are “rationally justified”.

2.2 Qualitative stage: Context and method alignment Building on an expansive literature review, the initial qualitative phase was explorative, descriptive and explanative in orientation (Johnson and Christensen 2004). This affords close attention to the knowledge sharing phenomenon of interest and facilitates nuanced understanding of the actor lifeworld (Habermas 1984) through processes of data deduction (Onwuegbuzie and Teddlie 2003).

Focus was directed at surfacing influences of individual knowledge sharing behaviour within a single communications operator post joint‐venture change. This organisation is subject to dynamic transformation processes impacting strategic realignment, restructuring and rebranding. Data was anonymised and aggregated where appropriate to protect identity. The collection process is informed by the “preunderstanding" (Gummesson 2000, p57) and emic awareness gained via insider‐researcher status. The potential for relativity issues was balanced through the etic position of non‐insider peers with whom anonymised emergent findings were discussed, the innovative range of techniques employed, and reflection on practitioner research learning experiences (Mercer 2007).

Research was conducted in Q3 2012 utilising the original observational framework STRIKE and focus groups as primary data collection methods, supported by photography and word cloud visualisation. Middle managers in the IT Support Function were identified as the target audience due to their pivotal brokering role in knowledge sharing practices (Nonaka and Nishiguchi 2001). Method detail and rationale are fully appraised.

STRIKE: STRuctured Interpretation of the Knowledge Environment

Drawing on systems analysis expertise, the author identified the potential to augment the supportive observational tool STROBE (Kendall and Kendall 1984). STRIKE is the emergent design artefact: an unobtrusive and multidimensional framework to observe and evaluate the knowledge environment comprising elements such as office layout, design and aesthetics, desk placement, workspace decoration, props, external knowledge sources and branding. It is posited that its adoption can align with and moreover, enrich interactive methods.

With reference to the Design Science Research method (Heje, Baskerville and Venable 2012), a descriptive ex‐ ante evaluation was completed by iteration during a pilot study in a comparative naturalistic setting. Face analysis was undertaken following an approach to Dr Gordon Rugg from Keele University, an expert in knowledge elicitation.

Focus Groups

Johnson and Scholes’s (1999) cultural web framework was employed within a focus group setting to aid examination of knowledge culture. Four sessions were undertaken, comprising 10 actors of equivalent position per group (N=40). A semi‐structured topic guide was prepared with informed consent obtained for audio recording and the use of resultant output material. Attention was afforded to semiotics, semantics and the potential impact of memory bias. Thematic analysis and word cloud visualisation of findings were utilised to provide flexible data evaluation with potential subjectivity issues addressed by engaging expert, non‐ participant review.

Photography

Photography may be utilised in both an illustrative and interpretive mode within which ethical concerns must be considered, notably consent, respect and confidentiality. Photographs are “quasi‐representational” in nature (Warren 2005, p861) and were employed to enrich and complement STRIKE observation, augmenting narrative. This facilitates communicability, understanding and aids visualisation of context, providing a frame of reference for reflection.

120

Sally Eaves Word Cloud Representation

Wordle is a “visually rich” medium that can be employed as a supplementary research tool (McNaught and Lam 2010, p630) supporting thematic analysis. No instances of its adoption are observed within the Business and Management domain. Wordle was utilised to create a word cloud analysis of participant responses from the focus group sessions. The cloud is a visual representation of the output text, based on word frequency and all data was saved privately to preserve confidentiality. The methodology by Rivadeneira et al. (2007) was consulted to facilitate the data analysis process.

2.3 Method integration and bridging between phases Using a thematic checklist format, observational findings from STRIKE were compared against focus group data to assess whether each component was confirmed, reversed, modified, supplemented or would benefit from further investigation. Focus group participants were then invited to attend a post evaluation session to discuss the interpretation, with the consensus achieved posited to add additional legitimacy. To optimise instrument fidelity, qualitative findings were compared to potential influences on knowledge sharing identified at literature review so that augmentations, additions or removal of scale construct items could be incorporated.

2.4 Quantitative stage: Context and method alignment This phase was descriptive and explanative in orientation. A purposeful sampling approach was adopted to identify middle management participants (N=78) across the IT Support functions of four leading UK communications operators, classified by market share. Survey distribution was undertaken via a web‐based option, drawing on Baatard’s (2012) technical guide to optimise balance between researcher requirements and respondent experience.

The dependent variables of tacit and explicit knowledge sharing were measured across self‐report questions employed or adapted from the behaviour scales of Yi (2009) and Reychav and Weisberg (2009). These were selected due to scope, relevance and demonstration of dimensionality, reliability and validity. Augmenting Ipe’s (2003) study, independent variables were aligned across five core influence factors: motivation to share, knowledge type (tacit/explicit distinction), opportunities to share, nature of the individual (personality, demographics) and culture. A five point Likert scale was adopted for attitudinal measurement with findings subject to reliability, correlation and regression analysis.

Open‐ended questions were included to encourage additional insight and elicit survey feedback, with data evaluated through thematic review and word cloud visualisation. Dr Gordon Rugg conducted content validity analysis on construct design and survey wording, with focus directed at reducing self‐reporting bias.

Awareness of potential dissonance between espoused and actual behaviour is reflected in the overall approach adopted, employing a range of methods of knowledge elicitation.

3. Integrated presentation and discussion of core findings Articulation and discussion of integrated findings is a noted challenge for mixed methods research (Flick 2007), especially within word limit constraints. Core results are therefore presented sequentially with integration and evaluation evidenced on an emergent basis. Textual and visual approaches are utilised to aid communicability.

3.1 STRIKE observation Table 1 demonstrates STRIKE in action and its practical implications, for example whether the lack of external information sources observed might reflect a knowledge gap. Photography has been incorporated to illustrate, enrich and complement researcher observation, providing both transparency and the ability to actualise place.

A reduced data set is presented to preserve anonymity.

121

Sally Eaves Table 1: STRIKE Evaluation Environmental Element Office Location and Environment

Description Meetings in open, equipped locations, facilitating interaction and exchange.

Functional, neutrally decorated office aesthetics.

Desk Placement

Combination of fixed partitioning and management collocation may negate optimisation of equal exchange.

Decoration

Frequent observations of objects from previous team events with branding associated with pre joint‐venture heritage. This reflects pride and implies nostalgia.

External Information Sources

Employees maintain personal libraries and individual subscriptions to professional bodies which may indicate self‐directed or hoarding behaviour.

Company magazines rather than industry journals are observed implying a lack of inward external knowledge flow.

Office Lighting and Colour

White neutral walls – furnished with memo‐boards and subtle accents of colour in communal spaces. Contrasts with bright, strongly decorated desk areas.

Supporting Photography

3.2 Focus groups Use of the cultural web afforded an effective scaffold for discussion regarding behavioural norms. Reflecting on use of visualisation as a means of representation in other practictioner contexts, Wordle was employed to evaluate findings in conjunction with the thematic analysis of transcripts. Figure 1 provides an example of this approach in praxis, demonstrating evaluation and communicability value.

Figure 1: Knowledge sharing barriers by word cloud visualisation

122

Sally Eaves Findings were compared to a thematic analysis with culture, time, change, systems and budget emergent as key issues. Respondent articulation of hoarding aligns with the interpretation of related behaviour identified via STRIKE, providing corroborative evidence. Word choice selection is illuminating, notably the negative connotations of “inadequate”, “stupidity” “ignored” and “isolated” stated in relation to current culture. This adds a layer of understanding to the evidence of nostalgia towards the pre‐joint venture state observed through STRIKE. The credibility and self‐awareness of participants is reinforced by observations of adept use of kinesic communication via body language, with purposeful hand gestures employed to emphasise a point made. This use of creative approaches and products as interpretive text is appropriate as part of methodological bricolage (Yardley 2008).

3.3 Emergent integration Qualitative findings broadly supported the knowledge sharing influences identified at literature review.

Adopting selection criteria of theme frequency and articulation (Cronholm and Hjalmarsson 2011), proposed independent variables and construct items across the five influence factors were then revisited: a core benefit of this emergent approach. As an example, amendments to address the potential for divisions between individual and organisational orientations towards knowledge management strategy were incorporated. Depth of insight achieved into the complex and evolving contextual dynamics within the single operator was also a key outcome, providing perspective and data richness for cross‐operator comparison.

3.4 Quantitative survey There are limitations to the extent in which extensive quantitative survey findings across 50 questions can be presented within word limit and structuring confines, a noted issue for mixed methods publications (O'Cathain, Nicholl and Murphy 2009). Consistent with the approach adopted throughout the paper, transparency is maintained with the intent to articulate pertinent overall findings within an integrated single article (Stange, Crabtree and Miller 2006).

The data obtained comprised a majority of males (N=60, 76.92%), in the age range 50‐59 (N=30, 38.46%) and with significant industry experience of more than 15 years (N=48, 61.54%). This aligns with the population profile of the UK IT and Telecommunications Sector (E‐Skills 2012). Reliability analyses were conducted on all scales in order to determine the level of internal consistency reliability, achieving acceptable Cronbach Alpha coefficients above .70.

A plethora of constructs related to motivation to share, knowledge type, opportunities to share, nature of the individual and culture were found to directly and significantly influence sharing practice across all operators: justifying the holistic approach and extending the work of Ipe (2003). Differences in behaviour are identified based on the type of knowledge shared, supporting the tacit and explicit distinction utilised. Some differences in sharing norms were observed in the joint venture organisation as opposed to the other operators, with reduced volition and capacity to share identified across constructs within the nature of the individual, opportunities to share and motivation to share factors.

3.5 QUAL‐QUAN triangulation A synthesising cross‐phase triangulation was performed, building on the between‐method evaluation of STRIKE and focus group data. Findings for core themes were compared across primary and secondary methods of enquiry. Use of a matrix format table facilitated the comparative process and provided a reflective focus for the researcher to bridge data sources, identifying areas of congruence and dissonance. Credibility was enhanced by a high level of traceability between empirical and behavioural findings. Insight was also achieved regarding the differences in sharing volition identified in the single operator as opposed to its competitors.

This is subject to additional study to develop predictive capability connecting transformational organisational change and knowledge sharing volition, aspects considered beyond the scope of this paper.

4. Benefits of approach and practical implications This study supports adoption of a pragmatic philosophical perspective in mixed methods research purpose and design, facilitating consistent conceptualisation and convergence. Combined with an emergent point of interface, it provides a scaffold for researchers to react and gain from in‐process research findings, as

123

Sally Eaves demonstrated by the augmentations to instrument design enacted following initial qualitative interpretation.

The sequential approach enabled cumulative development of understanding which was enhanced by undertaking triangulation both within and across phases. This design therefore benefits the depth of knowledge achieved regarding the phenomenon under study and further, provides a rigourous defence against validity criticisms. Robustness of findings is also enhanced by attention directed to key epistemic criteria (Marais 2012) and the transparent, sound justification and elucidation of research choices with challenges such as data presentation acknowledged, addressing a core criterion for study evaluation (Paul and Marfo 2001).

Regarding the specific research context, it is argued that the innovative combination of data collection, analysis and bridging techniques employed surfaced a breadth and nuanced range of knowledge sharing behavioural influences and facilitated comparison across organisational settings. This outcome could not have been achieved by a purely monomethod design or moreover, to the same scope and depth, in a mixed methods study that utilised a more restricted range of approaches. Underpinning this, the transparent adoption of theoretical and methodological bricolage was found to aid examination, querying and ordering of researcher resources and selection, resulting in unique data combinations and providing a scaffold for improvisation.

In terms of specific methods, word cloud visualisation is opined to be particularly effective as both a researcher evaluative tool and as a communicative device to engage an audience, therefore addressing a noted difficulty (Lieber 2009). This technique effectively enables multiple pieces of information to be represented in a compact and simultaneous manner. A verification check against focus group transcripts is recommended to ensure accurate representation.

Further, STRIKE demonstrates capacity to provide a rigorous, unobtrusive and multidimensional framework to facilitate observation and evaluation of knowledge environments, synthesising structure, creativity and contextual flexibility. Photography provides both interpretative and “evidential power" (Dant and Gilloch 2002, p17) to support the method. Utilisation of STRIKE can align with and enrich interactive techniques to elucidate the actor narrative and behavioural norms, motivators and inhibitors; thus, affording significant organisational diagnostic and potentially prescriptive capability. This technique is therefore regarded as especially utile to bridge the frequent dissonance between theory and practice with usability for researcher and practitioner alike. Following the successful descriptive ex‐ante and ex‐post naturalistic evaluation (Heje, Baskerville and Venable 2012), it will be subjected to further research to assess capacity to perform in a variety of contextual circumstances.

5. Conclusion This study contributes to both mixed methods and knowledge management literature. It confirms the applicability and benefits of a QUAL QUAN sequential, exploratory design to elucidate a critical dimension impacting across Business and Management disciplines: the panoptic, cross‐disciplinary range of influences affecting individual knowledge sharing behaviour. Considering the link between sharing practice and organisational performance (Rašula, Vukšić and Štemberger 2012), the design and encapsulated techniques afford particular value.

The capability to elucidate different perspectives and dimensions of a phenomenon may also be applied to a range of contextual settings and problem areas, combining nuanced local focus with wider generalisability. The method orientation supports traceability (Cronholm and Hjalmarsson 2011) and advancing previous findings, this configuration is also found to benefit contexts where researcher pre‐knowledge is high but the phenomenon is subject to dynamic change. Primary methods STRIKE, focus groups and the quantitative survey are innovatively supported and enriched by secondary techniques including photography, word cloud visualisation and attention to language. The multi‐layered, methological rigour achieved also mitigates the difficulties of mapping reports of behaviour onto actual behaviour, aiding validation of intersubjective agreement. Adoption of an emergent point of interface optimises instrument fidelity and combined with triangulation across behavioural and empirical data, moves from a mosaic to fusion (Bazeley 2010) of linguistic, textual and physical data, achieving both breadth and subtlety of understanding.

124

Sally Eaves Addressing the mixed methods challenges detailed by Bryman (2007), Lieber (2009) and Cameron (2010): the philosophical foundation and paradigmatic stance of the study is explicitly stated, method selection justified and robust within‐phase and cross‐phase integration and triangulation undertaken. Presentation challenges, usability and meaningful audience engagement are considered. Responding to the call for competence capacity building, researcher versatility, creativity and diversity of skill sets employed is both developed, and robustly demonstrated. Practical guidance is afforded for researcher and practictioner, underpinned by pertinent theory and empirical assessment and it is intended that this paper will both stimulate discussion and promote an increased application of approach.

Acknowledgements Gratitude is expressed to Dr Gordon Rugg for undertaking robust face and content validity analysis.

References Accenture (2013). Technology Vision 2013. Accenture. Ågerfalk, P. (2010). Getting pragmatic. European Journal of Information Systems, 19, pp. 251–256.

Baatard, G. (2012). A Technical Guide to Effective and Accessible Web Surveys. The Electronic Journal of Business Research Methods, pp. 101‐109.

BazeleY, P. (2010). Metaphors for integrated analysis in mixed methods research. Presented at the 6th International Mixed Methods Conference, Baltimore, USA, 11th July, 2010.

Bryman, A. (2007). Barriers to Integrating Quantitative and Qualitative Research. Journal of Mixed Methods Research, 1 (1), pp. 8‐22. rd Bryman, A. (2009). Social Research Methods. 3 ed., Oxford: Oxford University Press.

Cameron, R. (2011). Mixed Methods Research: The Five Ps Framework. The Electronic Journal of Business Research Methods, 9 (2), pp. 96‐108.

Collins, K., Onwuegbuzie, A. And Sutton, I. (2006). A model incorporating the rationale and purpose for conducting mixed methods research in special education and beyond. Learning Disabilities: A Contemporary Journal, 4, pp. 67‐100.

Cronholm, S. And Hjalmarsson, A. (2011). Experiences From Sequential Use of Mixed Methods. The Electronic Journal of Business Research Methods, 9 (2), pp. 87‐95.

Dant, T. And Gilloch, G. (2002). Pictures of the past: Benjamin and Barthes on photography and history. European Journal of Cultural Studies, 5 (1), pp. 5‐23. th Denzin, N. And Lincoln, Y. (2011). The SAGE Handbook of Qualitative Research. 4 ed., Sage.

Deuze, M. (2006). Participation, Remediation, Bricolage: Considering Principal Components of a Digital Culture. The Information Society, 22 (2), pp. 63‐75.

E‐Skills (2012). Technology Insights 2012: United Kingdom. e‐skills UK. Feilzer, M. (2010). Doing mixed methods research pragmatically: implications for the rediscovery of pragmatism as a research paradigm. Journal of Mixed Methods Research, 4 (1), pp. 6‐16.

Flick, U. (2007). The SAGE Qualitative Research Kit. London: Sage.

Greene, J. And Caracelli, V. (1997). Defining and describing the paradigm issue in mixed‐method evaluation. In: J. GREENE And V. CARACELLI (eds), Advances in mixed‐method evaluation: The challenges and benefits of integrating diverse paradigms (pp. 5‐17). Jossey‐Bass.

Gummesson, E. (2000). Qualitative Methods in Management Research. 2nd ed., Sage.

Habermas, J. (1984). The theory of communicative action, Volume 2. Boston: Beacon Press.

Heje, P‐H., Baskerville, R. And Venable, J. (2012). A Comprehensive Framework for Evaluation in Design Science Research.

Lecture Notes in Computer Science, 7286, pp. 423‐438.

Hirschheim, R. (1992). Information Systems Epistemology: An Historical Perspective. In: R. Galliers (ed.), Information Systems Research: Issues, Methods and Practical Guidelines, pp. 28‐60, London: Blackwell Scientific Publications.

Ipe, M. (2003). Knowledge Sharing in Organizations: A Conceptual Framework. Human Resource Development Review, 2 (4), pp. 337‐359.

Johnson, B. And Christensen, L. (2004). Educational research: Quantitative, qualitative, and mixed approaches. 2nd ed., Boston, Massachusetts: Pearson Education.

Johnson, G. And Scholes, K. (1999). Exploring Corporate Strategy. 5th ed., Prentice Hall.

Johnson, R. And Onwuegbuzie, A. (2004). Mixed Methods Research: A Research Paradigm Whose Time Has Come.

Educational Researcher, 33 (7), pp. 14‐26.

Kendall, K. And Kendall, J. (1984). STROBE: A Structured Approach to Observation of the Decision‐Making Environment.

Information and Management, 7, pp. 1‐11.

Kincheloe, J. (2005). On to the next level: Continuing the conceptualization of the bricolage. Qualitative Inquiry, 11 (3), pp.

323‐350.

Lieber, E. (2009). Mixing Qualitative and Quantitative Methods: Insights into Design and Analysis Issues. Journal of Ethnographic and Qualitative Research, 3, pp. 218‐227.

Marais, H. (2012). A Multi‐Methodological Framework for the Design and Evaluation of Complex Research Projects and Reports in Business and Management Studies. The Electronic Journal of Business Research Methods, 10 (2), pp. 64‐76.

125

Sally Eaves Mcnaught, C. And Lam, P. (2010). Using Wordle as a Supplementary Research Tool. The Qualitative Report, 15, pp. 630‐643.

Mercer, J. (2007). The challenges of insider research in educational institutions: wielding a double‐edged sword and resolving delicate dilemmas. Oxford Review of Education, 33 (1), pp. 1‐17.

Milton, J., Watkins, K., Studdard, S. And Burch, M. (2003). The ever widening gyre: Factors affecting change in adult education graduate programs in the United States. Adult Education Quarterly, 54 (1), pp. 23‐41.

Morgan, D. (2007). Paradigm Lost and Pragmatism Regained, Methodological Implications of Combining Qualitative and Quantitative Methods. Journal of Mixed Method Research, 1 (1), pp. 48‐76.

Morse, J. And Niehaus, L. (2009). Mixed Method Design: Principles and Procedures. Walnut Creek, USA: Left Coast Press.

Nonaka, I. And Nishiguchi, T. (2001). Knowledge Emergence: Social, Technical, and Evolutionary Dimensions of Knowledge Creation. Oxford University Press.

O'cathain, A., Nicholl, J. And Murphy, E. (2009). Structural issues affecting mixed methods studies in health research: a qualitative study. BMC Medical Research Methodology, http://www.biomedcentral.com/bmcmedresmethodol/content/99 (82).

Onwuegbuzie, A. And Teddlie, C. (2003). A framework for analyzing data in mixed methods research. In: A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 351‐383). Thousand Oaks, California: Sage.

Patton, M. (2002). Qualitative Research & Evaluation Methods. 3rd ed., London :Sage.

Paul, J. And Marfo, K. (2001). Preparation of Educational Researchers in Philosophical Foundations of Inquiry. Review of Educational Research, 71 (4), pp. 525–547.

Rallis, S. And Rossman, G. (2003). Mixed methods in evaluation contexts: A pragmatic framework. In: A. Tashalkkori And C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research, Sage, pp. 491‐512.

Rašula, J., Vukšić, V. And Štemberger, M. (2012). The Impact Of Knowledge Management On Organisational Performance.

Economic And Business Review, 14 (2), pp. 147–168.

Reychav, I. And Weisberg, J. (2009). Good for Workers, Good for Companies: How Knowledge Sharing benefits Individual Employees. Knowledge and Process Management, 16 (4), pp. 186–197.

Rivadeneira, A., Gruen, D., Muller, M. And Millen, D. (2007). Getting our head in the clouds: Toward evaluation studies of tagclouds. Proceedings of the SIGCHI conference on human factors in computing systems, (pp. 995–998). New York: ACM Press. rd Rosenberg, A. (2008). Philosophy Of Social Science. 3 ed., Westview Press.

Stange, K. Crabtree, B. And Miller, W. (2006). Multimethod Research, Annals of Family Medicine, 4, pp. 292‐294.

Venable, J. And Baskerville, R. (2012). Eating our own Cooking: Toward a More Rigorous Design Science of Research Methods. The Electronic Journal of Business Research Methods, 10 (2), pp. 141‐153.

Warren, S. (2005). Photography and Voice in Critical Qualitative Management Research. Accounting, Auditing & Accountability Journal, 18 (6), pp. 861‐882.

Wolcott, H. (2002). Writing up qualitative research . . . better. Qualitative Health Research, 12, pp. 91‐103.

Yardley, A. (May 2008). Piecing Together ‐ A Methodological Bricolage. Forum: Qualitative Social Research, 9 (2), Article 31.

YI, J. (2009). A measure of knowledge sharing behavior: Scale development and validation. Knowledge Management Research & Practice, 7, pp. 65–81.

126

A Multi‐Layered Approach to Surfacing and Analysing Organisational Narratives: Increasing Representational Authenticity Sally Eaves and John Walton Faculty of Arts, Computing, Engineering, and Sciences (ACES), Sheffield Hallam University, UK research@sallyeaves.co.uk J.R.Walton@shu.ac.uk

Abstract: This paper presents an integrated, multi‐layered approach to narrative inquiry, elucidating the evolving story of organisational culture through its members and their physical, textual, linguistic and visual dialogue. A dynamic joint venture scenario within the UK hi‐technology sector was explored to advance understanding of the impact of transformation level change, specifically its influence on shared belief systems, values and behavioural norms. STRIKE – STructured Interpretation of the Knowledge Environment is introduced as an innovative technique to support narrative inquiry, providing a structured, unobtrusive framework to observe, record, evaluate and articulate the organisational setting. A manifestation of narrative in physical dialogue is elucidated from which the underlying emotional narrative can be surfaced. Focus groups were conducted alongside STRIKE to acquire a first order retrospective and contemporaneous narrative of culture and enable cross‐method triangulation. Attention was given to non‐verbal signals such as Chronemic, Paralinguistic, Kinesic and Proxemic communication and participants were also afforded opportunities to develop creative output in order to optimise engagement. Photography was employed to enrich STRIKE observation and document focus group output, affording high evidential value whilst providing a frame of reference for reflection. These tools enable a multiplicity of perspectives on narrative as part of methological bricolage. Rich, nuanced and multi‐textured understanding is developed, as well as the identification of connections, timbre and subjugated knowledge. A highly emotional and nostalgic context was established with actors’ sense of self strongly aligned with the pre‐joint venture organisation and its brand values, norms and expectations. Credibility and authenticity of findings is enhanced through data triangulation indicating traceability across methods, and from the contextual preservation attained through STRIKE. The multi‐layered approach presented can facilitate researcher reflexivity and sense‐making, while for the audience, it may be employed to help communicate and connect research findings. In particular, STRIKE demonstrates utility, quality and efficacy as a design artefact following ex‐post evaluation. This systematic method of narrative inquiry is suitable for standardisation and alongside a diagnostic/prescriptive capacity, affords both researcher and practictioner value in its application.

Keywords: narrative, interpretive processes, authenticity, voice, reflexive practice, joint venture outcomes

1. Introduction: Truth and authenticity in narrative inquiry Narrative is ubiquitous, natural and diverse in form (Barthes 1977), affording “sequentiality” (Bruner 1990, p43) and providing an organising framework of temporality. An increasing number of studies may be described as narrative, but as a broad and nebulous term these vary significantly (Riessman 2008). But what is the truth of narrative inquiry? This may be considered from different dimensions: a correspondence with the actuality of the world, the consensus of experienced viewpoints, coherence to an integrated belief system, and a pragmatic perspective embodying truth with practical utility (Kirkham 1995). Verisimilitude or the appearance of reality may be considered a criterion of a narrative way of knowing (Rorty 1979).

This mode of representation is not without critique. White (1990, p26) argues that qualities coveted for research benefits, namely naturalness and familiarity actually make it suspect for use in “any field of study aspiring to the status of science”. Narrative inquiry challenges modernist, traditional perspectives on truth and reality, the nature of knowledge, ways of knowing and the essence of self. Lived experience is emphasised as a catalyst for the social creation and construction of narrative knowledge through shared stories and their associated meanings (Bruner 1990).

This paper considers narrative from the perspective of both multi‐dimensional means of elucidation and as the lived–through phenomenon of participants. Aligning with a paradigm shift from realism towards constructivism and additionally, a postmodernist cultural shift (Huttunen, Heikkinen and Syrjälä 2002), this approach is considered germane to advance understanding of the intersection between culture, experience and change (Clandinin and Connelly 2000).

Narrative authenticity is also debated (Verhesschen 1999) but may be addressed in consideration of the inherent relationships between the artefacts, people and places which underpin it (Jones 2010). A

127

Sally Eaves and John Walton multidimensional phenomenon, authenticity impacts across the research process: from researcher role and processes of reflection, to data collection, evaluation and representation. Guidance was drawn from Winter’s (2002) principle of dialectics with attention afforded to polyphony or the means in which different voices may be heard. A creative range of techniques were employed to optimise actor engagement, data richness and sense‐making, while facilitating researcher critical self‐reflection.

1.1 Exploring organisational narratives Continually framed, styled and shaped in consideration of audience and purpose, an organisational narrative may afford intent to persuade to action, to re‐tell, to educate or to inspire. Unique, temporal and closely aligned to identity, it necessitates a level of congruence and controllability to communicate espoused values and core business activity. It can also form a bridge between historical roots and future direction.

Organisational narratives may be identified under alternate names: the spin associated with public relations management, the legends of corporate history (Gabriel 2000) and the powerful brand story that binds community affiliation, exemplified by Apple (Muniz and Schau 2005).

The association between narrative and the construction or deconstruction of identity (Riessman 2008) can be profound and merits evaluation from an actor perspective, post significant organisational change. Along with social foundations, it is argued that narrative as identity may also be considered an inherently aesthetic phenomenon: deeply felt, embodied and capable of being accessed and interpreted through different presentational forms.

1.2 The human story of transformational change A multi‐faceted narrative lens affords depth and breadth of inquiry into the sometimes congruent but often conflicting layers of individual, collective and organisational story. The knowledge intensive UK Communications industry provides the contextual background. This dynamic environment faces complex challenges including evolving consumer behaviour, context‐based services and converging data architecture (Accenture 2012). Described as “the new core competency” (Ross 2009), joint venture agreements are increasingly utilised to provide financial, economic and operational synergies.

Buy‐in to both the legitimacy of integration processes and the new organisational form is acknowledged as core to successful partnering outcomes (Gole and Morris 2007). It is therefore critical to enhance understanding of the affect of transformation on the organisational actors required to accept, legitimise and actualise change, notably the means in which individuals internalise and rationalise the new narratives. This study addresses a lacuna of empirical research in this area.

2. Research methodology Complex evaluations of real‐world scenarios frequently necessitate robust consideration of situated actions and problems (Cresswell 2009), and a combination of different, flexible approaches to investigation. To develop nuanced contextual understanding, a qualitative approach is adopted, informed by pragmatism (Dewey 1988) alongside interactive constructivism (Neubert 2001). This affords full consideration of experience, habit and communications embedded in context and awareness of the researchers’ perspective as both self and distant‐observers.

Data collection is informed by the “preunderstanding" (Gummesson 2000, p57) of the principal investigator as an insider‐researcher. This enables enhanced acceptance and capacity to acquire understanding in praxis, specifically an emic awareness of internal cultural dynamics including symbolism. It is acknowledged that

issues have been raised regarding "the relativity of insider status" (Hockey 1993, p199) but these are balanced by the non‐insider, etic position of the co‐author.

Narrative inquiry is selected as the most appropriate means to explore the experiential life‐world of actors whilst affording researcher self‐reflexivity. From an ontological perspective, a narrative inquiry considers organisational life as a storied reality. Epistemologically, knowledge is constructed within this reality and expressed through language conveyed in varying forms that can be difficult to elucidate.

128

Sally Eaves and John Walton

2.1 Research setting, data collection and analysis A synopsis of company background is provided with identity anonymised for confidentiality. Communications Firm A has operated in the UK hi‐technology sector for 20 years and following commencement of a 50:50% joint‐venture with less established competitor Firm B, embarked on a major restructuring and rebranding programme in Q2 2012. The alliance strategy centres on cost leadership through optimisation synergies, cost reduction and consolidation. The joint venture is referred to as Firm‐JV and consideration of previous culture relates to Firm A ‐ the regional headquarter site studied being populated by its former employees. Research was conducted in Q3 2012 using the original observational framework STRIKE in conjunction with focus groups. Method detail and rationale are now fully explicated.

2.2 STRIKE ‐ STRuctured Interpretation of the knowledge environment The physical environment is recognised to illuminate organisational practices and norms (Davenport 1997). It is proposed that attention to this dimension is also revelatory of the individual and collective narratives of the actors within, and is congruent with the “visual turn” in narrative inquiry (Riessman 2008, p181). To advance this, STRIKE is afforded as an augmentation to the system analysis observational framework STROBE (Kendall and Kendall 1984), with originality demonstrated in terms of scope, breadth and flexibility of design, and data acquisition and evaluation methods. A notable change is a move away from focussing on the environment of a particular executive level decision‐maker, to consider the wider office situation experienced by the majority of actors: those who are required to actualise transformational change in praxis.

Tool development was influenced by researcher concern that methodological reliance on direct participant story‐telling may result in increased potential for misinterpretation. Story articulation can be subject to high levels of distortion (Tversky 2004) and within focus groups; issues of groupthink or emotional contagion (Goleman 2005) may be influential. Additionally, with issues of power relations between researcher and researched a critical concern (Clandinin and Connelly 2000), the neutral and unobtrusive nature of this narrative approach can counteract against this problem. It is opined therefore, that STRIKE affords a path that can avoid attenuation of the data captured, so that as much of the characteristics of the system in focus as possible can be presented to the interpretive activity. The inter‐relationships between actor and environment are preserved.

Prior to this work, drawing on the Design Science Research Method (Heje, Baskerville and Venable 2012), a descriptive ex‐ante evaluation of the tool was undertaken by iteration during a pilot study in a comparative naturalistic setting. Focus was given to the environmental elements to be included for observation as it is posited that the physical narrative expressed in these dimensions can surface cultural nuances and the timbre of emotional stories. Face validity analysis was undertaken following an approach to Dr Gordon Rugg from Keele University, a knowledge elicitation expert.

To enact the resultant method, walk‐through sessions were conducted to systematically observe explicit environmental elements: office design, aesthetics, workspace decoration, supporting equipment, knowledge sources, clothing, branding and exterior presentation. Photography was used to provide an additional layer of documentation of the research setting (Banks 2007), enrich observation, aid reflection and support researcher neutrality with the intent to achieve a third‐person shot (Monaco 2009). Findings were compared with focus group data to aid confirmation or negation of interpretations and facilitate triangulation, identifying areas of narrative dissonance if present.

2.3 Focus groups Johnson and Scholes’ (1999) cultural web framework aided examination of organisational culture and the deeper level assumptions, beliefs and shared artefacts of the actors within (Schein 2010). The target audience for sessions was middle management whose beliefs are notably influential in turbulent conditions (Coltman, Devinney and Midgley 2008). This provides an alternative means of surfacing narrative alongside STRIKE to achieve more complete understanding.

Four sessions were undertaken over 12 days: two contemporaneous in orientation and two retrospective, comprising 10 actors of equivalent position per group (N=40). Focus groups were selected for their capacity to observe collective sensemaking in praxis (Boje 2008), notably the relational action of storytelling which could

129

Sally Eaves and John Walton not be achieved through individual interviews. A semi‐structured topic guide was used with informed consent obtained for audio recording and the use of output material. To optimise conduits for expression, participants were provided with artistic materials and ArtPad (2013), a self‐directional web 2.0 drawing application.

Attention was given to semantics, semiotics and the potential impact of memory bias. Thematic analysis and radar charting supported flexible data evaluation with possible subjectivity issues addressed by expert, non‐ participant review.

2.4 Reflexivity A research diary was used to record feelings, hunches and events which can add to data assemblage and its evaluation (Clandinin and Connelly 2000). Further, participant evaluation of the focus group inquiry was obtained on an anonymous, voluntary basis via a web‐link survey. This supports reflexive action, foregrounding information that can aid authentic interpretation.

3. The assemblage and evaluation of narrative Clandinin and Connelly’s (2000) three‐dimensional narrative inquiry space provides an appropriate scaffold to introduce and assess situated findings, affording continuity by progressing through the dimensions of the past, present and future whilst considering the interaction between personal and social elements. Salient fragments of narrative are presented and evaluated across the range of methods utilised, demonstrating multi‐layered narrative interpretation in praxis.

3.1 The past Focus group artistic output was revelatory of the strength of former culture, as illustrated in Figure 1.

Figure 1: Group retrospective perspective of firm a culture by free‐hand drawing A passionate response and evidence of the important historic and symbolic cultural context is afforded. The vibrant colour and imagery observed in the output material echoes the creativity and enthusiasm collectively exhibited, a confidence that anything is possible under “visionary” leadership. The principal‐investigators’ research diary is revelatory of empathy with this position. “I felt I’d made it” is an annotated response to recounting the start of their career with Firm A.

3.2 The present Three core dimensions emergent from the STRIKE assessment are presented in Table 1. Differences in the interpretation of narrative meaning are observed in the aesthetical juxtaposition between personal and public spaces, tension between defunct and current branding, and personal rather than organisational knowledge management. The surfacing of emotional narrative from physical dialogue is also illustrated. This approach conforms to Wittgenstein’s (2001) picture theory of meaning affording a pictorial, representational and logical level.

130

Sally Eaves and John Walton Table 1: Salient narratives elucidated via STRIKE evaluation Environmental Element

Description

Supporting Photography

Office Aesthetics

Disconnect is observed between public and office areas. The clean lines, precision and subtle branding of public spaces contrasts with the vibrant personalisation of workspaces which display Firm A branded artefacts.

Branding

New Firm‐JV posters and internal magazines lack characterisation and communication of values. There is minimal observation of actors wearing newly rebranded clothing.

This “faceless” approach is a distinct contrast with the highly personalised, fun, visual and value‐laden imagery previously employed. This remains prominent in many workspaces.

Workspace Decoration

Personalisation of desks with objects (mugs, coasters) and team‐building memorabilia associated with Firm A heritage and officially defunct branding. This implies nostalgia, reflects pride and is revelatory of the importance of identity expression.

The demonstration of Firm A branded artefacts indicates strong cultural internalisation and is considered an example

of representation and reification, aligned to organisational memory. It is opined that actor legitimisation is built upon this shared physical dialogue (Berger and Luckmann 1966). Despite the contemporaneous focus and unobtrusive observational perspective of STRIKE, the narrative of the past remains omnipresent.

This cultural dissonance aligns with focus group findings. The four cultural webs produced were evaluated to establish central themes, aided by radar chart comparison. Figure 2 is an innovative presentation of the aggregated data. This is a complementary, supportive tool rather than directly part of STRIKE itself, designed to provide synthesis and optimise communicability. It was created through a percentage conversation of participants individually instantiating a particular element of the cultural web whereby 100% equates to everyone agreeing to its importance. The order of the dimensions around the diagram is not significant: all aspects are equally weighted. 100% is positioned centrally to represent the convergence point of the dimensions of culture, facilitating awareness of the element(s) which most contribute to the underlying organisational paradigm.

131

Sally Eaves and John Walton

Figure 2: Radar chart comparison of cultural webs This perspective foregrounds the importance of Stories and Symbols to the pre‐joint venture life‐world as cited by 100% of participants. Such highly valued dimensions are disassociated from the new organisation as exemplified by the comment, “the only new stories are communication gaffs ‐ a case of least said ....”. It is past sticky stories that are positively shared as a form of cultural continuation, as supported by the prominence of cultural artefacts observed using STRIKE.

3.3 The future?

Focus group ArtPad output provided consistently powerful interpretations of Firm‐JV’s future, as illustrated by one actor’s contribution in Figure 3.

Figure 3: Actor perspective of firm‐JV’s future using ArtPad Elements of destruction, loss and a weakened position are emphasised as well as ambiguity towards the future. From the evocative imagery a metaphor of DNA destruction is inferred: the building blocks of the organisation are being taken apart. This interpretation is supported by concerns raised in group discussion, exemplified by the comment “I feel like I am fighting to hold on to our heritage”, alongside the nostalgic symbolism identified during STRIKE evaluation.

3.4 Narrative authenticity A sensemaking perspective was employed to reflect on observations, participant response, researcher reaction and the belief systems inherent in these. This included consideration of intentional or unintentional distortion

132

Sally Eaves and John Walton and the potential to develop inaccurate narrative perceptions. This is explored through the recurring contextual theme of nostalgia: this “pervasive” phenomenon can dominate outlook (Gabriel 1993, p119) and is typically highly idealised, selective and imbued with symbolism. The dynamics of interaction within focus groups can create conditions for “Emotion contagion” (Goleman 2005, p104), a subtle tacit exchange whereby moods may be synchronised and moreover, unintentionally influenced through researcher dialogue. The resultant potential for over‐interpreting narrative significance is balanced by the use of STRIKE which is indirect not interactive, and combined with photography, enables increased neutrality and transparency of observation.

In this case, nostalgia is interpreted as revelatory of the strength of affiliation retained towards former culture and brand. This may be as equally illustrative of current “discontents” (Gabriel 1993, p137) as the expressed past. The validity of this interpretation is supported by the tangible, omnipresent artefacts associated with previous organisational heritage observed through STRIKE: authenticity is preserved and not impacted by researcher‐participant or cross‐participant influences. Attention to word selection, specifically the cumulative effect of choices such as “loss”, “recall”, “longing” and “mourning” was also confirmatory. The credibility and self‐awareness of participants was noted in their adept use of linguistic devices including expressive sounds, and by kinesic communication through purposeful body language. This echoes the dialogic/performance narrative dimension described by Riessman (2008).

3.5 Triangulation of narrative methods of inquiry STRIKE findings were evaluated against those from the focus groups to explore whether narratives are confirmed, reversed, modified, supplemented or would benefit from additional inquiry (Kendall and Kendall 1984). In this research, a high degree of congruence is observed as demonstrated in Table 2, through the core emergent stories of acculturative stress and personal knowledge management.

Table 2: Matrix triangulation of primary narratives STRIKE Analysis Story 1: Personal Knowledge Management

Focus Groups: Evaluation Across Conduits

Maintenance of personal libraries and individual subscriptions to professional bodies which may indicate hoarding norms. No sharing activity published on internal notice boards or evidence of team resource libraries.

Availability of desk based rather than communal storage areas.

Participants discuss personal rather than organisational knowledge ownership and make sharing decisions based on value perceptions.

“I don’t feel at all comfortable sharing my knowledge ‐ Its mine and I use it where it will deliver maximum personal impact”.

“My sharing activity has reduced…things change so fast, I have to make a judgement call regarding what could be valuable in the future”.

Story 2: Acculturative Stress Juxtaposition between:

The prominence of artefacts associated with cultural heritage, reflecting pride and identity. Alongside the continuation of Firm A branded dress code, these ritualistic behaviours may be considered evidence of mimesis (Potolsky 2006). Desk space is highly personalised with demonstrations of individual eclectic dimensions: typically nostalgic, bold and blazoned in colour.

and:

The monomorphic, aesthetic design style of the new joint venture entity which is neutral and non distinct as exemplified by corporate posters which lack character and personalisation. Visual identity, established as an important element of pre‐joint venture culture, is perceived as at threat.

133

Social comparisons concern less favourable organisational circumstances as opposed to the past, a potential indicator of acculturative stress (Vaara 2000). ArtPad output in Figure 3 is highly evocative of this condition, as exemplified by the “what is next?” statement and lit fuse imagery. It is also supported by focus group commentary: “It feels too soon, I am mourning what we did before".

Juxtaposition is perceived between personal and social identity. In Firm A social identity is strong: “…it made you feel like you belonged; proud, confident and excited” By contrast, identification with Firm‐JV is currently perceived as an embarrassment: “Communication gaffs are becoming legendary now… it’s embarrassing.”

Sally Eaves and John Walton

4. Discussion of findings As “cornerstones of understanding” (Remenyi 2005, p138) stories create knowledge and in this study, their nuanced identification is revelatory of a joint venture firm held in tension. The approved rhetoric for change delivered through authorised texts such as new branding and physical demonstration through office aesthetics, is observed to be in counterpoint to the internal rhetoric proliferated by actors across their physical, textual, linguistic and visual dialogue.

Tension between competing logic is emergent (Brown, Ainsworth and Grant 2012). Individual and collective stories are striking for their emotion‐evoking pathos: informed by history, established values and norms, and imbued with nostalgia. This contrasts with the logos or logic focussed rhetoric which underpins the new espoused organisational narrative, as demonstrated in functional aesthetics and attempts at acculturation. The organisational story has become dissonant from both individual and the socially constructed collective narrative of the actors within.

Actors reveal lives strongly defined by the corporate lived self. This was a positive, aspirational experience pre‐ joint venture as this life‐world afforded a distinctive personality, strong values and visionary leadership. A sense of loss and concern for continuity is evident with the preservation of symbols of the former brand constituting a shared code of member‐to‐member communication, sustaining value homophily, behaviours and narrative memorialisation (Bell 2012). Problems are identified in terms of narrative relevancy, future orientation, and core competency as encapsulated in the brand and its actor ambassadors. It is often assumed that reactions to market conditions such as joint ventures cause creative destruction (Schumpeter 1942), but there remains a danger that competences which are core but hard to articulate are in fact unintentionally destroyed.

4.1 Methodological benefits and the contribution of STRIKE The adoption of an inventive, integrated combination of data collection and analysis techniques allows the semantic layer to become more transparent. Increasingly granular understanding of how meanings are constructed, maintained and used by organisational actors becomes possible. In particular, STRIKE evaluation elucidates the importance of human interaction with the physical as well as the social environment to create meaning.

Authenticity is supported by reflexive research practices and the transparency of approaches such as STRIKE, whilst the creative range of techniques employed optimises the potential for participant engagement to encourage a depth, breadth and richness of data. Using a triangulation matrix, emergent narratives can be traced across methods to provide corroboration. This is equally utile for identifying points of dissonance as observed between actors as a collective and the narrative vision of the joint venture organisation. This overall methodology reduces the potential for the misinterpretation of narrative (Heje, Baskerville and Venable 2012).

Expansive assessment of a design artefact is “critical” (Hevner et al. 2004, p82), and it is argued that STRIKE demonstrates utility, quality and efficacy ex‐post, aligning with its ex‐ante naturalistic pilot evaluation. Fully combinable with interactive qualitative methods, STRIKE can confirm, negate or explicate findings as part of triangulation. Its systematic approach to data analysis and assessment mitigates criticisms of methodological rigour in observational research and affords standardisation potential appropriate for verification and longitudinal studies, with the potential to aid diagnosis and prescription regarding issues identified.

It is argued therefore that STRIKE presents particular value for bridging the frequent dissonance between theory and practice, with usability for researcher and practitioner. For an audience, the highly communicable and visual style allows “the reader into the process" (Dickie 2003, p51) of interpretation and analysis. Research will be extended to develop cross‐case analysis and assess capability to perform in a variety of contextual circumstances.

5. Conclusions This paper foregrounds the use of innovative multi‐method, multi‐layered narrative inquiry to surface and analyse the human impact of transformational change, affording a holistic perspective of the cultural organisational DNA. Narrative serves a function of identity construction (Riessman 2008) and moreover,

134

Sally Eaves and John Walton identity preservation or memorialisation for the actors within: a developing disconnect between the authorised corporate and lived experience of narrative is richly interpreted. Findings also signpost the danger of competence destroying change, extending the joint venture research of Gilmore and Seitchik (2001).

The research supports the pictorial turn (Mitchell 2005) as a counterpoint to the widely recognised linguistic turn in business and management research: combining synchronic and diachronic elements of story to provide new means of seeing (Bell 2012), both for researcher and audience. STRIKE elucidates narrative from an original physical and visual perspective from which emotional nuances and underlying issues may be surfaced.

This responds to the call for “added diversity" (Riessman 2008, p200) in the developing field of narrative methodology. The researched, the researcher and the research audience are all considered narrators.

Utilisation of an overall approach encapsulating contextualised physical, textual, linguistic and visual dialogue can enhance authenticity of representation across this tripartite, hermeneutic circle of interpretation.

Acknowledgements Gratitude is expressed to Dr Gordon Rugg for conducting face validity analysis.

References Accenture (2012). Technology Vision 2012. Accenture.

Art Pad (2013). art.com [online]. Last accessed 30 January 2013 at URL: http://artpad.art.com/artpad/painter/ BANKS, M. (2007). Using Visual Data in Qualitative Research. London: Sage.

Barthes, R. (1977). Image‐Music‐Text. London: Fontana.

Bell, E. (2012). Ways of seeing organisational death: a critical semiotic analysis of organisational memorialisation. Visual Studies, 27 (1), pp. 4‐17.

Berger, P. And Luckmann, T. (1966). The Social Construction of Reality: A Treatise in the Sociology of Knowledge. London: Penguin Books.

Boje, D. (2008). Storytelling Organizations. London: Sage.

Brown, A., Ainsworth, S. And Grant, D. (2012). The Rhetoric of Institutional Change. Organization Studies, 33 (3), pp. 297– 321.

Bruner, J. (1990). Acts of Meaning. London: Harvard University Press.

Clandinin, D. And Connelly, F. (2000). Narrative Inquiry: Experience and Story in Qualitative Research. San Francisco: Jossey‐ Bass Publishers.

Coltman, T., Devinney, T. And Midgley, D. (2008). The value of managerial beliefs in turbulent environments: Managerial orientation and e‐business advantage. Journal of Strategy and Management, 1 (2), pp. 181‐197. rd Creswell, J. (2009). Research design: Qualitative, quantitative, and mixed methods. 3 ed., Sage.

Davenport, T. (1997). Information Ecology. Oxford University Press.

Dewey, J. (1988). Construction and Criticism. In: J. BOYDSTON (ed)., The Later Works, Volume 5 (1929/30), pp. 125‐143, Carbondale and Edwardsville: Southern Illinois University Press.

Dickie, V. (2003). Data analysis in qualitative research: A plea for sharing the magic and the effort. American Journal of Occupational Therapy, 57, pp. 49‐56.

Gabriel, Y. (1993). Organizational Nostalgia – Reflections On ‘The Golden Age’. In: S. Fineman (Ed), Emotion In Organizations, Pp. 118‐141, Sage.

Gabriel, Y. (2000). Storytelling in organizations: Facts, fictions, and fantasies. Oxford University Press.

Gilmore, T. And Seitchik, M. (2001). Consulting to Tensions in a Business Joint Venture. Organisational & Social Dynamics, 2, pp.1‐16. rd Gole, W. And Morris, J. (2007). Mergers and Acquisitions: Business Strategies for Accountants. 3 ed., John Wiley & Sons.

Goleman, D. (2005). Emotional Intelligence. New York: Bantam Books.

Gummesson, E. (2000). Qualitative Methods in Management Research. 2nd ed., Sage.

Heje, P‐H., Baskerville, R. And Venable, J. (2012). A Comprehensive Framework for Evaluation in Design Science Research.

Lecture Notes in Computer Science, 7286, pp. 423‐438.

Hevner, A., March, S., Park, J. And Ram, S. (2004). Design Science In Information Systems Research. MIS Quarterly, 28(1), pp. 75‐105.

Hockey, J. (1993). Research methods ‐‐ researching peers and familiar settings. Research papers in Education, 8 (2), pp.

199‐225.

Huttunen, R. Heikkinen, H. And Syrjälä, L. (2002). Narrative Research: Voices of Teachers and Philosophers. Jyväskylä: SoPhi. Johansson, A. (2001). Translating experience as a basis for reflection and action. Consulting as story making. 2nd HSS Research Conference, Halmstad, 9‐11 May. JOHNSON, G. And SCHOLES, K. (1999). Exploring Corporate Strategy. 5th ed., Prentice Hall.

Jones, S. (2010). Negotiating Authentic Objects And Authentic Selves: Beyond the Deconstruction of Authenticity.

Journal of Material Culture, 15 (2), pp. 181–203.

Kendall, K. And Kendall, J. (1984). STROBE: A Structured Approach to Observation of the Decision‐Making Environment.

Information and Management, 7, pp. 1‐11.

135

Sally Eaves and John Walton Kirkham, R. (1995). Theories of Truth: A Critical Introduction. A Bradford Book.

Mitchell, W. (2005). What Do Pictures Want? Chicago: University of Chicago Press.

Monaco, J. (2009). How to Read a Film. 4th ed., Oxford University Press.

Muñiz, A. And Schau, H. (2005). Religiosity in the abandoned Apple Newton brand community. Journal of Consumer Research, 31 (4), pp. 737‐747.

Neubert, S. (2001). Pragmatism and Constructivism in Contemporary Philosophical Discourse. University of Cologne.

Potolsky, M. (2006). Mimesis. Abingdon: Routledge.

Pratt, M. and RAFAELI, A. (2001). Symbols as a Language of Organizational Relationships. Research in Organizational Behavior, 23, pp. 93‐103.

Remenyi, D. (2005). Tell me a Story – A way to Knowledge. The Electronic Journal of Business Research Methodology, 3 (2), pp 133‐140.

Riessman, C. (2008). Narrative Methods for the Human Sciences. California, USA: SAGE. Rorty, R. (1979). Philosophy and the mirror of nature. Princeton: Princeton University Press.

Ross, L. (2009). Collaboration: the new core competency in an era of joint ventures and alliances. CIMA. Schein, E. (2010). Organizational Culture and Leadership. 4th ed., Jossey Bass.

Schumpeter, J. (1942). Capitalism, Socialism, and Democracy. Harper & Bros.

Tversky, B. (2004). Narratives of Space, Time, and Life. Mind & Language, 19 (4), pp. 380–392.

Vaara, E. (2000). Constructions of Cultural Differences in Post‐Merger Change Processes: A Sensemaking Perspective on Finnish‐Swedish Cases. Management, 3 (3), pp. 81‐110.

Verhesschen, P. (1999) On judging the interpretation. Paper presented at the Annual Meeting of the American Educational Research Association, Montreal, Quebec, April 19‐23.

White, H. (1990). The Content of the Form: Narrative Discourse and Historical Representation. The Johns Hopkins University Press.

Winter, R. (2002). Truth or fiction: problems of validity and authenticity in narratives of action research. Educational Action Research, 10 (1), pp. 143 – 154. http://www.tandfonline.com/toc/reac20/15/1 nd Wittgenstein, L. (2001). Tractatus Logico‐Philosophicus. 2 ed., Routledge.

136

Comparison of Methodologies of CSR Index – Application to the PSI 20 Companies Paula Fernandes1, Ana Paula Monte1, Rui Pimenta2 and Sandra Afonso3 1 Economics and Management Department, School of Technology and Management ‐ Polytechnic Institute of Bragança, Bragança, Portugal and NECE (UBI), Covilhã, Portugal 2 Biostatistics and Health Management Units, ESTSP, Polytechnic Institute of Porto, Porto, Portugal 3 School of Technology and Management ‐ Polytechnic Institute of Bragança, Bragança, Portugal pof@ipb.pt apmonte@ipb.pt rep@estsp.ipp.pt a24528@alunos.ipb.pt

Abstract: Corporate Social Responsibility (CSR) has assumed increasing importance over the past several decades, globally resulting from the companies’ response to the demands of markets and society for responsible growth, both social and environmentally. At a European level, the CSR was initially boosted in 1993 by the former President of the European Commission, Jacques Delors, who called for more socially responsible action from companies. It culminated in 2001 with the publication of the Green Paper on CSR, by the European Commission in which the areas and guidelines for desirable social responsibility practices were defined. The measurement of social or CSR performance has not been consensual, which accounts for the existence of several methods. These use a variety of criteria ranging from expert evaluation to the analysis of reports and other documents or considerations of performance measures of pollution control. Each of these criteria is not exempt from criticism and limitations, as they are one‐dimensional or do not reflect all aspects of CSR. Based on these measures, some evaluation CSR indices were developed, having been used in decision‐making and positioning of the companies in their social and environmental performance. The most commonly used indices are those provided by the rating agency Kinder, Lindenberg and Domini index, or the reputation index of Fortune. This paper aims to compare the application of two methods in defining the levels of CSR from a CSR index, constructed for 19 of the 20 companies comprising the PSI‐20, the Portuguese stock market index, the Euronext Lisbon. One of the methodologies defines five intervals of CSR using the cut‐point method; the other considers a scale of 5 points (5 grades) with constant intervals. In order to construct the index, 247 items were considered, comprising the different dimensions of social responsibility suggested by the European Commission Green Paper on CRS of 2001 and the guidelines of the Global Reporting Initiative.

We analyzed the contents of sustainability reports and financial reports of companies under research in order to assign a score (which will be 0 if the item is not observed, 0.5 if is only partially observed or 1 whether it is observed) for measurement of each item of responsibility, since there are no quantifiable and objective social responsibility indicators.

The CSR index of a company, at any given time, is the sum of the scores across all 247 items. The categorization of companies in the CSR levels depends on the referred methodology to the definition of intervals. In brief, by using the two approaches, we found that the Portuguese companies which were analyzed denote a growing sensitivity in the adhesion and dissemination of their social responsibility practices, investing in a more specific and detailed disclosure and, in particular, complying with the guidelines of the Global Reporting Initiative.

Keywords: corporate social responsibility, CSR index, cut‐point method, PSI‐20

1. Introduction Despite the evidence of social concerns by companies previous to their written conceptualization, Corporate Social Responsibility (CSR) emerges as a formal written concept, in the United States, in the 1950’s, with the publication by Howard R. Bowen, entitled "Social responsibilities of the businessman", marking the beginning of modern literature on CSR (Carroll 1999; Carroll & Shabana 2010).

In the European context, CSR was initially boosted in 1993 by the former President of the European Commission, Jacques Delors, who called for more socially responsible actions from companies. Since then, the European Commission has engaged in several initiatives to promote CSR. The publication of the Green Paper marked the commitment to build a European framework for CSR. It launched the challenge to articulate CSR with the goals set in the European Council of Lisbon 2000, namely to build the world’s most dynamic and competitive economy, based on knowledge, and to create an “European Enterprise”, open to innovation and entrepreneurship. Although these goals have not yet been achieved, they continue to make sense, especially

137

Paula Fernandes et al. in the challenging times we are facing, where more responsibilities are demanded and CSR can play an important role, setting the path to a more responsible and sustainable Europe.

The European Commission, in the Green Paper, considered CSR in a voluntary basis, urging all companies, in a generalized way, to contribute to a more equal society and to a cleaner environment, assuming responsibilities towards employees and stakeholders. Corporate social responsibility was framed in two dimensions: (1) an internal dimension which includes several aspects related with “human resources management", "health and safety at work" "adapting to change" and "management of environmental impacts and natural resources"; (2) an external dimension, including "local communities", "business partners, suppliers and consumers", and "human rights and global environmental concerns" (COM 2001).

In Portugal, although many companies used CSR practices in an informal way, it was only after the celebration of international agreements, more specifically, the European Lisbon Conference of 2000 (CECOA 2004), that it begun being used as a systematic practice. Most of Portuguese companies report CSR based on GRI considering the goals of triple bottom line: economic, social and environmental.

The difficulties to obtain systematic and reliable information, the inexistence of CSR measure models or indexes that could be used in a diverse approach are some of the limitations on researches in this area. This investigation proposes to compare the application of two methods in defining the levels of CSR from a CSR index, constructed for 19 of the 20 companies comprising the PSI‐20, the Portuguese stock market index, the Euronext Lisbon, comprising the different dimensions of social responsibility suggested by the European Commission Green Paper on CRS of 2001 and the guidelines of the Global Reporting Initiative.

To this end, this paper is divided into five further sections apart from the introduction: in the second section it is intended to make an overview on the thematic under consideration after which the methodology that supports the study is presented, followed by the presentation and analysis of the results and, finally, the presentation of the most relevant conclusions of this research work.

2. Background Globalization has increased economic, social and political interdependence, raising awareness to the responsibilities and the role that organizations must undertake in society, considering not only their economic impact but also the way their activities affect environment and society. The greater the power and influence of an organization, the greater the impact of its action its responsibility demands. Pressures, environmental incidents, corruption, unethical conduct and undignifying working conditions are some of the actions that emphasize the need for a more responsible conduct by companies.

According to Zadek (1999), Freeman and Hasnaoui (2011), despite the fact that organizations are increasingly under pressure, the main motivation to turn towards social responsibility relates to organizational changes and a market driven by globalization and technological development. The transparency of business imposed by the media and the new communication technologies are also further reasons pointed by European Commission in this context (COM 2001).

A growing number of investors also require knowledge on companies’ responsibility actions; employees feel more motivated when working for a company with a good image that undertakes initiatives that contribute towards a better society; more consumers choose to buy products and services of socially responsible companies (Maignan, Ferrell & Hult 1999; Cho & Patten 2008; Lama & Muyzenberg 2008; Lioui & Sharma 2012). All these changes encourage organizations to undertake CSR, relating their social performance, and some take the opportunity to improve their image and reputation.

For some, CSR has become a fashion as many multinational companies mention it. Simultaneously, it has become a business and a new profession. However, they argue that for most companies CSR still remains a "cosmetic treatment" (The Economist, 2005, p. 10). Although CSR actions can be based on good intentions, that does not mean that because companies are embracing CSR, that in practice they are really making a difference and being socially responsible. According to Jones and Nisbet (2011) companies select the causes and the social priorities they support from areas favorable to company strategy and market image. In most of the cases, they are identified with shareholder interests not even taking in to consideration problems that are

138

Paula Fernandes et al. caused by business activities. So it is important to measure social performance choosing CSR measurement that comprises a large scale of actions in diverse areas, ensuring a realistic vision.

The measurement of social or CSR performance is one of the main difficulties and has not been consensual, which accounts for the existence of several methods. These use a variety of criteria ranging from expert evaluation to the analysis of reports and other documents or considerations of performance measures of pollution control. Each of these criteria is not exempt from criticism and limitations, as they are one‐ dimensional or do not reflect all aspects of CSR. To Balabanis, Phillips and Lyall (1998), expert evaluating depends on the available information. Analysis of reports can also be questioned because they are internally produced by the organizations, representing their one vision, and measures of pollution control are more representative to industry companies.

Several researches used Kinder, Lindenberg and Domini index or data base and reputation index of Fortune (e.g., Griffin & Mahon 1997; Waddock & Graves 1997; Stanwick & Stanwick 1998; McWilliams & Siegel 2000; Goss & Roberts 2011; Ghoul, Guedhami, Kwok & Mishra 2011; Harjoto & Jo 2011). The Kinder, Lindenberg and Domini index includes CSR information about community relations, employee’s relation, environment, product, considerations with women and minorities, military contracts, nuclear power and South Africa engagement (Griffin & Mahon 1997). This index was considered by Waddock and Graves (1997) as the best one available considering its diverse approach, although they recognize some limitations. Also, for Goss and Roberts (2011), and Harjoto and Jo (2011), Kinder, Lindenberg and Domini index is the most comprehensive and widely accepted on CSR research.

The Fortune index measures the ten greater companies in each activity sector according to eight attributes related with reputation, management quality, product quality, innovation, responsibilities with community and environment (Griffin & Mahon 1997). For Wood and Jones (1997) Fortune index is questionable because it is based on opinions of senior executives, directors and financial analysts, being management quality the most valorized attribute instead of attributes more related with social performance.

European researchers also recognize difficulties in the measurement of CSR. Szekeley and Knirsch (2005), conclude that many German companies adopted the guidelines of the Global Reporting Initiative (GRI), to measure their sustainability, according to economic, environmental and social goals, diverging on the aspects they chose to report. The authors consider GRI as a good start, and a tool to be improved, but not enough for the structural changes that they assumed companies need to undertake internally to become more sustainable and responsible.

Leite and Rebelo (2010) also found difficulties to measure CSR in the Portuguese context. The authors analyzed the implementation of CSR in some Portuguese companies, using five criteria, from certification norms to ratings and awards related with social responsibility. They concluded it was important to set CSR measurement models that could serve, not only to quantify, but also to select the best practices considering sustainability and stakeholders expectations.

In this research, in order to construct a CSR index, 247 items were considered, comprising the different dimensions of social responsibility suggested by the European Commission Green Paper on CRS of 2001, and the guidelines of the Global Reporting Initiative used by most of the Portuguese companies, encompassing and following this approach.

3. Methodology This paper aims to compare the application of two methods in defining the levels of CSR from a CSR index, constructed for 19 of the 20 companies comprising the PSI‐20, the Portuguese stock market index, the Euronext Lisbon. These are Altri, BCP; BES, BPI, Brisa, Cimpor, EDP, Galp, Jeronimo Martins (JM), Mota‐Engil (MOTEN), Portucel, Portugal Telecom (PT), REN, Semapa, Sonae Industria (SOIN), Sonae, SonaeCom (SOCOM), Teixeira Duarte (TEIXDU), Zon. It was considered the time period, 2005 to 2009, in this research.

The 239 items analyzed to construct the CRS index form groups of variables that measure both dimensions (COM, 2001) of CSR, internal (116 items) and external (123 items). The internal dimension (STDI) includes the Responsible Management (15 items), Human Resources Management (36 items), Health and Safety at Work

139

Paula Fernandes et al.

(17 items), Business Ethics (18 items), and Environmental and Natural Resources Management (30 items). The external dimension (STDE) includes Local communities (29 items), Stakeholders (17 items), Human Rights (10 items), Environmental and Philanthropic Global Concerns (10 items), and Other CSR Instruments (57 items).

For that, and considering each year of analysis, the contents of sustainability reports and financial reports of companies under research were analysed in order to assign a score (which will be 0 if the item is not observed, 0.5 if is only partially observed or 1 whether it is observed) for measurement of each item of responsibility, since there are no quantifiable and objective social responsibility indicators.

After adding the scores obtained in the different items of each group of variables according to the methodology described above, we obtained the scores for each group of variables. To obtain the Index of Corporate Social Responsibility (ICSR), we proceeded to sum the scores in each group of variables considered in each of its dimensions. The entire procedure can be represented by the formula below.

Where:

xij = scale item response i in category (group element) j, m = number of items in each category of analysis, and n = number of categories.

In order to measure the dimensions’ score in each year, and for each company, under analysis, the total item (variable) was measured. Then, and for each company, the mean of variable scores for the five years under analysis was calculated, in order to achieve internal and external CSR index. Therefore, to identify the level of CSR, two methodologies based on five intervals of CSR were applied, one using the Cut‐point method; the other considering a scale of 5 points (5 grades) with constant intervals. This scale was obtained using the Sturges’ Rule for 19 observations. The categorization of companies in the CSR levels depends on the referred methodology to the definition of intervals.

4. Results and discussion Applying the methodology described above, the following results were obtained. According to the results shown in figure 1, is possible to observe that all companies under analysis are more focus on internal dimension, as the averages are higher. The highest STDI index was observed in EDP company (70,5), followed

by BES (65,5) and BCP (63,8). In the opposite side, the lowest STDI index, are found for companies Zon (14,6), Teixeira Duarte (21,1) and BPI (21,2). Analysing the STDE index, the highest index was achieved by EDP (47,7), BCP (47,4) and BES (46,8). Zon (8,5), Teixeira Duarte (11,7) and BPI (18,8) were the companies presenting the lowest index. It can be concluded that the companies that presented the highest STDI were the same that presented also the highest STDE Index as for the lowest index.

Analysing figure 1, it can be evidenced that BPI, Portucel, Zon and Teixeira Duarte were those that presented the shortest distance between STDI and STDE indexes, which may indicate that these companies give the same importance for both dimensions even though they have lowscores.

Figure 2 shows the companies that are above and below STDI and STDE index average. To construct this graph, the distance between the company’s STDI index and the STDI index average (48,6 points)was calculated and the same was carried out for the STDE index (average is 31,6 points). Each company is represented according to its distance to the average considering each CSR dimension. The STDI Index distance to its average (ADSTDI) is represented on the horizontal axis and the STDE index distance to its average (ADSTDE) on the vertical axis.

In order to analyze this diagram, the companies positioned in the Quadrant I (ADSTDI and ADSTDE positive) have the “ideal” position, however its position can improved if it to increase the scores in both dimensions. In Quadrant II (ADSTDI negative and ADSTDE positive), the companies may improve its position increasing its scores in STDI index to achieve the “ideal” position. Those companies positioned in Quadrant III (ADSTDI and ADSTDE positive), have the “worst” position as they have to improve in both dimensions. Finally, in Quadrant IV (ADSTDI positive and ADSTDE negative), the companies may improve their position by increasing the scores in STDE index to achieve the “ideal” position.

140

Paula Fernandes et al.

Figure 1: Internal and external corporate social responsibility index According to these results, in figure 2, it was observed that Portucel may improve its position by increasing in the STDI scores in the future. Companies such as Altri, GALP and Jeronimo Martins can improve their position in the external dimension. Those positioned in Quadrant III are Zon, Teixeira Duarte, BPI, Sonae, Sonae Industria, Semapa and Mota‐Engil, the negative distance to average STDI and STDE, indicates that they must improve both dimensions in order to reach higher CSR indexes. The other companies are already positioned in the “ideal” Quadrant.

Figure 2: Positioning of companies with respect to average distance of CRS dimensions

141

Paula Fernandes et al.

In an effort to match a level of CSR for each company the two methodologies described in the previous section were used (one based on Cut‐point method and other on constant ranges). Thus, for both methodologies five levels of CSR were fixed, corresponding to a Likert scale of five points. Those are: 1 ‐ Very low CSR; 2 ‐ Low CSR; 3 ‐ Middling CSR; 4 ‐ High CSR; 5 ‐ Very high CSR. Table 1 shows the range for each CSR level regarding the methodologies in use.

By the total CSR index and according with the constant range methodology it is observed that there are 42,1% of companies classified with Middling CSR and the same percentage with Low CSR, as is shown in table 1.

There were no companies classified in the highest levels. On the other hand, with Cut‐point methodology, the companies were spread among all ranges. This last methodology seems to be more equitable and, therefore, less penalizing.

Table 1: Comparison of categorization of Portuguese companies of the PSI‐20 in CSR level Level of CSR Very Low CSR Low CSR Middling CSR High CSR Very High CSR

Constant Range Range % Companies <= 42 15,8 [43 ‐ 85[ 42,1 [85 ‐ 128[ 42,1 [128 ‐ 171[ ‐ >= 171 ‐

Cut‐point method Range % Companies <= 59,9 21,1 [60 ‐ 78,9[ 15,8 [78,9 ‐ 87,2[ 21,1 [87,2 ‐ 107,9[ 21,1 >= 107,9 21,1

Consequently, it can be said that the majority of the companies under analysis, denoted sensibility to a responsible practice and its accountability. Using the cut‐point methodology, 63,3% of Portuguese companies of the PSI‐20 may be classified as middling social responsible or more while using the constant range only 42% could be classified as so. In this way, the use of one methodology or another may influence the mapping of the companies in each level of CSR due to differences in interval range, that is, for instance, to classify the company in the very low CSR range, the company should present a score lower or equal to 42 (a CSR index lower or equal than 42) in the constant range and 59,9 in the cut point methodology. This methodology, as can be shown in table 1, spreads the companies among intervals and the distribution is less penalizing and does not concentrate the companies in few intervals.

5. Conclusion The main objective of this paper was to compare the application of two methods in defining the levels of CSR from a CSR index, constructed for 19 of the 20 companies comprising the PSI‐20, the Portuguese stock market index, the Euronext Lisbon. One of the methodologies defines five intervals of CSR using the Cut‐point method; the other considers a scale of 5 points (5 grades) with constant intervals. These methodologies to the definition of intervals categorized the companies in the CSR levels differently. It was found that the Portuguese companies which were analyzed denote a growing sensitivity in the adhesion and dissemination of their social responsibility practices, investing in a more specific and detailed disclosure and, in particular, complying with the guidelines of the Global Reporting Initiative.

These different methodologies categorized the companies in the CSR levels differently. The methodology of Cut‐point yielded greater discrimination in classification practices of corporate social responsibility in the Portuguese context. Nevertheless, it was found that the Portuguese companies which were analyzed, in both methodologies, denote a growing sensitivity in the adhesion and dissemination of their social responsibility practices, investing in a more specific and detailed disclosure and, in particular, complying with the guidelines of the Global Reporting Initiative.

This paper presents the development of a new model for measuring CSR which is a starting point for building a European index evaluation of CSR, according to the dimensions defined by the European Commission. The model was tested in the Portuguese context. Two different methodologies were analyzed and their results compared. In a future work, we intend to expand the application of the index to an international level with particular emphasis on the European context.

Some limitation should be taken into consideration. In the methodology of calculation of ICSR followed in this paper, the different groups of variables assume different weights in the index since the number of items in each group is considered sometimes substantially different. If we assume that each group and each item has the same importance, we must weigh each group by the number of groups considered in each dimension and

142

Paula Fernandes et al. the number of items in each group. We will review this issue and its impact in terms of the relative positioning of the companies in future work.

We are aware that the construction of indices and rankings will always be subject to some criticism and objections due to the multiplicity of approaches and tool analysis. However, the possibility of making them publicly available supply instruments that must be taken into consideration and discussed. The score on the ICSR should preferably be addressed by the company with a view to continuous improvement, regardless of the alternative analysis tools. More important than analyzing the absolute positioning of the company in terms of ICSR is that the index will enable change practices in order to obtain a better position in future editions.

References Balabanis, G., Phillips, H. and Lyall, J. (1998). Corporate social responsibility and economic performance in the top British companies: are they linked?. European Business Review, 98 (1), 25‐44.

Carroll, A. (1999). Corporate social responsibility: Evolution of a definitional construct. Business Horizons, 38 (3), 268‐295.

Carroll, A. and Shabana, K. (2010). The business case for corporate social responsibility: A review of concepts, research and practice. International Journal of Management Reviews, 1, 85‐105.

CECOA. (2004). Responsabilidade social das empresas ‐ Estado de arte em Portugal. Centro de Formação Profissional para o Comércio e Afins. Consulted November 11, 2011, http://www.sairdacasca.com/recursos/docs/CECOA.pdf.

Cho, C. and Patten, D. (2007). The Role of Environmental Disclosures as Tools of Legitimacy: A Research Note. Accounting, Organizations and Society, 32, 639‐647.

COM (2001). GREEN PAPER: Promoting a European framework for Corporate Social Responsibility. COM 366. Brussels.

Commission of the European Community. Consulted October 15, 2010, from http://eur‐ lex.europa.eu/LexUriServ/site/en/com/2001/com2001_0366en01.pdf.

Freeman, I. and Hasnaoui, A. (2011). The Meaning of Corporate Social Responsibility: The Vision of Four Nations. Journal of Business Ethics, 100 (3), 419‐443.

Ghoul, S., Guedhami, O., Kwok, C. and Mishra, D. (2011). Does corporate social responsibility affect the cost of capital?.

Journal of Banking & Finance, 35 (9), 2388‐2406.

Goss, A. and Roberts, G. (2011). The impact of Corporate Social Responsibility on the cost of bank loans. Journal of Banking & Finance, 35 (7), 1794‐1810.

Griffin, J. and Mahon, J. (1997). The corporate social performance and corporate financial performance debate. Business and Society, 36 (1), 5‐31.

Harjoto, M. and Jo, H. (2011). Corporate governance and CSR nexus. Journal of Business Ethics, 100, 45‐67.

Jones, B. and Nisbet, P. (2011). A better model for socio‐economic governance?. Revue de la régulation, 9. Online since 15 June 2011, http://regulation.revues.org/9158.

Lama, D. and Muyzenberg, L. (2008). O caminho para a liderança (Trad. S. Oliveira). Alfragide: Publicações Dom Quixote.

(Original work published in 2008).

Leite, C. and Rebelo, T, (2010). Explorando, caracterizando e promovendo a responsabilidade social das empresas em Portugal. Proceedings of the Eighth National Symposium on Research in Psychology, Universidade do Minho, Portugal.

Lioui, A. and Sharma, Z. (2012). Environmental corporate social responsibility and financial performance: Disentangling direct and indirect effects. Ecological Economics, 78, 100‐111.

McWilliams, A. and Siegel, D. (2000). Corporate social responsibility and financial performance: correlation or misspecification. Strategic Management Journal, 21, 603‐609.

Stanwick, P. and Stanwick, S. (1998). The relationship between corporate social performance, and organizational size, financial performance and environmental performance: An empirical examination. Journal of Business Ethics, 17, 195‐204.

Szekeley, F. and Knirsch, M. (2005). Responsible leadership and corporate social responsibility metrics for sustainable performance. European Management Journal, 23 (6), 628‐647.

The Economist. (2005). The good company. Consulted July 17, 2011, http://www.economist.com/node/3555212.

Waddock, S., and Graves, S. (1997). The corporate social performance ‐ Financial performance link. Strategic Management Journal, 18 (4), 303‐319.

Wood, D. and Jones, R. (1995). Stakeholders Mismatching: A Theoretical problem in empirical research on corporate social performance. International Journal of Organizational Analysis, 3 (3), 229‐267.

Zadek, S. (1999). Stalking Sustainability. Greener Management International. 23, 1‐12.

143

Assessing the Learning Style Preferences of Postgraduate General Business Management Students Using VARK Elena Fitkov‐Norris and Ara Yeghiazarian Kingston University, Kingston‐Upon‐Thames, UK e.fitkov‐norris@kingston.ac.uk a.yeghiazarian@kingston.ac.uk

Abstract: This article sets out to assess the learning style preference profile of an international cohort of business management students on a master’s degree in a UK institution using the VARK learning style questionnaire. The effect of demographic factors such as gender, past educational and work experience and age on learning style preference is examined with the intention of identifying appropriate support mechanisms for students learning that could be implemented on this programme. Results show that the most prevalent learning preference for this cohort of general business management students are kinaesthetic and auditory, with significantly fewer students using read/write and visual approaches. The educational and cultural background of students does not play a significant role in determining students learning preferences, although students from masculine cultures or those without any prior work experience are significantly more likely to have a strong preference for a single learning modality. As students with single mode preferences tend to underperform as compared to their multimodal peers, lecturers should attempt to provide the learning materials in a variety of different ways to support these higher risk students.

Keywords: learning preferences, VARK, learning modality, postgraduate, education, business management

1. Introduction The UK higher education landscape has changed significantly over the last 10 year by becoming competitive and with recent reforms in UK tuition fees increasing the pressure of universities to recruit students from abroad (Altbach et al. 2010). International students face many hurdles adapting to the host country culture and work ethics (Carty et al. 1998) and this transition is particularly challenging for masters level students who join intensive year‐long postgraduate programmes of study and have significantly shorter periods to adjust to different teaching and assessment methods (Morrison et al. 2005). Therefore universities need to put in place adequate academic teaching and learning mechanisms that will enable international students to fulfil their potential (Leontidis & Halatsis 2009). Research has been carried out on aspects of teaching style, students learning preference and styles, classroom climate and student motivation and attempted to quantify the impact of these factors individually and in conjunction with each other on students’ engagement, performance and skills development (Coffield et al. 2004; Davies 1997). While the evidence of the advantages of matching students learning preferences and styles with teaching methods is not consistent (Dobson 2010), there is some indication that aligning the teaching style with the students learning preferences may bring benefits to learners (Popescu 2009), with students being happier when taught in their preferred learning style (Haq et al.

2012). Furthermore, evidence is emerging that there exists strong association between a student’s perceived learning style and their performance on a programme of study (Hsieh et al. 2012), highlighting the need to ensure that teaching materials are presented in a variety of styles.

The modal learning preferences of undergraduate and postgraduate level physiology, nursing, dental and medical students from different cultures and backgrounds have been studied extensively using the VARK learning modality questionnaire (Dobson 2010; Barman et al. 2009). The results show that some aspects of students’ backgrounds such as English language proficiency or the setting where the students grew up (metropolitan or rural) have significant associations with specific learning preferences (Koch et al. 2011; James et al. 2011). Some researchers report no association between gender and modal learning preferences (Ates & Altun 2008; Baykan & Nacar 2007), while others report significant modal learning preference differences between male and female students, although no consensus exists weather males or females are more likely to have a unimodal preference (Isman & Gundogan 2009: Wehrwein et al. 2007).

While these results offer some insight into the learning approaches taken by students, and how the modality preferred by students could be influenced by their background and gender, the inconsistency in results suggest that there may be some significant cohort or discipline specific aspects that make generalisation to students from other disciplines tenuous. A search of the academic literature did not identify any studies that have used

144

Elena Fitkov‐Norris and Ara Yeghiazarian the VARK questionnaire to assess the modal learning preferences of internationally diverse postgraduate cohorts of students on general business management degrees.

This article aims to bridge this gap in two ways. First, it explores the distribution of learning preference modalities for a cohort of post graduate business students and identifies the most prevalent learning modality when Fleming’s VARK questionnaire is used. Second, it analyses the strength of the relationship between learning modality and students’ gender, work experience and cultural background using Hofstede’s culture dimensions.

2. Culture and education Evidence shows that business administration students from France and Germany demonstrate significantly different learning approaches, indicating that cultural and educational background may have an impact on the way students learn (Barmeyer 2004). Cultural traditions may also limit the efficacy of educational practices transported from one culture to another (Richardson 2004) or influence students’ scientific epistemological beliefs (Lin et al. 2013). Advances in cognitive neuroscience have shown the strong influence of culture on brain structure, resulting in cross‐cultural differences in the mechanisms underlying cognitive functions such as arithmetic and reading (Ansari 2012).

Culture is defined by Geert Hofstede as “… the collective programming of the human mind that distinguishes the members of one human group from those of another. Culture in this sense is a system of collectively held values.” (Hofstede 1980). Hofstede identified five dimensions to national culture: power distance, individualism versus collectivism, masculinity versus femininity, uncertainty avoidance and long versus short term orientation (Minkov & Hofstede 2011). The five cultural dimensions have also been linked by Hofstede to educational culture as described below (Hofstede 2008):

Power distance reflects the extent to which the less powerful members of institutions and organisations accept that power is distributed unequally. In countries with large power distance students depend on teachers, who initiate communication and are treated with respect, education is teacher centred and teachers are guru’s who transfer knowledge. In countries with a small power distance teachers and students treat each other as equals. Teachers are experts who impart knowledge in a student‐centred education and more student‐initiated communication is present.

In masculine societies emotional gender roles are distinct, with men being assertive and tough, and women being more concerned with the quality of life. In feminine societies, men and women are supposed to be modest, tender and focused on the quality of life. In masculine societies brilliant teachers are admired, competition and being the best in class is encouraged, and failing in school is considered a major setback. In feminine societies, friendly teachers are liked the most, and over‐ambition is unpopular.

Individualistic societies have loose ties between individuals, with everyone expected to look after self and immediate family, as opposed to a collectivistic society, where individuals are part of strong in‐groups that last a lifetime. In individualistic societies educational systems encourage initiative, with students expected to speak in class when they need or want to with the purpose of education being to learn how to learn. In contrast, in collectivistic societies, the purpose of education is to learn how to do things, and students are discouraged from speaking in class or using their initiative.

Uncertainty avoidance reflects the extent to which members of a culture feel threatened by ambiguous or unknown situations. In cultures with strong uncertainty avoidance students want to know the right answer, and expect the teacher to have it; students are under pressure to conform, while in culture with weak uncertainty avoidance, students want good discussion and are tolerant to differences in class.

Long‐term orientation is the extent to which individuals adapt themselves and are prepared to avoid gratification to reach a desirable future. In long term orientation cultures students are more likely attribute their success to effort, studying hard is the norm and, as a rule, students perform well in mathematics and applied science subjects. In short term orientation countries students may attribute success to luck, with pursuit of enjoyment being normal and students performing well in theoretical, abstract sciences.

145

Elena Fitkov‐Norris and Ara Yeghiazarian

3. Learning styles and VARK modalities The research field of learning styles is extensive but also conceptually confusing and marked by some contradictory results. However, there is some consensus from learning style theories that learners have different preferences in the way that the information is learnt. It is now accepted that students learn in individual and different ways, absorbing and processing information in a unique manner, which suits the individual’s personality and is influenced by their culture (Lin et al. 2013).

An extensive review of the different learning style instruments can be seen in Coffield et al. (2004) and a pragmatic way to present different models in just three different groups was developed by Curry (1987) who used the shape of an onion (Figure 1).

Figure 1: Curry’s onion model The inner core of the diagram represents cognitive learning styles, which describe an individual’s habitual and distinct mode of acquiring knowledge. Instruments designed to measure learning styles attempt to evaluate psychological aspects of learning and the complex cognitive thinking and information analysis strategies used by the learners (Coffield et al. 2004). The second layer defines how information is processed and retained by a learner, while the top level shows the instructional preference of the learners, i.e. the mode used by the individual to collect information from the outside environment. Instructional preferences act as an “input” to learning and thinking. VARK is a tool for measuring the preference of people for four different types of information acquisition, namely Visual, Aural, Read/Write and Kinaesthetic. While most modern style theories focus on the cognitive process aspects of learning styles and take a holistic/ analytical approach to learning on one dimension and a visual/verbal approach to learning on another dimension (Riding, 2001) which limits their practical appeal, VARK was designed with the objective of helping students understand their preferred learning style and provide them with strategies that compliment their preferences, hence allowing them to gain more from their studies (http://www.vark‐learn.com/english/page.asp?p=helpsheets). The VARK questionnaire was designed by Fleming in 1987 and has undergone a number of modifications and improvements and the latest version has been validated (Leite et al. 2010). Students who have a preference for just one of the four styles are termed unimodal, students who use two or three learning styles, are bimodal and trimodal, respectively, and students who use all four modes of learning are multimodal (Fleming, 2009).

4. Method This study collected data on students’ educational background and learning preferences using a two‐part questionnaire. The first part gathered information about the students’ background, including their educational history, age, gender and work experience. The second part of the questionnaire assessed the students’ learning style preferences using Fleming’s VARK questionnaire version 7.1 (http://www.vark‐ learn.com/english/index.asp). The educational background of the students was categorised based on the country where the students did their undergraduate degree, as this was their most recent educational experience. Each country was described in terms of the five dimensions defined by Hofstede (2008), namely:

146

Elena Fitkov‐Norris and Ara Yeghiazarian power distance (large or small), masculinity level (masculine or feminine), individualism (individualistic or collectivistic), uncertainty avoidance (weak or strong) and orientation (short, medium or long term).

The analysis was carried out using IBM SPSS Statistics version 19.0. Association between nominal variables was assessed for significance using the χ‐squared test for association, while Analysis of Variance and Paired Samples T‐tests were used for analysis of the learning preferences across the different groups.

5. Results A total of 105 students completed the questionnaire and the demographic and cultural characteristics of the sample are shown in Table 1.

Table 1: Demographic and cultural characteristics of sample

Age Gender

Time lived in the UK

Subject of completed UG degree

Work experience

Culture Dimensions

Male Female Less than 1 year 1‐3 years 3 years and more/Lifetime Business & Management Art & Design Social Sciences Science, Engineering & Computing No Yes Small Power Distance Large Weak Uncertainty Avoidance Strong Individualist Individualist vs. Collectivist Collectivist Masculine Masculine vs. Feminine Feminine Short term Orientation Medium term Long term

N

%

Mean

33 72 60 23 22 61 6 23 13 48 57 40 63 51 52 44 59 58 45 34 13 26

31.4% 68.6% 57.1% 21.9% 21.0% 59.2% 5.8% 22.3% 12.6% 45.7% 54.3% 38.8% 61.2% 49.5% 50.5% 42.7% 57.3% 56.3% 43.7% 46.6% 17.8% 35.6%

26.89

Standard deviation 5.24

As can be seen fro Table 1, the average age of the students was 26 years and the majority were female. The students come from a wide range of countries with a fairly large contingent from UK, although approximately half the students (57%) have lived in the UK for less that a year and therefore have not been exposed to the UK educational culture for an extensive period.

5.1 Learning preference distribution As can be seen from Table 2, when assessed on the frequency of choice of the four learning styles, the most prevalent learning preferences for this sample of business postgraduate students are auditory and kinaesthetic (t = 1.32, df = 100, two tailed p = .187). Significantly fewer students had a read/write preference (t = ‐2.36, df = 101, two tailed p = .004) with visual being the least popular (t = ‐6.22, df = 97, two tailed p = .000).

Table 2: VARK preference modes

Mean Standard deviation Visual 4.44 2.51 Auditory 6.56 3.03 Read/Write 5.42 2.79 Kinaesthetic 6.17 2.72

147

Elena Fitkov‐Norris and Ara Yeghiazarian The learning style preferences of business students classified by the VARK assessment tool show that a large proportion of students (38.1%) do not have a particular preference and use all four modes (Figure 2). Just over half of students (58.1%) use a unimodal approach to learning, a small minority used bimodal. The sample did not contain any students who used three modes. Of the 61 students who preferred a unimodal approach, the majority (46%) used aural, while the remaining three modes, kinaesthetic, visual and read/write, were spread equally among students (with approximately 18% expressing preference for each mode).

Figure 2: VARK learning preference distribution The most common strength of learning preference across postgraduate business students was mild, used by 48.6% (Figure 3). The percentage of students who used all four modes equally was also relatively high (38.1%).

Only a small minority of students reported at least one strong or very strong preference (13.3%).

Figure 3: Strength of VARK learning preferences

5.2 Relationship between culture, educational background, gender, work experience and learning styles The analysis showed no significant association between culture dimensions and number of modes used by learners. There is also no evidence to support the existence of association between power distance, uncertainty avoidance, individualism and collectivism and orientation and the strength of the learning preferences exhibited by the students. However, there is evidence of a significant association between the masculinity/femininity of the culture and the learning preferences used by students (χ2= 6.791, df = 2, p = .034) with students from feminine cultures more likely to adopt a mild learning preference while students from masculine cultures more likely to adopt a strong learning preference.

No significant association was found between gender and the number of modes used by students or the strength of the students learning preferences. In a similar vein there was no association between students’ previous degree subject and learning preference, but a significant association was found between work experience in the field of study and learning preference (χ2= 7.125, df = 2, p = .028), with students who had

148

Elena Fitkov‐Norris and Ara Yeghiazarian previously worked in their field of study being less likely to exhibit a strong learning preference compared to students who did not have previous work experience.

6. Discussion 6.1 Learning preference distribution The findings showed that in terms of frequency of occurrence for the four basic learning modalities, business students utilised kinaesthetic and auditory learning modes to the same extent, with significantly fewer uses of the read/write mode, while visual mode was the least popular. Similar high preference for kinaesthetic mode of learning has been found in by a number of researchers in examining nursing students or students interested in the health profession (James et al. 2011; Turner et al. 2011; Breckler et al. 2009). Only one study has reported a significantly high aural preference in their students (Koch et al. 2011) and this study investigated the preferences of postgraduate students and also had a sample with high proportion of female students. This may be an indication that as students’ progress they gradually improve their capacity to use the auditory mode of learning. On the other hand it is possible that the results are biased and this difference is due to females having a higher preference for auditory learning. However, this sample did not support this hypothesis as no significant differences were found between males and females in the frequency choice of the four modal preferences. Interestingly, the postgraduate business students did not score very highly on read/write preference, which is utilised extensively by nursing and medical students at both undergraduate and postgraduate level (Koch et al. 2011; James et al. 2011; Turner et al. 2011; Breckler et al. 2009; Alkhasawneh et al. 2008). This suggests that there could be a difference in the skills set required by the two disciplines and warrants further investigation. The visual mode of learning was least popular both among business and medical and nursing students (Koch et al. 2011; James et al. 2011; Turner et al. 2011; Breckler et al. 2009; Alkhasawneh et al. 2008). The high preference of business and management student for kinaesthetic and aural modes of learning suggests that lecturers have to ensure that their sessions are highly interactive (Turner et al. 2011) to keep the students engaged and increases the students satisfaction with their studies. In addition, the finding suggests that business postgraduate students are likely to benefit from lectures and learning using auditory input. Thus a voice‐over a set of PowerPoint slides could significantly increase the effectiveness of lecture slides as a revision tool, for example.

The learning preferences of students classified using the VARK questionnaire show that, surprisingly, the majority of business postgraduate students were unimodal (58.1%), followed by multimodal, and very few bimodal preferences. Other surveys of postgraduate dental and medical students have reported multimodal majority (Koch et al. 2011; Tantawi 2009) and preference for multi modality seems to be prevalent in undergraduate medical and physiological students (James et al. 2011; Breckler et al. 2009; Koch et al. 2011).

Although there is some evidence that as students mature, they develop a preference for multimodal learning (Meehan‐Andrews 2009), the majority of students on engineering and science related degrees tend to prefer unimodal approach to learning (Eudoxie 2011; Ictenbas & Eryilmaz 2011). The variation in this cohort may be due to variations in the students’ background as 12.6% of students within the sample had engineering or science background. Unfortunately, the small sample size precludes interpretation of the results to test this hypothesis for statistical significance between UG degree subject and learning preferences and highlight the diverse backgrounds of the students undertaking postgraduate business studies.

Classification of students in terms of the strength of their preferences showed that a small minority of students (13.3%) have at least one strong or very strong learning preference. Although comparison with other studies is not possible as other researchers have not reported this characteristic explicitly, there is some evidence that students with balanced and multimodal profiles perform better than their unimodal peers (Eudoxie 2011; Alkhasawneh et al. 2008). This implies that international postgraduate business students with single strong learning preference may be at a higher risk of under‐performing on the course (Bartram & Bailey 2010; Bartoli 2011). Lecturers have to balance carefully the presentation of their materials to ensure that they support the students with strong unimodal preferences, who may be at a higher risk of failure while at the same time ensure that these students have an opportunity to develop their ability to utilize alternative preference modes.

149

Elena Fitkov‐Norris and Ara Yeghiazarian

6.2 Gender and learning preferences No significant difference was found between gender and the modal learning preferences of postgraduate business students. These results corroborate previous studies which have shown that although a larger proportion of male students are likely to have a unimodal preference (either mild or strong), compared to their female peers (who tend to be multimodal), the difference is not significant (Dobson 2010).

6.3 Association between culture and learning preferences Many aspects of the cultural background of postgraduate business students such as power distance, uncertainty avoidance, individualism and collectivism and orientation do not have a significant association with students learning preferences, both by number of modes used and strength of the learning preference. This suggests that the nature of interactions between students and teachers has limited impact on students learning modalities. So students exposed to teacher centred education, accustomed to getting the right answer and discouraged to speak up in class would utilise the same set of learning modalities as students exposed to student centred education, accustomed to ambiguity and encouraged to speak up in class. The findings are at odds with earlier work by (James et al. 2011) who found that the background (rural or metropolitan) of nursing students had a significant association with their choice of learning modality and suggest that learning modalities to a large extent are inherent, rather than an aspect of culture and training (Isman & Gundogan 2009; Koch et al. 2011).

Only the masculinity vs. femininity dimension of culture associated significantly with learning modality preferences with students from feminine cultures being significantly less likely to exhibit strong preferences, compared to their counterparts from masculine cultures. As masculine cultures are likely to encourage competitiveness among students and put pressure on them to do their best, it is possible that student utilise any means of learning that offers them a competitive advantage, instead of developing a range of learning approaches. Students from feminine cultures, on the other hand, do not work under the same pressure to excel and may have more incentive to explore various learning approaches. As students with strong learning preferences are at a higher risk of under‐performing, when compared to their peers with multiple learning preferences, this finding highlights the need for adequate support of postgraduate business students from a masculine cultural background who, faced with a new culture may be at risk of being overwhelmed by the pressure to perform.

6.4 Work experience and learning preferences The results show that postgraduate students with some work experience are significantly less likely to have a strong learning preference as opposed to their counterparts without any wok experience in their field of study.

Experience with problem solving in real life situations seems to encourage students to evaluate possible solutions from a variety of angles and thus encourages students to utilise all learning modes. This is confirmed by research which showed that utilising problem‐based learning in class changes students learning preferences and encourages students to develop more balanced profiles (Alkhasawneh et al. 2008). This finding has implications for postgraduate business admissions tutors who may be hesitant to accept students who did not distinguish themselves academically during their first degree, but who have some work experience in their intended field of study. If these students have indeed developed a preference to use all four learning modalities when problem solving in the workplace, then they could be well equipped to cope with the challenges of postgraduate study.

7. Conclusions and future work This explorative study set out to examine the learning modality preferences of postgraduate business students and examine the association between the students’ cultural demographic background and their learning preferences. The study identified a significant association between the strength of the students learning preferences, their cultural background and the extent of their work experience in the field of study. These findings highlight that there is scope to improve the academic support to international postgraduate students by exposing them to a range of learning resources that would encourage development of the students learning modalities overall such as case studies, problem based learning and blended learning. Future work will focus on trying to further understand the complex relationship between cultural background and learning modality preferences by carrying repeat studies with a larger sample size to ensure that students from all learning

150

Elena Fitkov‐Norris and Ara Yeghiazarian modalities are represented in the study as well as utilising more in depth research methods such as interviews and focus groups.

References Alkhasawneh, I.M. et al., 2008. Problem‐based learning (PBL): Assessing students’ learning preferences using vark. Nurse Education Today, 28(5), pp.572–579.

Altbach, P.G., Reisberg, L. & Rumbley, L.E., 2010. Trends in Global Higher Education, Published with support from SIDA/SAREC. Ansari, D., 2012. Culture and education: new frontiers in brain plasticity. Trends in Cognitive Sciences, 16(2), pp.93–95.

Ates, A. & Altun, E., 2008. Learning styles and preferences for students of computer education and instructional technologies. Egitim Arastirmalari ‐ Eurasian Journal of Educational Research, 8(30), pp.1–16.

Barman, A., Jaafar, R. & Rahim, A.F.B.A., 2009. Medical Students' Learning Styles in Universiti Sains Malaysia. International Medical Journal, 16(4), pp.257–260.

Barmeyer, C., 2004. Learning styles and their impact on cross‐cultural training: An international comparison in France, Germany and Quebec. International Journal of Intercultural Relations, 28(6), pp.577–594.

Bartoli, A., 2011. Assessment and International Students–black African Social Work students. Enhancing the Learner Experience in Higher Education, 3(1), pp.45–58.

Bartram, B. & Bailey, C., 2010. Assessment preferences: a comparison of UK/international students at an English university.

Research in Post‐Compulsory Education, 15(2), pp.177–187.

Baykan, Z. & Nacar, M., 2007. Learning styles of first‐year medical students attending Erciyes University in Kayseri, Turkey.

AJP: Advances in Physiology Education, 31(2), pp.158–160.

Breckler, J., Joun, D. & Ngo, H., 2009. Learning styles of physiology students interested in the health professions. AJP: Advances in Physiology Education, 33(1), pp.30–36.

Carty, R.M. et al., 1998. Teaching international nursing students: Challenges and strategies. Journal of Professional Nursing, 14(1), pp.34–42.

Coffield, F. et al., 2004. Learning Styles and Pedagogy in Post‐16 Learning, Learning and Skills Research Centre.

Davies, S.M., Rutledge, C.M. & Davies, T.C., 1997. The impact of student learning styles on interviewing skills and academic performance. Teaching and Learning in Medicine, 9(2), pp.131–135.

Dobson, J.L., 2010. A comparison between learning style preferences and sex, status, and course performance. AJP: Advances in Physiology Education, 34(4), pp.197–204.

Eudoxie, G.D., 2011. Learning Styles among Students in an Advanced Soil Management Class: Impact on Students' Performance. Journal of Natural Resources and Life Sciences Education, 40, pp.137–143.

Fleming, N., 1995. I'm different; not dumb Modes of presentation (V.A.R.K.) in the tertiary classroom. In A. Zelmer, ed.

Research and Development in Higher Education, Proceedings of the 1995 Annual Conference of the Higher Education and Research Development Society of Australasia (HERDSA), HERDSA. pp. 308–313.

Foster, N. et al., 2010. Assessing the Influence of Gender, Learning Style, and Pre‐entry Experience on Student Response to Delivery of a Novel Veterinary Curriculum. Journal Of Veterinary Medical Education, 37(3), pp. 266–275.

Haq, S.M. et al., 2012. Students' Learning Styles Require Modified Teaching Strategies. Journal of Rawalpindi Medical College (JRMC), 16(2), pp.191‐193.

Hofstede, G., 2008. Cultural Differences in Teaching and Learning. In FUHU conference on Education and Training in the Multicultural Classroom.

Hofstede, G., 1980. Culture and organizations. International Studies of Management & Organization, 10(4), pp.15–41.

Hsieh, C., Mache, M. & Knudson, D., 2012. Does student learning style affect performance on different formats of biomechanics examinations? Sports Biomechanics, 11(1), pp.108–119.

Ictenbas, B.D. & Eryilmaz, H., 2011. Determining Learning Styles Of Engineering Students To Improve The Design Of A Service Course. Procedia ‐ Social and Behavioral Sciences, 28, pp.342–346.

Isman, C.A. & Gundogan, N.U., 2009. The influence of digit ratio on the gender difference in learning style preferences.

Personality and Individual Differences, 46(4), pp.424–427.

James, S., D'Amore, A. & Thomas, T., 2011. Learning preferences of first year nursing and midwifery students: Utilising VARK. YNEDT, 31(4), pp.417–423.

Koch, J. et al., 2011. Learning preference as a predictor of academic performance in first year accelerated graduate entry nursing students: A prospective follow‐up study. YNEDT, 31(6), pp.611–616.

Leite, W.L., Svinicki, M. & Shi, Y., 2010. Attempted Validation of the Scores of the VARK: Learning Styles Inventory With Multitrait‐Multimethod Confirmatory Factor Analysis Models. Educational and Psychological Measurement, 70(2), pp.323–339.

Leontidis, M. & Halatsis, C., 2009. Integrating Learning Styles and Personality Traits into an Affective Model to Support Learner’s Learning. Advances in Web Based Learning–ICWL 2009, pp.225–234.

Lin, T.‐J. et al., 2013. High school students’ scientific epistemological beliefs, motivation in learning science, and their relationships: A comparative study within the Chinese culture. International Journal of Educational Development, 33(1), pp.37–47.

Minkov, M. & Hofstede, G., 2011. Hofstede's Fifth Dimension: New Evidence From the World Values Survey. Journal of Cross‐Cultural Psychology, 43(1), pp.3–14.

151

Elena Fitkov‐Norris and Ara Yeghiazarian Morrison, J. et al., 2005. Researching the performance of international students in the UK. Studies In Higher Education, 30(3), pp.327–337.

Popescu, E., 2009. Evaluating the impact of adaptation to learning styles in a Web‐based educational system. Advances in Web Based Learning–ICWL 2009, pp.343–352.

Richardson, P.M., 2004. Possible influences of Arabic‐Islamic culture on the reflective practices proposed for an education degree at the Higher Colleges of Technology in the United Arab Emirates. International Journal of Educational Development, 24(4), pp.429–436.

Riding, R. (2001). Nature and effects of cognitive style. In R. Sternberg & L. F. Zhang (Eds.), Perspectives on thinking, learning, and cognitive styles (pp. 39–59). Lawrence Erlbaum Associates Inc. Tantawi, El, M.M.A., 2009. Factors affecting postgraduate dental students’ performance in a biostatistics and research design course. Journal of Dental Education, 73(5), pp.614–623.

Terri A. Meehan‐Andrews, 2009. Teaching mode efficiency and learning preferences of first year nursing students. YNEDT, 29(1), pp.24–32.

Turner, D.A. et al., 2011. Do pediatric residents prefer interactive learning? Educational challenges in the duty hours era.

Medical Teacher, 33(6), pp.494–496.

Wehrwein, E.A., Lujan, H.L. & DiCarlo, S.E., 2007. Gender differences in learning style preferences among undergraduate physiology students. AJP: Advances in Physiology Education, 31(2), pp.153–157.

152

The Limits of Relativism in Managerial Thinking Simona Hašková and Pavel Kolář University of Economics in Praque, Faculty of Management in Jindřichův Hradec, Czech Republic haskovas@post.cz kolar@fm.vse.cz

Abstract: The diversity of the broad field, which management is, is reflected in the diversity of approaches, tools and methods of argument used. Therefore, we meet here with a “vague description of the general concepts”, by which faith is mainly spread, and with the “case‐based reasoning” approach, which is focused on changing faith to knowledge. Both of these extreme approaches in managerial thinking are useful utilizing different tools. However, the problem occurs if the segment of the field with the first approach overlaps with the segment of the field of the second approach and the approaches do not communicate and are unable to agree on the result. The deadlock state, where each one has its own truth and does not care about the truths of the others too much, is known as relativism. This state does not benefit the general development. The way to avoid it is mutual communication, which has to lead to a discussion focused on the presentation and the defense of one’s own views of the world. If such a discussion should be fruitful, the attitudes of both sides must be taken into account. It also has to deal with the question of how each one reaches their judgments (the individual thinking processes, in which people convince themselves) and how others refer to their judgments (i.e. how people want to convince others about the validity of their judgments). As a part of this debate the question of the maximum possible extent, in which to maintain relativism while at the same time agreeing on the common result, should be then raised and answered. This contribution raises this question and tries to give a general answer. The search of relativism limits is the search for acceptable truths for both sides. Therefore, initially, the terms "faith", "truth" and "knowledge" are stated in the required context. Then the essential features of the above‐mentioned extreme approaches are outlined – the “case‐based reasoning” approach and the “vague description”, including other possible types of the distortion of intuitive judgments. On this basis a proposal of the general rules is then formulated, observance to which should facilitate the interdisciplinary communication leading to the achievement of mutual agreement.

Keywords: management, relativism, case‐based reasoning, judgment, deductive validity, inductive validity

1. Introduction The development of the division of labor in the field of economics and management has been and still is accompanied by the emergence of new disciplines and specializations in the university faculties. Each specialization has a tendency to create its own tools tailored to the problems it needs to solve. Hand in hand with this the specializations of the same field tend to have different objectives, which, in some cases, may even be contradictory. For such specializations, for instance, can be regarded environmental management (the aim ‐ achieving sustainable development by the economical use of resources with adequate consumption) and marketing (the aim ‐ increasing turnover by the generating of maximum consumption). And so as we are witnessing a clash of various ethnic and other cultures we also witness in the field of economics and management a clash of different styles of thinking and related methods of argumentation. In addition to arguments on the basis of logical deduction based on consequence, which leads to "absolutely valid" conclusions, in the field of management various inductive methods based on generalization are widely used.

The conclusions of these procedures then show different degrees of validity and thus credibility. Instead of "absolutely valid" conclusions are presented only the conclusions "relatively valid”, which allow alternative views of the same thing. In economics it results in, for example, completely opposing views of various experts of government economic policy. As the specific long‐time experience of authors with teaching management calculations at the Faculty of Management at the University of Prague have shown, in the academic field the answer to the same question can vary dramatically depending on in which department or specialization the problem is solved; the consequences of that can result in disagreement in the assessment of bachelor thesis, diploma thesis and scientific works.

In response to what J. Peregrin mentions in (Peregrin, 2008 and 2011a) in general, here come into play once again one of the old philosophical problems, called relativism: “What necessarily is true for me, is it true for another? Or is it possible for everyone to have their own truth?" The negative answer to the first question does not imply a positive answer to the second question. A logical consequence of total relativism would be the end of communication. If mutual communication is needed for life and for the development of each scientific field, then this need necessarily requires (Peregrin, 2008 and 2010) that any justified relativism has

153

Simona Hašková and Pavel Kolář some limitations. If the limit of permissible relativism should be where agreement is still possible, it cannot be determined without taking into account how each reaches its own judgments, i.e. individual thinking processes, in which people convince themselves, and how individuals refer to their judgments, i.e. how they want to convince others of the validity of their judgments.

For insight into the virtual thought processes the authors of this paper have drawn primarily from the bestseller by Daniel Kahneman in (Kahneman, 2011), Nobel Prize winner of Economics in 2002, which the author bases on the knowledge of social psychology and cognitive science, and the results of a number of specific case studies, where he demonstrates miracles and also mistakes of intuition, on which we all mostly subconsciously rely in our deliberations. A good tool for assessing which types of arguments, justifications or evidence are acceptable and which are not within the professional communication, is logic. The essential that we rely on in this context mainly is the system of classical propositional logic and its extensions and alternatives, as presented in (Peregrin, 2004; Priest, 2000 and 2008).

The search of relativity limits is the search for acceptable truths for all. In the following, we first put the faith, truth and knowledge in the required context. Then we focus on the demonstration of the advantages of the “case‐based reasoning” approach. We avoid the pitfalls of vagueness and intuitive reasoning, and within a limited range of this contribution we formulate the basic rules, the respect of which can facilitate, in which the authors believe, reciprocal persuasive reasoning and the interdisciplinary communication when solving the problems.

2. Faith versus knowledge Some modern logicians (e.g. R. Smullyan, 2012) define knowledge as a justified belief. The faith itself is, according to them, just a matter of the individual psyche: “I can believe in what I want”. To know then means to believe correctly ‐ that what I believe must be valid, i.e. must be true. What in this sense is currently true and what is not true depends little on the individual. Mainly this is given by the state of affairs of the current world. The consistency of the state of affairs of the world with the faith justifies faith to be the right faith, i.e. knowledge. This consistency is the reason that negotiation on the basis of knowledge has a chance to be more successful than negotiation based on mere faith.

Spreading the faith is possible by declamation, in which everything fits together and creates a coherent story.

Confidence in the faith is given mainly by the coherence of the story and by one’s own feelings on the subject.

There is no need to have all the evidence. To the illusion of validity of faith often suffices that people who we like or in whom we believe hold the same views (Kahneman, 2011). In contrast, to “knowledge” the mere illusion of validity is not enough. For validity or truth there must be an objective fact. Objective facts are communicated through statements. Unlike unrestrained mouthing or declamation the statement is a form of expression, with which a certain responsibility is associated (Peregrin, 2011b), which is a commitment to justify the statement convincingly if necessary.

The standard method of convincing reasoning by derivation on the basis of resulting in modern logic was systematically developed by Gottlob Frege (e.g. in Kolman, 2002) more than a century ago. Today the situation is such that there is one standard or "classic" logic on the propositional and predicate level together with a number of "non‐classical" logics arising in an effort to better capture different aspects of what we find in natural language. The systematic review of the most important of them can be found in (Peregrin, 2004). Every logic has its own syntax, axiomatics and semantics. Syntax defines the sentences of the language of logic.

Axiomatics states what can be inferred from “what”. Semantics is a system assigning truth characteristics to sentences. The relationship between axiomatics and semantics of the logic is the guarantor of the transfer of premise (axioms, theorems and assumptions) to the conclusion in each step of the derivation process.

Consequently, each conclusion derived from all valid premise, is also valid.

Thus managerial calculations lead to valid conclusions in the case studies solved by the “case‐based reasoning” approach, which retains and uses the specifics of the particular tasks. This approach on the theoretical level is based on the extended axiomatics of intuitionistic logic with interpretation in modal Kripke semantics (Peregrin, 2004). On the application level it consists of the conclusions of proven constructive elements while proceeding “from bottom upwards", much like building a house of bricks.

154

Simona Hašková and Pavel Kolář

3. The “case‐based reasoning” approach The aim of theoretical research is usually not to solve one specific problem but a class of similar problems. In contrast, the manager usually faces the need to solve his particular problem as effectively as possible. Let us consider the following illustrative example taken from the thesis (Hašková, 2010), the coauthor of this contribution: In a square room with the side a should be placed a rectangular runner of width c < a in such a way that its corners touch all four walls of the room. The question is: What should be the length x of it?

Since the procedure of solving the general version of the problem for a rectangular room with the sides a and b is known (see Figure 1), an approach can be offered "from a general to a special" considering an additional 4 2 2 2 2 condition a = b in the condition for the solution. By this the condition changes to x – 2∙(a + c )∙x + 4∙a ∙c∙x – 2 2 4 2∙a ∙c + c = 0. After substituting the relevant lengths for a and c by an iterative method or by the appropriate computer software application, e.g. Maple we find the value of x. As we can see, solving a special case by means of this approach made it not any easier than by dealing with the more general case, which it includes.

It will be different if we begin to solve the task from "bottom" instead of from "top". If we proceed from Figure 2, the solution is immediately clear and simple. The length of the runner can be determined without iteration and without a computer. The fact that its result is the same as in the previous approach can be 4 2 2 2 2 verified by substituting the expression (√2 · a – c) for x in the propositional form x – 2∙(a + c )∙x + 4∙a ∙c∙x – 2 2 4 2∙a ∙c + c = 0. From it arises the true statement that is owing to the values 0 < c < a the tautology. Although it is not certain if the relation x = √2 · a – c is deducible from the condition x4 – 2∙(a2 + c2)∙x2 + 4∙a2∙c∙x – 2∙a2∙c2 + c4 = 0, it does not make sense to even attempt it. It would be difficult, and moreover unnecessary.

x4 + (–b2 – a2 – 2c2)∙x2 + 4∙a∙b∙c∙x – c2∙(a2 + b2) + c4 = 0

Figure 1: The sketch of the runner position and the condition that the solution must fulfill The comparison of both solutions, the first is based on Figure 1 and the second on Figure 2, confirms the empirical knowledge that the "tailor‐made" solutions from the very beginning are more efficient than the additionally modified general solution. Prof. R.L. Ackoff in (Ackoff, 1987) reflected this opinion in the thesis, which says that interesting problems for us, and important in practice, often have properties that allow considerable simplification and economization in finding a solution, and which became the basic credo of the “case‐based reasoning” approach. Unlike the canonical approach of theorists who, in an effort to affect mainly “the common” delete the specifics of tasks, the “case‐based reasoning” approach builds on the specifics (Ackoff, 2006). On the logic level it is possible to state that while the theorists are more interested in the theorems derivable from undeniable truths (axioms), the practitioners, which managers belong to, have in their premise in addition to the axioms the assumptions that take into account the specifics of tasks. From the premise containing the valid assumptions can be derived more valid conclusions than from premise without assumptions. Greater choice leads to greater adequacy of the solution.

155

Simona Hašková and Pavel Kolář

x = √2 · a – c Figure 2: Illustrations for solving the same task by case‐based reasoning approach The criterion of the validity of the judgment in the “case‐based reasoning” approach is the derivability from the premise, that places stricter requirements on derivation than conventional logic, by means of which it becomes the constructive element. If it proves impossible to deduce the judgment from premise, it does not mean that the negation of the judgment from the premise has to be derivable and therefore valid. Not to be valid may not be the same thing as being invalid, which contradicts the law of excluded middle (Tertium non datur) of classical logic. This along with the already mentioned constructivism approaches the logic of “case‐ based reasoning” to the intuitionistic logic. Although the intuitionistic logic already contains partly modality, any uncertain or temporal judgments need to be taken into account, which require an explicit extension of that side of things (Peregrin, 2000).

The problem‐solving by the “case‐based reasoning” approach is mainly presented in the form of case studies, in which individual subsequent partial solution steps are justified in detail. For instance, when deciding on acceptance or rejection of the project or other business plans according to the quantitative criteria of economic efficiency, the justifications tend to have the form of elementary and detailed calculations made by the tools of financial mathematics on the basis of the adequate models of specific situations. The specific examples of such case studies can be found in (Hašková, 2010; Hašková and Kolář, 2011; Hašková and Kolář, 2012; Hašková, 2012).

In an effort to reveal the core of the problem within the “case‐based reasoning” approach, it pays great attention to the selection of the point of view on the solved problem. What it is is illustrated by the following simple example: The manager responsible for organizing a tennis tournament wants to know how many games will be played if 100 participants are registered. He can solve it by constructing the structure game plan ‐ the spider of the tournament: In the final there will be one match. In the semifinal there will be two matches, four before it, etc. In the first round there are 64 places for 100 players. That means that 28 players will be put into the first round and the other 72 players will compete in the preliminary round to advance to the first round to assign the remaining 36 places (28 + 36 = 64). Then he will calculate: 1 + 2 + 4 + 8 + 16 + 32 + 36 = 99. The result is: 99 matches should be played. From another perspective he could say: If we have a single winner then 99 players must lose in the tournament. Since just one player loses in each match, 99 matches must be played.

Finding the suitable angles of view is more art than science. How it is possible to use the specifics of particular tasks Prof. Ackoff demonstrates in (Ackoff, 1987) in many examples (mini studies). The thesis " knowledge will emerge from many solutions” outlines the path to mastering this art, on which principle neural networks partially work and that is the opposite of the thesis of the standard school frontal teaching “from knowledge emerges the solution". A number of observations, comments, suggestions and insights on this topic can be found both in (Ackoff, 2006; Ackoff and Addison, 2007; Hašková, 2010). The first of the mentioned theses, which is also the fundamental thesis of the philosophy of the “case‐based reasoning” approach, recently achieved significant success in the field of artificial intelligence.

156

Simona Hašková and Pavel Kolář The computing system Watson, which is based on it, defeated the two most successful human champions in the world‐renowned knowledge competition Jeopardy! (e.g. Watson (computer), Wikipedia). In this competition the questions are not repeated from the previous rounds, and so for the computer memory, to which among other things, an encyclopedia of all disciplines was saved, the core of the problem in terms of communication was to “understand" the question in human language. The effort of leading experts experimenting with different methods and approaches, which was accompanied by a series of failures, also concurs with this thesis. Within its framework the saved past questions in the memory were linked to the correct answers by the adaptable semantic graphs. In the subsequent phase of the experimental learning the system fine‐tuned its semantic graphs on the basis of evaluating its own mistakes, on which basis it generated the most likely correct answers. Although the moment that the creators could finally enthusiastically exclaim "it understood it" remained a long way off, WATSON ultimately won in the public contest and beat two human opponents; thus becoming the "smartest machine in the world."

4. Pitfalls of vagueness and intuition By the vagueness of the predicate of property or relation we mean a certain degree of uncertainty that is allowable on the basis of the assumption that small changes to objects will not have any influence in the attribution of a predicate (Priest, 2010). Neglecting the specifics of concrete tasks has within the general considerations a similar effect to bringing in the phenomenon of vagueness in the definitions. Figure 3 is an illustration of the origin of such an illusion, which may cause the phenomenon of vagueness.

Figure 3: The origin of an illusion as a result of vagueness In Figure 3 there are plotted two completely identical right‐angled triangles highlighted by a thick line. From the upper triangle it is possible to cut a rectangle of a six square height of the underlying network. By this the original triangle is broken into three parts: the mentioned rectangle, the upper smaller and the left larger triangle. Furthermore, the rectangle can be cut into two different parts according to the lines marked therein.

The original large triangle then splits into four parts. In the lower part of Figure 3 these four parts are put together in the triangle ground plan so that there a space area occurs of the dimension of four colored squares of the underlying network. The sum of the mentioned four areas is less in the latter configuration than in the first configuration as it does not cover the original layout. It says that the summing of areas of planar surfaces is generally not commutative as by their rearrangement we can get a different sum. The result can be "verified" experimentally. It is very likely that at a reasonably accurate slitting of the upper triangle and by careful rearrangement of its parts according to the instructions the experimenter will find the same thing.

The analysis by the “case‐based reasoning” approach reveals that this "knowledge" is only an illusion: The rectangle of the 6 square height in the upper part of the picture has its upper left vertex on the hypotenuse of the underlying triangle and so it must have the same width x as the length of the horizontal cathetus of the small triangle lying above it. Therefore, from the similarity of triangles x = 4 ∙ 26 / 10 = 10,4

157

Simona Hašková and Pavel Kolář squares instead of the 10 squares that are plotted there. This 4 % inaccuracy was covered by the vagueness of the outline, i.e. the uncertainty of the thick line. At the bottom of the picture then the width y of the rectangle should be analogously y = 6 ∙ 26 / 10 = 15,6 squares, instead of the 16 squares plotted there. This inaccuracy is covered, this time in the opposite direction, again by the vagueness of the outline. The absolute values of both inaccuracies are 0,4 of the square and they eliminate each other. Therefore, the picture seems to be correct. If we draw the picture exactly, we would get Figure 4.

Figure 4: The exact result by rearranging the parts of the picture The correct width of the rectangle in the upper part of Figure 4 extends the small triangle in comparison to Figure 3 of the 0,4 square and shortens the big one by the same amount. It extends the rectangle in the bottom of the picture about the same value. The result of the recommended rearranging of the four parts is then obvious. For the coinciding of the hypotenuses of the partial triangles it is necessary to reduce the width of the rectangle by cutting the column from one or the other of the side of the 0,8 square width, by means of which it will be possible to move both partial triangles together. The cut column has the area of 4 ∙ 0,8 = 3,2 squares, which is exactly the same as the blank area of 2 ∙ 1,6 = 3,2 squares in the rectangle. By cutting off the cut column in the middle of the height we get two strips of the area of 2 x 0,8, which exactly fill the empty space. The sum of the areas in the second case is then the same as in the first case, so it does not cause any distortion of the commutative law of the summing of planar surface areas.

The emergence of the illusion in Figure 3 is due to the fact that even though all of the construction steps to the result, i.e. cutting the upper triangle and rearranging its parts when forming the bottom triangle, are valid within permissible vagueness, they lead to an invalid conclusion. In Figure 4 the vagueness was not allowed, and therefore the illusion did not occur. The phenomenon of vagueness "annoys" also in the logic (Svoboda and Peregrin, 2009) called the Sorites paradox (Paradox of the heap). Given that the cause of this paradox is the non‐differentiation between the complete and the nearly complete truth, it is often technically solved within the framework of fuzzy logic (see examples in Priest (2000) and in more detail in Hájek and Novák (2003)).

In management the limits of the vagueness are not limited “from above”, which can be demonstrated in the example of the predicate "to be a source of funding”. In many textbooks (e.g. Valach (2010)) we can read that depreciation is the source of financing. This claim, however, ignores the fact that:

The final phase of the financing is the payment in money and there is no way to transform the depreciation to money. Believing that it is possible is almost the same as believing that you can still buy something for the banana that you just ate.

Depreciation can reduce outflows of made money, e.g. by reducing the state tax levy, or by preventing any undesirable use, thus keep it for the desirable usage. However, the depreciation can not make money.

158

Simona Hašková and Pavel Kolář Despite the fact that the thesis has not been proven empirically, i.e. no specific case of funding from depreciation is known, except for easily refutable misunderstandings and mistakes, and is not the conclusion of any deductive or inductive valid judgment, i.e. there are counterexamples to it, some believe in it. The confidence in faith, which is not justified, is the result of subjective thought processes, by which people convince themselves. Thus we move from the objective to the subjective ‐ to the ideas of how the mind works.

The author of the model of mind presented in (Kahneman, 2011) assumes that the completion of a conclusion is the result of the collaboration of two hypothetical systems, System 1 and System 2, which different people use to a different degree. System 1, functioning automatically, spontaneously and constantly, generates the intuitive processes in the mind (associations) and is a sort of machine for quick conclusions. This "machine" is monitored and controlled by intentionally functioning, and therefore only at times, System 2, which is energy‐ intensive, slower, and which only allows some of the outputs of System 1 to enter the consciousness; others being suppressed or modified. We can say that problem solving by the “case‐based reasoning” approach keeps System 2 fully occupied. The strength of System 1 that is responsible for intuitive thinking is the speed, ease and immediacy, by means of which we reach intuitive conclusions, and the ease with which we can believe them. However, there are also various pitfalls hidden within it in the form of the systematic distortion of judgment. Besides the already discussed vagueness, often accompanying the analogical reasoning, it also includes the different types of halo‐effect (the exaggerated emotional coherence satisfying an unconscious need to harmonize the view of the world), the principle WYSIATI (What You See Is All There Is), the effect of anchoring or framing, the heuristic of availability, various forms of priming, the substitution of questions, the hindsight distortion, the affective heuristics, the illusion of understanding, skills, validity or focus, and many others.

Daniel Kahneman in (Kahneman, 2011) collectively calls the latter set of illusions the "illusion of experts." He points out that the subjective confidence in one’s own judgment is not proof of its correctness. He points out that the word “to know” can only be used if we are able to substantiate or prove what we think. And then in a number of well‐documented studies he shows the devastating results of brokers, financial and investment advisors and other economic experts whom he reproaches in that they often "ignore their ignorance" (both the known unknowns and the unknown unknowns) and in that they are unduly confident. In this respect he also says: "The professional culture strongly supports the illusion of validity and skills. We know that people can unfailingly trust any opinion, no matter how absurd it is if they are supported by a community of like‐minded people. Due to the professional culture of the financial community it is not surprising that there are a large number of people who believe that they belong among the select few who can do what others cannot do."

The community of academics who write textbooks and the heuristics of availability support the explanation of why people, who have no practical experience with financing companies from internal resources, can believe that the depreciation is the source of financing. If people have no association with funding then it results in the fact that they will not recognize the difference between the depreciation and the liquid components of profit.

They believe in textbooks and their faith, to which they are anchored, and advocate by means of superficial, confused and easily refuted arguments. To those who have the experience with funding, the associations are available, but none of them are concerned with the depreciation. They feel the imbalances in the availability, are without exception skeptical of the thesis and refute it in their arguments.

5. Proposals for general rules of the game Respect of the beneficial degree of relativism in managerial thinking requires that if there is a dispute it should be settled on the basis of technical discussions by both sides, by means of which contradictory statements will be analyzed. Each participant of the discussion should be equipped with the minimum qualification for such a debate, which R. Smullyan implies in (Smullyan, 2012), and with the knowledge of the difference between the statement on the one hand and the non‐binding mouthing or declamation on the other (Peregrin, 2011):

The ability to listen, understand and record the argument, i.e. do not let it go unnoticed;

The ability of self‐awareness, i.e. when you learn something, to be able to recall that you know it;

The ability to produce real statements in terms of (Peregrin, 2011).

In practice, this is not as obvious as it may seem at first glance. The statement is associated with the commitment to justify it if necessary, i.e. when it is challenged, questioned or denied. The problems are usually with both self‐awareness and reasoning.

159

Simona Hašková and Pavel Kolář

The requirements for reasoning:

In the hypothetic‐deductive model of the scientific cognition the deductive validity judgments either logical, analytical or constructive are assessed on the basis of resulting, which usually has the character of mathematical derivation;

In the case of inductive reasoning within which it is operated with the degrees of validity on the basis of probability, the judgment is valid if and only if p(c/a) > p(⌐c/a), where c is the conclusion, ⌐c its negation, a is the conjunction of premises, p(c/a), p(⌐c/a) are the conditional probabilities. The statement with a higher degree of validity is preferable;

In the case of general statements that fail to be verified, it is necessary to attempt to prove their falsification by a counterexample – the statement can be considered valid unless it is disproved by the counterexample.

In the case of intuitive considerations, it is necessary to eliminate as far as possible the factors distorting the conclusion. For this purpose it is appropriate:

To take into consideration the agreement on what is and what is not necessary when solving the problem and then to keep to it, which would minimize the distortion caused by the factor WYSIATI and by the heuristics of availability;

To prefer the “case‐based reasoning” approach when justifying the conclusions, to avoid consistently unconstructive considerations and concepts (see for example the concept of the Laffer curve, etc.), to have an constructive at least hypothetical example to each statement, to consider only within the framework of essential vagueness;

To consider a degree of the distortion of conclusions caused by substitution of questions within the models of solved problems.

In the opinion of the authors the three stated proposals should be sufficient basis for the formulation of rules in terms of hypothetical imperatives (see Peregrin, 2011) tailored to the concrete subject of the dispute.

6. Conclusion A historically evolving variety of approaches, tools, methods of teaching and reasoning in modern management helps on the one hand to successfully manage the issues of such a broadly focused field, to which management undoubtedly belong. However, on the other hand, where the segments overlap, the different views on the same thing may occur for the same reasons depending on which segment of management deals with the problem. If this becomes the subject of a dispute, it needs to be solved and a solution agreed upon.

The aim of this paper is to highlight the existence of problems of this kind, to indicate their nature and possible ways of finding a solution. The nature of the problems is the existence of different truths from different perspectives. Each perspective has its own specifics; it prefers something and it overlooks something else. Each segment defends its own point of view, from which the phenomenon called relativism enters the game. To agree on something in order to resolve the dispute it is not necessary to regard everything as the same while getting rid of the positives that relativism brings to segments. It is necessary to find its proper degree. In the search for the degree the authors of the contribution analyze the problem from objective and subjective viewpoints and then they formulate the general rules, observance of which, as the authors believe, will help to find the right degree. The consequences of this will contribute to the increase of success in managerial decisions.

References Ackoff, R. L. (1987) The Art of Problem Solving: Accompanied by Ackoff's Fables, John Wiley, New York.

Ackoff, R. L. (2006) A Major Mistake That Managers Make, [online], Wikipedia® http://www.acasa.upenn.edu/A_MAJOR_MISTAKE.pdf Ackoff, R. L. and Addison, H. J. (2007) Managemen f‐Laws, Triarchy Press, United Kingdom.

Hájek, P. Novák, V. (2003) „The Sorites Paradox and Fuzzy Logic“, International Journal of General Systems, Vol. 32, pp.

373‐383.

Hašková, S. (2010) The Fundamentals of Theory and Praxis of Managerial Calculations, Dissertation, University of Economics in Prague, [online] https://www.vse.cz/vskp/21678_Z.

160

Simona Hašková and Pavel Kolář Hašková, S. (2012) „The rational approaches to economization of natural resources”, Sborník příspěvků z vědecké konference, Nakladatelství Oeconomica, Jindřichův Hradec.

Hašková, S. and Kolář, P. (2011) “The mathematic modeling of process economization of natural resources”, The th Proceedings of the 29 International Conference of Mathematical Methods in Economics 2011, Professional Publishing, Prague.

Hašková, S. and Kolář, P. (2012) „Determination of mutually acceptable price of used manufacturing equipment”, The th Proceedings of the 30 International Conference of Mathematical Methods in Economics 2011, Silesian University in Opava.

Kahneman, D. (2011) Thinking, Fast and Slow, Farrar, Straus and Giroux, New York Kolman, V. (2002) Logika Gottloba Frega, nakladatelství Filosofia, Praha.

Novotný, M. (1988) S algebrou od jazyka ke gramatice a zpět, Academia, Praha.

Peregrin, J. (2004) Logika a logiky, nakladatelství Akademie věd České republiky, Praha.

Peregrin, J. (2008) Filosofie pro normální lidi, nakladatelství Dokořán, s.r.o., Praha.

Peregrin, J. (2011a) Myšlení a pravidla, nakladatelství Dokořán, s.r.o., Praha, [online] www:jarda.peregrin.cz Peregrin, J. (2000) „Absolute and Relative Concepts in Logic”, Logica Yearbook 2000, pp 71‐77, [online] http://jarda.peregrin.cz/mybibl/PDFTxt/424.pdf Peregrin, J. (2010) „Following the Rules of Discourse”, International Review of Pragmatics 2, [online] http://jarda.peregrin.cz/mybibl/PDFTxt/553.pdf Peregrin, J. (2011b) „Logic and Reasoning“, [online] http://jarda.peregrin.cz/mybibl/PDFTxt/576.pdf Priest, G. (2000) Logic: A Very Short Introduction, Oxford University Press Inc., New York.

Priest, G. (2008) “Many‐Valued Modal Logic: a Simple Approach”, Review of Symbolic Logic 1, pp. 190‐203, doi:10.1017/S1755020308080179 Priest, G. (2010) “Inclosures, Vagueness and Self‐Reference”, Notre Dame J. Formal Logic, Vol. 51 Number (1), pp. 69‐84, doi:10.1215/00294527‐2010‐005 Smullyan, R. M. (2012) Forever Undecided: A Puzzle Guide to Gödel, Knopf, New York.

Svoboda, V., Peregrin, J. (2009) Od jazyka k logice: filozofický úvod do moderní logiky, nakladatelství Academia, Praha.

Thagard, P. (2005) Mind: Introduction to cognitive science, The MIT Press, Massachusetts.

Valach, J. (2010) Investiční rozhodování a dlouhodobé financování, nakladatelství Ekopress,s.r.o., Praha.

Watson (computer) From Wikipedia, the free encyclopedia [online] http://en.wikipedia.org/wiki/Watson_(computer)

161

Happy on the Year Abroad? A Mixed Qualitative Methods Approach to Evaluating the Worth of Home Institution Staff Visits to Undergraduates on their Year Abroad Deborah Knowles and Karen Kufuor Westminster Business School, University of Westminster, London, UK d.s.knowles@wmin.ac.uk kufuork@wmin.ac.uk

Abstract: This paper presents the research methods and brief interim findings of a small study on the value of university staff visits to international partner HE institutions in relation to students’ experience of the Year Abroad within their Business and Management degree. Literature in the fields of both Higher Education and Research Methods is used to provide a guide to the philosophy and practice of this study. The Year Abroad itself is shown to bring a number of personal and career‐enhancing benefits to students. A salient issue in this area is that despite the many positive aspects discussed in the literature, it is less popular among UK and US non‐language students than with those from other countries. The central theme of this Research Methods paper, however, is an explanation of the methodological concerns and the methods employed in a transparent fashion in order to show how the findings and conclusions of the study of a complex phenomenon in which the researchers have an avowed “interest” may be clearly seen to derive from rigorous research. A number of issues in the research process are addressed, including the various methodological, contextual and organizational factors which impact on the choice of methods, the extent to which methods are mixed, and the real‐life messiness of the process. The research population of university staff and students comprises two broad groups not only with a number of different perspectives on the subject matter, but also requiring different methods of participation in the study. The paper discusses one type of method which has been unsuccessful in terms of response rate, showing what attempts are made to remedy this situation. Brief interim findings from the primary methods show that the staff members interviewed are broadly in agreement about benefits to students and the home institutions, whereas students so far reveal a tension between the need to support students on their Year Abroad and allowing them their independence.

NB: The conference presentation will include further findings.

Keywords: higher education; institutional visits, international partner institutions, qualitative research, research methods, year abroad

1. Introduction This paper presents the research methods used in a small study of the complex phenomenon of university staff visits to international partner HE institutions. The emphasis is on a mixed methods, qualitative approach which seeks to make sense of how people subjectively perceive their “social world” (Hesse‐Biber, 2010:455; Morse, 2010). Pritchard (2012:135) depicts articles on mixed qualitative methods research as typically obscure about how and why methods have been combined due to a lack of clarity about the research methods employed and an absence of the researcher’s voice in explaining what was done. This study contributes to knowledge of the university Year Abroad; however, this paper is intended to contribute to the Research Methods literature on the experience of designing and conducting research in real‐life complex situations. We address a number of issues in the research process including the various factors which impact on the choice of methods, the extent to which methods are mixed, and the real‐life messiness of the process.

Students on a UK four‐year, non‐language, undergraduate degree “with Year Abroad” typically spend their third year either studying at an overseas partner institution in both semesters or in a combination of study in one semester and work placement in the other, whilst remaining students of their UK institution. Whilst on their Year Abroad they are often found to be unaware of many of the important issues that affect them (Fane, 2012). In an attempt to enhance their experience, universities provide copious advance information; include preparation for the Year Abroad in the curriculum; remain in touch with students during their Year Abroad; and maintain intranet advice sites. Staff may also visit students during their Year Abroad but this seems to occur less often than formerly, although students at our Business School are routinely visited at least once on UK work placements. The study discussed in this paper addresses a gap in knowledge of the value of such international visits by student‐facing staff, and its findings are intended to inform policy making on this practice.

162

Deborah Knowles and Karen Kufuor

2. Background The 2012 UK Quality Code for Higher Education published by the Quality Assurance Agency for Higher Education (QAA) includes as one of its eight “Key Values” that “Staff are supported, enabling them in turn to support students’ learning experiences” (QAA, 2012a:unnumbered fifth page). Staff visits to overseas partner institutions are not a requirement of the QAA. In the relevant section of the Quality Code there is, quite properly, an emphasis on due diligence relating to educational standards and financial viability. The only paragraph which hints at home institutional staff having direct knowledge of the facilities offered by the overseas partner relates to their responsibility for ensuring the accuracy and accessibility of information about such opportunities (QAA, 2012b:44).

Promoting the attractions of studying abroad on a student exchange at an overseas partner institution is an important aspect of internationalising the undergraduate curriculum, a key issue in Higher Education in recent years (Ghasempoor et al, 2011). A Year Abroad is a “life‐changing experience which can have a positive effect on students’ long‐term career prospects” by increasing students’ ability to acquire “internationally transferable skills and knowledge” (British Council, 2012, Crossman and Clarke, 2010; Jackson and Chapman, 2012:54; see also Johnston et al, 2005).

A number of studies investigate the benefits of the Year Abroad experience, identifying a number of personal and professional (CV‐enhancing) benefits to studying abroad (Crossman and Clarke, 2010; Hackney at al, 2012).

The three‐fold benefits of the Year Abroad proposed by Johnston et al (2005) include growth in self‐confidence as a result of utilising problem‐solving skills in the new environment; and the learning acquired from exposure to a different culture which challenges existing perceptions. Both these types of learning extend the student’s critical faculties. They are relevant to the context of non‐language degrees such as those in the Business School which aim to equip graduates with a global mindset. The third benefit is, of course, enhanced linguistic capability. Hackney et al (2012) found that students on International Business degrees were most likely to have the characteristics including openness, knowledge of cultural differences and low levels of ethnocentrism which increase their willingness to study abroad. In the context of international, post‐graduate tourism students who were studying abroad in the UK, Brown (2009) concludes that the benefits are an increase in inter‐cultural competence and a change in “self‐understanding”.

Although studying abroad is very popular with students internationally, non‐language students from UK and US universities are less likely to participate than students from other countries. (British Council, 2012; Hackney et al, 2012; Toncar et al, 2006). Even within the context of UK language degrees the Year Abroad is no longer considered an uncontested necessity (Canning, 2004). However, participation rates are reported to be growing in both countries, with a 150% growth in the US in the 2000‐2010 decade and a 6.5% increase in the UK from 2010‐11 to 2011‐12 (British Council, 2012; Hackney et al, 2012).

International institutional links depend on human efforts to maintain them after the formal business of establishing the association has concluded. Those who are directly concerned with this are the academic and administrative staff. Notwithstanding Shore and Groen’s (2009) autoethnographic study of a trusting personal relationship between academic staff at the heart of successful international partnership, which they find gives them pleasure in their working lives, and despite a continuing academic interest in the Year Abroad as a student experience, there is a dearth of research on effects of home staff visits to overseas partner institutions. This study is intended to address that gap in knowledge.

3. Research question What is the value of home‐staff visits to overseas partner institutions?

Who benefits and how do they benefit?

What are the disadvantages?

Could the benefits be achieved in any other way?

4. Methodological concerns In discussing how this study was conducted it is important to distinguish between “methods” – what we did – and “methodology” – the study of methods, or why we did it (Bryman, 2008).

163

Deborah Knowles and Karen Kufuor Previous research on the international experience of HE staff and students shows a common thread of small studies with a great emphasis on the participants’ voice (Brown, 2009; Crossman and Clarke, 2010; Shore and Groen, 2009). It is clear that in researching a complex issue, the relativist ontological position of reality depending on whose perspective is viewed is a practical one in the search for evidence. The social constructionist epistemological approach is intended to increase understanding of the phenomenon by foregrounding the various stakeholder interests (Easterby‐Smith et al, 2012:19 and 24). An important further consideration in the research design is the degree of the researchers’ subjectivity in the topic and in the process. In this study the involvement of the researchers as academic course leaders of students who spend a Year Abroad and in maintaining relationships with the overseas partner institutions leads to an ethnographic approach (Yanow et al, 2012).

The over‐riding purpose for this study is to understand the arguments about the value of staff visits to international partner institutions in order to come to an understanding of whether (or to what extent) it is worth continuing with them. It is therefore important to gather the opinions and experiences of the various stakeholders. The data must encompass multiple perspectives, those of students and of various student‐facing professionals involved in organising the experience of the Year Abroad. These must be set within the framework of secondary data from institutional and public documentation on the Year Abroad.

As this is a study which needs to “make sense of the subjective and socially constructed meanings expressed about the phenomenon being studied”, it is already evident that a qualitative, interpretive approach is required. The context of findings, which is accessible in qualitative data but “stripped” from quantitative data, is therefore salient in establishing their relevance (Guba and Lincoln, 1998:197; Saunders et al, 2012:163). It is important to find ways of “enter[ing] into” the thinking of the participants”. In this study the methodology is also required to acknowledge that the “observer[s are] part of what is being observed” (Crossman and Clarke, 2010:603; Easterby‐Smith et al, 2012:25).

Brown (2009) used an ethnographic method in a study of international post‐graduate students at her own university. She was able to get to know them over a period of time and to observe them in a naturalistic setting. In the case of the present study an ethnographic approach is inherent in our involvement in the students’ preparation to go on the Year Abroad and in our contact with them during their period overseas. This is not a central feature of the method, but it is important to show that data collection has occurred in the context of a broader and deeper understanding of “the field”.

The circumstances of the researchers’ avowed involvement – even “interest”, given our desire to provide the best possible experience for our students – seem unusual, and almost mischievously counter to the detached researcher of the Positivist school (Easterby‐Smith et al, 2012). It seems to us that even within “objective” scientific research there is always a researcher interest, such as that of the medical researcher who is searching for a cure for disease to benefit humanity including him/herself and his/her family. However, in acknowledging our admission it is important to demonstrate the rigour of the study by showing that the appropriateness of the methods, the presentation of findings and the logic of conclusions will withstand close scrutiny.

The choice of method depends on methodological concerns of the types of data which are most likely to lead to a recognisable answer to the research question(s), and also norms within the field of study. However, Buchanan and Bryman (2007) point out that the carrying out of these ideals must also take into account what is possible in the context. Organizational, personal, political, historical and ethical concerns must be considered. The research team consists of two very busy academics. Our organizational and personal concerns mean that we lack the time to interview a large sample.

The historical and political concerns here refer to the previous practice of insistence on staff visiting overseas partners on an annual basis. The intention in the past was that all students would be visited on their Year Abroad. This has now dwindled to almost no visits at all since 2010.

In this study, although the issue of home institution visits to partner institutions seems innocuous, there are particular ethical concerns. The students in the home institution are made aware that their responses contribute to an academic study which is intended to inform policy decision making. It is not connected in any

164

Deborah Knowles and Karen Kufuor way with assessment on their degree 1 . The “call” for respondents has been posted in much the same way as the usual academic and social notices. The university’s Research Ethics procedure is being followed.

Sampling is an important consideration with regard to research quality. It is clear that the types of participant are determined by the research question: it may concern only one group or a variety of people with different roles and/or perspectives. Data may be collected in different ways from diverse groups as long as this is made clear in discussion of methods and findings. Caveats about the unrepresentativeness of the small sample must be oft‐repeated, but with assurances as to the transparency of method which offers the reader the opportunity to gauge the usefulness of the conclusions.

The differences in situation of the stakeholders require different methods to elicit their views and documentary evidence is required to complete the study. The usual application of mixed methods research designs is that of blending qualitative and quantitative methods together in order to gain data which consists of both words and numbers. Discussion of these typically focuses on their sequencing or the establishment of a hierarchy of importance (Bryman, 2008; Morse, 2010; Pritchard, 2012). Saunders et al (2012:165) concur with the view of Tashakkori and Teddlie (2010) that the use of plural methods within a single paradigm constitutes “multimethod research” rather than mixed methods. However, Morse’s (2010) definition of mixed qualitative research as one main method which could stand by itself with one or more complementary methods which are not sufficiently substantial to stand alone in the study is more applicable to this research design in which one of the primary data sources is supported by the other and by secondary sources

Finally, in this section we would like to highlight the issue of research as an iterative and messy process in which “muddling through” seems to be the reality. In our previous experience as well as in this study, there are stops and starts, blind alleys and re‐routes. Our aim in the research design is to elicit the necessary data from appropriate parties and relevant written sources in order to arrive at an answer to the question of the value of overseas partner visits. We have made strenuous efforts to tailor method to respondent group and to make changes where necessary (Pettigrew, 1990:222; Saunders et al, 2012).

5. Research methods The research methods employed in this study are generally in line with those in similar previous research in the field of students’ international experience (e.g. Brown, 2009; Crossman and Clarke, 2010; Shore and Groen, 2009). The qualitative stance is particularly concerned to give participants a voice in order to discover whether and to what extent home staff visits to overseas partner institutions have a value. For this reason we feel that a large quantitative survey would not be appropriate.

The total population concerned consists of many thousands of students and staff each year. According to the British Council (2012) UK students undertook 13,668 periods of study or work placement in the academic year 2011‐12. Student‐facing academic and administrative staff from our own institution, overseas partner institutions and another UK institution who have carried out partner visits are interviewed using a semi‐ structured schedule of largely open questions requesting accounts of their experience on visits and their views on any positive and negative organizational and personal outcomes accruing from these. Staff participants are in a peer relationship with the researchers, insofar as this is possible between researcher and researchee, therefore a semi‐structured interview is appropriate to tease out issues. The response rate among the staff has so far been 100%.

It was decided not to interview students face‐to‐face for two reasons: firstly, students are in a dependent relationship with university staff and therefore a “distanced” technique seems to be more appropriate.

Secondly, the research requires data from them concerning only their own benefits and/or disbenefits from staff visits (as a single stake‐holder group), therefore less complexity is anticipated.

Students were initially approached in a way that appeared unintrusive. We placed a posting on our university’s closed, 217‐member Facebook group site for Exchange Students (incoming as well as outgoing) asking students about their experiences and opinions regarding home staff visits. The students were told in advance that responses were required as part of our research project, and they were assured of anonymity. The respondent was asked for his permission to use his account in the study. However, we recognise that this method of

1

UK universities do not offer “class credit” for participation in research studies.

165

Deborah Knowles and Karen Kufuor posting on a quasi‐public forum names the respondent and posts their response for their peers to read. This raises ethical concerns which we hoped to address. Firstly, respondents would not post any thoughts that they did not want read by fellow members of the site or made public by us as researchers (albeit anonymously) and secondly, there is no suggestion of a quid‐pro‐quo relationship between the researcher/lecturer and the student. A second internet based opportunity for gathering the student perspective on the visits has taken the form of a discussion thread question, posted at our request on the public www.thirdyearabroad.com website.

These approaches are quite different from that of a 2008, US study entitled “Tastes, Ties and Time” in which broad‐based profile data from the Facebook accounts of 1,700 students were used without their knowledge or permission. Although those researchers acted in good faith, claiming that the information was already in the public domain and that furthermore they had taken pains to anonymise it, it did not take long for the university to be identified and for a number of ethical concerns to arise (Zimmer, 2010).

Unfortunately, each of these internet postings has received only one response so far. These are both very useful but we would like to receive a greater range of student response if possible. We have now re‐packaged the research instrument for students in an attempt to allay any fears that they may have concerning anonymity, or shyness before their peers. An anonymised return “qualitative questionnaire” has been posted on the closed Facebook site with a deadline of 24 June 2013. One student has so far responded and a reminder has been posted alongside an acknowledgement to that person. We use the term “qualitative questionnaire” to denote one in which the only closed questions are the “demographics” which ask the respondent to categorise themselves according to, for example, gender, whether they have yet undertaken their Year Abroad and whether they have received a visit from home institution staff. All other questioning on experiences and opinions is open with plenty of space for full, “qualitative” responses. Questions are all optional.

6. Interim findings The study is still on‐going but findings are available from both populations. These brief interim primary research findings will focus on effects on current Year Abroad students and on the home institution.

As has been noted there has been a disappointingly low response rate from the students. However, the responses received from two students currently on their Year Abroad, are detailed and useful, and also contradictory, providing powerful signals to student concerns. Student A, an anonymous student who had not received a visit, felt that visits from home‐institution staff were unnecessary as: “[...] with the use of the internet, it is easy to get in contact with someone from the university [...] if you have any problems.”

Student B, a male Business School student, who had received a visit, was more positive about the experience, however: “I think it is very important to have a visit from the home institution because it provides a sense of normalcy and comfort as well as being able to discuss any issues students have with the foreign institution.”

Student A, could see only disbenefits from such a visit: “The year abroad is a great way to improve upon your own independence and I think that it could be undermined with home universities arranging visits.”

Student B was “enjoying most of [his] time [there]” but was nonetheless “very happy to see a familiar face”.

He saw possible benefits to other students: “[...] for any student who is alone a visit would be very beneficial for morale if the individual is not having a good experience.”

Three semi‐structured staff interviews have been carried out. These were face‐to‐face with an administrator (Admin 1) and an academic (Academic 1) at our School and by phone with an academic (Academic 2) at one of our overseas partner institutions. The interviews were recorded both in writing and digitally. They lasted between thirty‐three and sixty‐one minutes depending on how much detail the participant wanted to include.

Following from the above findings on benefits to students, respondents mentioned both learning about the partner institution and assessing whether the environment and lifestyle would suit a particular group of students in future and resolving current students’ conflicts with staff there.

166

Deborah Knowles and Karen Kufuor Academic 2 described the visits as a “support mechanism”. She spoke of “personal contact” in interaction with both the home students and staff of the host institution which is not possible in email correspondence.

Academic 1 concurred that a visit would “finalise” an issue that would otherwise require long term email correspondence. These views about the inadequacy of email in reaching an understanding are counter to the assertion of Student A that the internet makes contact easy. Academic 2 is able to resolve conflicts which may escalate due to a student’s embarrassment about their lack of language fluency. At one incredulous UK university she has verified students’ home grades. Admin 1 (who has carried out only one visit) felt that it was important for the students to “connect with someone from home” and use her in an informal setting as a sounding board because she was not from the host institution. One student had expressed a fear that her grades would go down there if she complained.

In terms of effects on the home institution, Admin 1 was very enthusiastic about the amount she had learnt about the partner institution and its surrounding area. Reflecting on her role as an adviser to students who are planning their Year Abroad, she said, “I feel I’m more valuable to the students”. However, she did find disadvantages in knowing too much as she felt that the town is “difficult to sell, it’s not a party town”. She would like to investigate the possibility of developing an institutional partnership with a university in a nearby famous touristic city. Academic 1 talked of establishing a rapport with host‐institution staff, mentioning again that this was “more than an email link”. She said, “I think you do more for people you can put a face to”. At the time of her visits, there was a requirement to visit all institutions at least once per year in the second semester. She felt that visiting so often was unnecessary unless a problem had been identified and also, if there was a problem the second semester was too late to begin addressing it. Like Admin 1 she had experienced negative learning. At one institution, with which the partnership has since been terminated, staff had been friendly but the visit was not satisfying, she was never shown around. Academic 2 said that she returned home with a “grounded idea on what’s working and what isn’t”. She needs to get to know what type of institution is best for her home‐institution students and does that by visiting, to “get a feel”.

7. Conclusions From the brief interim findings for which this short paper has space, it is clear that staff who have visited overseas partner institutions have been satisfied as to the benefits to (so far) home students and the home institution. There is a strong preference for face‐to‐face contact with partner staff and with home students on their Year Abroad. The learning offered by the Year Abroad to students – challenge to perceptions and the development of problem‐solving skills (Johnston et al, 2005) ‐ is seen to sometimes require assistance, or at least reassurance, from home‐institution staff. The home institution benefits from stronger relationships formed with the partner institutions. Disadvantages are fewer. According to the staff they mainly relate either to negative learning about an institution or its environment (which might actually be seen as a benefit of the visit) and to the timing of visits. The findings from the students additionally indicate a tension between supporting students and allowing them their independence.

Assessing the value of home staff visits to overseas partner institutions is seen as a complex issue, which impacts on the experience of students on their Year Abroad. The qualitative methodology has been carefully chosen to reflect the ontological and epistemological stance of the study. Due to the researchers’ involvement in the situation it is considered particularly important to describe the methods used and the findings generated as transparently as possible in order to uphold the reliability of the research and the validity of the conclusions.

Mixed qualitative methods are used to elicit data from the different groups of respondents and from public and internal, university documents. The findings that have so far been collected from the primary methods have been rich and significant. It is clear that the topic is worthy of study.

References British Council (2012) “Record Number of UK Students Study or Work Abroad in Europe with Erasmus”, available at URL http://www.britishcouncil.org/press/record‐number‐uk‐students‐study‐work‐abroad‐erasmus, accessed 3.1.2013.

Brown, L. (2009) “The Transformative Power of the International Sojourn: An Ethnographic Study of the International Student Experience”, Annals of Tourism Research, Vol. 36, No. 3, pp. 502‐521.

Bryman, A. (2008) “Of Methods and Methodology”, Qualitative Research in Organizations and Management, Vol. 3, No. 2, pp. 159‐168.

167

Deborah Knowles and Karen Kufuor Buchanan, D. and Bryman A. (2007) “Contextualizing Methods Choice in Organizational Research”, Organizational Research Methods, Vol. 10, No. 3, pp. 483‐501.

Canning, J. (2004) “Provision of Three‐Year Degrees in Languages: An Overview”, Centre for Languages, Linguistics and Area Studies, available at URL: https://www.llas.ac.uk/resources/paper/2170#toc_1, accessed 24.5.2013.

Crossman, J.E. and Clarke, M. (2010) “International Experience and Graduate Employability: Stakeholder Perceptions on the Connection”, Higher Education, Vol. 59, No. 5, pp. 599‐613. th Easterby‐Smith, M., Thorpe, R. and Jackson, P. (2012) Management Research (4 Edition) London, Sage.

Fane, L. (2012) “Interview with Lizzie Fane, Founder of ThirdYearAbroad.com”, available at http://www.thirdyearabroad.com/blog/item/1150‐lizzie‐fane.html, dated 13.03.2012, accessed 03.01.2013.

Ghasempoor, A., Liaghatdar, M.J., Jafari, E. (2011) “The Internationalization of the Curriculum”, Higher Education Studies, Vol. 1, No. 2, 35‐40.

Guba, E. and Lincoln, Y. (1998) “Competing Paradigms in Qualitative Research”, In Denzin, N. And Lincoln, Y. (eds.) (1998) The Landscape of Qualitative Research, London, Sage, pp. 195‐220.

Hackney, K., Boggs, D. and Borozan, A. (2012) “An Empirical Study of Student Willingness to Study Abroad”, Journal of Teaching in International Business, Vol. 23, No. 2, pp. 123‐144.

Hesse‐Biber, S. (2010) “Qualitative Approaches to Mixed Methods Practice”, Qualitative Inquiry, Vol 16, No. 6, pp. 455‐468.

Jackson, D. and Chapman E. (2012) “Non‐Technical Competencies in Undergraduate Business Degree Programs: Australian and UK Perspectives”, Studies in Higher Education, Vol. 37, No. 5, pp. 541‐567.

Johnston, B., Myles, F., Mitchell, R. and Ford P. (2005) “The Year Abroad: A Critical Moment”, University of Southampton, available at http://www.critical.soton.ac.uk/LLAS%20paper.pdf, accessed 03.01.2013.

Lincoln, Y. and Guba, E.G. (1985) Naturalistic Enquiry, Beverly Hills, CA, Sage, cited by Saunders et al (2012), p. 194.

Morse, J.M. (2010) “Simultaneous and Sequential Qualitative Mixed Method Designs”, Qualitative Inquiry, Vol 16, No. 6, pp. 483‐491.

Oleson, V. (1998) “Feminisms and Models of Qualitative Research”, In Denzin, N. and Lincoln, Y. (eds.) (1998) The Landscape of Qualitative Research, London, Sage, pp. 300‐332.

Pettigrew, A M. (1990) “Contextualist Research: A Natural Way to Link Theory and Practice”, In Lawler, E.E. et al (Eds.) Doing Research that is Useful for Theory and Practice, London, Jossey‐Bass, pp. 222‐274.)

Pritchard, K. (2012) “Combining Qualitative Methods”, In Symon, G. and Cassell, C. (2012) Qualitative Organizational Research, London, Sage, pp. 132‐148.

QAA (2012a) The UK Quality Code for Education: A Brief Guide, available at URL: http://www.qaa.ac.uk/Publications/InformationAndGuidance/Documents/quality‐code‐brief‐guide.pdf, accessed 26.5.2013.

QAA (2012b) “Managing Higher Education Provision with Others”, Chapter B10 In QAA (2012) The UK Quality Code for Higher Education, Gloucester, The Quality Assurance Agency for Higher Education, available at URL: http://www.qaa.ac.uk/Publications/InformationAndGuidance/Documents/Quality‐Code‐Chapter‐B10.pdf, accessed 26.5.2013.

Saunders, M., Lewis, P. and Thornhill, A. (2012) Research Methods for Business Students (6th Edition), Harlow, Pearson.

Shore, S. and Groen, J. (2009) “After the Ink Dries: Doing Collaborative International Work in Higher Education”, Studies in Higher Education, Vol. 34, No. 5, pp. 533‐546. nd Tashakkori, A. And Teddlie, C. (Eds.) (2010) The Sage Handbook of Mixed Methods in Social and Behavioural Research (2 Edition), Thousand Oaks, Sage, cited by Saunders et al (2012), p. 165.

Toncar, M.F., Reid, J.S. and Anderson, C.E. (2006) “Perceptions and Preferences of Study Abroad: Do Business Students have Different Needs?”, Journal of Teaching in International Business, Vol. 17, Nos. 1‐2, pp. 61‐80 Yanow, D., Ybama, S. and van Hulst, M. (2012) “Practising Organizational Ethnography”, In Symon, G. and Cassell, C. (2012) Qualitative Organizational Research, London, Sage, pp. 331‐350.

Zimmer, M. (2010) “ ‘But the data is already public’: on the ethics of research in Facebook”, Ethics and Information Technology, Vol. 12, pp. 313‐325.

168

Understanding the Australian Indigenous Entrepreneurs Through Narratives Branka Krivokapic‐Skoko1, Tracey Trudgett1, Sonya Pearce2, Mark Morrison1, Jock Collins2 and P.K. Basu1 1 Faculty of Business, Charles Sturt University, Bathurst, Australia 2 University of Technology Sydney, Australia bkrivokapic@csu.edu.au ttrudgett@csu.edu.au Sonya.Pearce@uts.edu.au Jock.Collins@uts.edu.au pkbasu@csu.edu.au

Abstract: This paper focuses on the categorical content of the narratives of a sample of 35 Australian Indigenous entrepreneurs. The narrative case studies revealed patterns and coherence across Indigenous entrepreneurs talking about success factors and making sense of their business experience. The paper also outlines principles of culturally appropriate research methods for doing research in Indigenous communities and protocols which enable researchers to confidently engage with Indigenous Australians. Protocols such as the use of appropriate communication methods, the development of mutual trust and the need for reciprocity provide researchers with basic guidelines to follow regarding consultation and negotiation with Indigenous Australians. Six sets of values which underpin core principles to research with Aboriginal and Torres Strait Islander people in Australia are reciprocity, respect, responsibility, sprit and integrity, and survival and protection. One significant observation identified by the researchers was the strong sense of community amongst the Indigenous entrepreneurs. Many entrepreneurs were part of significant formal and informal networks and were often volunteers or members of community groups and organisations. Mutual sharing was also noteworthy, particularly with the local Indigenous community. Many Indigenous entrepreneurs provided financial assistance, employment opportunities and sought to aid the Indigenous community in whatever way they could via their business. A further issue when considering the factors influencing Indigenous business success is that success is often evaluated differently from the perspective of the Indigenous entrepreneur. For many Indigenous people and communities, profit is not seen as the definitive measure of business success, and other measures such as securing employment of community members have greater importance.

While carrying out a thematic analysis of the data it became clear that the narratives were central to how Indigenous entrepreneurs recall, make sense of their experiencing in start up and running the businesses. The accounts gathered from the narrative case studies also revealed a tension between self‐identity and the constraints the identity place on business operations. Furthermore, the perception of the success was largely determined by self‐identify and particular context of Indigenous culture and community.

Keywords: Indigenous entrepreneurs; narratives; business success

1. Introduction Issues relating to power and control have historically caused problems between Indigenous peoples around the world and researchers across the board (West, Stewart, Foster & Usher, 2012). In Australia, historically the mainstream view has shaped society’s knowledge and understanding of the Indigenous cultures shaping legislation and policy further marginalising Indigenous people (Lavallee & Poole, 2010; Rigney, 1999). Historical power imbalance between researchers and Indigenous people called for deep transformations regarding methodological approaches to doing research in the Indigenous context (West et al., 2012). Tuhiwai Smith (1999) observed that when Indigenous peoples become researchers and not merely the researched, the activity of research is transformed in several ways (Smith, 1999). This can include questions being framed differently, priorities ranked differently, problems can be defined differently and people participate on different terms. This has lead to the development of Indigenous Standpoint Theory (Nakata, 1998) which encourages Indigenous researchers to develop unique standpoint theories, so previous research on Indigenous people can be reviewed and challenged.

In research in the area of Indigenous entrepreneurship, a key proponent of Indigenous standpoint theory Dennis Foley (2003, 2008a, 2008b) has highlighted the importance of Indigenous entrepreneurship to achieving self‐determination though economic independence from the position of the Indigenous person as a researcher and subject. In should be noted here that research on Indigenous entrepreneurship in Australia and other countries (Dana and Anderson 2007) tends to be constructed through a ‘deficit’ model. With literature

169

Branka Krivokapic‐Skoko et al. on Indigenous enterprises in Australia suggests that Indigenous businesses face a wide range of barriers and problems in establishing and developing business enterprises. The barriers faced by potential Indigenous entrepreneurs include a lack of basic business skills and financial literacy (Collins, 2004); a limited pool of skilled labour, poor mentoring and inadequate business advisory support in the establishment phase (Altman, 2001; Collins, 2004) and a lack of access to finance and education (Collins, 2004). This exclusion is reproduced in systems and policies imposed on Indigenous people and is manifested in many forms, both overt and covert, all downward leveling norms or “push factors” (Wood and Davidson, 2011) operate to reduce and limit opportunity, forcing the more ambitious to engage in forms of entrepreneurship as a means of both escape and survival. They also connect to very real political issues about the consequences of how we understand Indigenous economic development and policy formation, particularly as it relates to female Indigenous entrepreneurs (Pearce, 2013).

Hence the literature on Indigenous entrepreneurs focuses mainly on the barriers (including cultural barriers) that are faced, and tends to ignore the strengths and positive cultural and other attributes that Indigenous entrepreneurs bring to the enterprise. These have been identified by some scholars as a colonial impact, resulting in the underdevelopment of Indigenous social and human capital (Foley, 2008a, 2008b). Foley’s work suggests that Indigenous businesses face both racial and gender discrimination from mainstream society as well as discrimination from Indigenous communities, emerging from a lack of Indigenous understanding of entrepreneurs. Indigenous standpoint theory thus represents a foundation for formulating, theorising, discourse development and analysis and developing methodological approaches in research which give voice and credence to Indigenous knowledge and knowledge making – Indigenous sense‐making. By adopting this approach with this research project on success factors of Indigenous enterprises, we can develop a deeper understanding of Indigenous issues within the business sector which will hopefully lead to better outcomes for Indigenous entrepreneurs in the future.

This paper firstly briefly provides some insights into conceptual and empirical literature on the factors influencing the success of Indigenous entrepreneurship in Australia. The paper then reflects on formal and informal protocols which exist to enable researchers to confidently engage with Indigenous Australians. The paper then further specifies conceptual model of the factors influencing business success of Indigenous enterprises in Australia, the research design and principles of carrying culturally appropriate research methods, and finishes by outlining some narrative accounts gathered in the field work done during 2011 and 2012. In particular the use of narrative as a discourse and its application to Indigenous Standpoint Theory in bringing alive the voices of the Australian Indigenous entrepreneurs – from theory to method to understanding the role of narratives to in Indigenous research practice.

2. Factors influencing business success Indigenous entrepreneurship in Australia: Literature review According to the literature business success of the Indigenous entrepreneurship in Australia is seen to be directly influenced by the existence of various forms of capital. This includes human capital (Altman, 2001, Collins, 2004, Baguley, 2007), social capital (Flamsteed and Golding, 2005), financial capital (Collins 2004, Foley 2006), physical and technological capital (Altman, 2002, Flamsteed and Golding, 2005) as well as institutional capital which relates to governance structures. As well as directly influencing business success, these forms of capital also have effects on each other. For example, recent work by Foley (2008) suggests that social capital leads to the development of human capital, and vice versa. Social capital may also influence governance arrangements because of social learning about more effective structures from contact with a larger array of information sources. Ability to access financial capital would also be expected to be a function of human capital, as human capital is likely to influence entrepreneurs ability to complete administrative processes required to secure loans.

In addition to these forms of capital, it is accepted that business success will be directly and indirectly influenced by cultural capital. Cultural capital incorporates entrepreneurs worldview, and includes such facets as attitudes towards mainstream Australia, respect for diversity (including the success of others and attitudes towards commercial activity), desire for change and concern for future generations. Previous research by Flamsteed and Golding (2005), Foley (2006) and Baguley (2007) has demonstrated the influence of cultural capital on business development. In the conceptual model, cultural capital is also expected to influence institutional capital because governance structures will be determined in part by what is acceptable to

170

Branka Krivokapic‐Skoko et al.

Indigenous communities. Cultural capital also influences social and human capital as it affects the acceptability of developing business skills and relating to mainstream Australia (Foley, 2006).

3. Narrating Indigenous entrepreneurship in Australia Aboriginal people have a long and unbroken history of storytelling, which is reflected today in many forms including song, dance and art (Brady 1998, Pearce and Cox 2009). In short, storytelling has brought together families and communities in understanding their collective narrative histories (Atkinson 2002). In the Dreamtime, as in modernity, storytelling remains a central feature of Indigenous cultural transmission—we now use it to tell stories of our enterprises and new endeavours (Pearce, 2013).

This research paper aims at increasing our understanding of the challenges facing Australian Indigenous enterprises in a range of geographical, economic and social settings through a largely qualitative research process which includes narratives. The research focused on identifying a broad range of factors which influence the success of Indigenous enterprises, such as culture, governance arrangements, human capital, access to financial capital, and availability of physical and technological capital and personal characteristics. In carrying out this project we were also interested in exploring the extent to which, and the way in which, the activities of Indigenous enterprises are linked to Indigenous culture by providing goods and services that are unique to Indigenous culture – such as Indigenous art, tourism, food and artefacts – or by providing general goods and services to Indigenous communities themselves. We were are also interested in exploring how Indigenous social capital (including family, tribal and broader social networks as well as leadership and mentoring capacity) provide a basis to sustain (and to sometime constrain) the formation of, and survival of, Indigenous enterprises. A further issue when considering the factors influencing Indigenous business success is that success is often evaluated differently from the perspective of the Indigenous entrepreneur. For many Indigenous people and communities, profit is not seen as the definitive measure of business success, and other measures such as securing employment of community members have greater importance. For many Indigenous entrepreneurs there is also the challenge of balancing cultural integrity with the concept of commercialisation.

This is where narratives play an important role as they provide the space between theory and method, as narratives in organisational cultural studies offer the possibility for providing some invigoratingly fresh insights into many aspects of popular culture, including those conventionally regarded as the ‘other’, such as the stories of Aboriginal people (Pearce and Cox 2009). These stories of are important in many ways, as they not only help us to work through future endeavours they also help us to understand our history and culture (Robertson, Demosthenous and Demosthenous 2005 in Pearce, 2013).

Narrative as a form of qualitative research methodology is central to this research project, particularly in the form of interviews, narratives and auto ethnographies as a way of documenting the experiences of the respondents in a manner sensitive to the complexities of the relationships between researcher and informant and aware of the complexities of trust, interactions and obligations that characterise research into Indigenous entrepreneurs involved in private and social enterprises.

Understanding and comprehending the differences between cultures is crucial element for doing research in the Aboriginal and Torres Strait Islander context (NHMRC, 2003).Another crucial element in Aboriginal and Torres Strait Islander research is the explicit recognition and commitment to respect for Aboriginal and Torres Strait Islander cultural values and principles (NHMRC, 2003). Respect is a major element in the forming of trust between researchers and the Aboriginal and Torres Strait Islander community and trustworthy relationships with local community members should be created and maintained wherever possible. At every stage, research with and about Aboriginal and Torres Strait Islander peoples must involve a process of meaningful engagement and reciprocity (AIATSIS, 2011).

There is a number of formal and informal protocols which exist to enable researchers to confidently engage with Indigenous Australians. Protocols such as the use of appropriate communication methods, the development of mutual trust and the need for reciprocity provide researchers with basic guidelines to follow regarding consultation and negotiation with Indigenous Australians. These protocols are outlined in the AIATSIS Guidelines for Ethical Research in Indigenous Studies (2011) and the NHMRC’s Values and Ethics: Guidelines for Ethical Conduct in Aboriginal and Torres Strait Islander Health Research (2003).

171

Branka Krivokapic‐Skoko et al.

The following six sets of values underpin core principles to research with Aboriginal and Torres Strait Islander people in Australia (Table 1).

Table 1: Core principles to doing research with Aboriginal and Torres Strait Islander people in Australia Reciprocity Recognition and “ensuring that research outcomes include equitable benefits of value to Aboriginal and Torres Strait Islander communities or individuals” (NHMRC, 2003).

Respect At the beginning of any research project, researchers and Aboriginal and Torres Strait Islander people should agree on all aspects of the research project to ensure a respectful relationship is established, maintained and continued.

Responsibility These responsibilities include those to country, kinship bonds, caring for others and the maintenance of harmony and balance within and between the physical and spiritual realms (NHMRC, 2003). Research is considered ethical when the differing responsibilities are established and agreed upon (including those of the researchers), participants are protected, trust is maintained and accountability is clear (NHMRC, 2003).

Equality When researching Aboriginal and Torres Strait Islander communities, equality can be demonstrated by considering whether research agreements have the strength to sustain equality and whether participating Aboriginal and Torres Strait Islander communities have understood and expressed satisfaction with proposed research, its potential benefits and the distribution of those benefits (NHMRC, 2003).

Spirit and Integrity Spirit and integrity is an encompassing value that brings together all others to form a consistent cohesion. It consists of two components, the first being the continuity between past, present and future generations. “The second is about behaviour, which maintains the coherence of Aboriginal and Torres Strait Islander values and cultures” (NHMRC, 2003).

Survival and Protection “Research with Aboriginal and Torres Strait Islander peoples has had a chequered past, and many Aboriginal and Torres Strait Islander communities do not trust researchers for fear of manipulation and unfair treatment.

Researchers will need to demonstrate their trustworthiness through “ethical negotiation, conduct and dissemination of research” (NHMRC, 2003).

We undertook 35 in‐depth interviews of Indigenous entrepreneurs in urban, regional and remote areas of Victoria, New South Wales, Queensland, Western Australia and the Northern Territory. Theoretical sampling was done based on gender, industry and age of the entrepreneurs as well as the duration of being involved in the business. We sources interviewees from the Black Pages, Koori Mail, Aboriginal Business Directory and other national Indigenous business portal and particularly through personal contacts. Interviews were conducted by two members of the research team and generally lasted between 2.5 and 3 hours.

We used interviews as a main data – gathering approach. Narratives produced in interview are seen by social researchers (De Fina, 2009 for instance) as worthwhile and valid source of data and our primary attention was placed on what is said in terms how each participant’s business evolved and what was perceived as critical factors contributing to the success. Following Riessman (2008) our approach to narrative analysis was the thematic analysis (as being the most common approach) which focused exclusive on content. We were focusing on the categorical content of the narratives and were looking for patterns and coherence across Indigenous entrepreneurs talking about success factors and making sense of their business experience. And it was expected the themes were influenced by both prior theories, the purpose of the research project and the data themselves.

Regarding data analysis we strictly followed four distinct levels of analysis as advocated by Cope (2005, pp.

178‐179) . The Level 1 analysis included ‘a personal sense‐making process and getting to know each of Indigenous entrepreneurs included in the qualitative sampling. During the Level 2 analysis a case study narrative was organised for each participant, as a story of how these Indigenous entrepreneurs started their businesses, particularly focusing on the success factors. These case study narratives were both chronological and thematic. During the Level 3 analysis we identified commonalities and specifics across the case narratives, and we were looking for quotations, patterns, themes and specific illustrating some aspects of success factors, access to resources, etc. It then led in Level 4 analysis towards further clustering of the empirical evidence supporting some of issues around measures and conceptualisation of success and constraints to success for Indigenous business owners.

172

Branka Krivokapic‐Skoko et al.

4. Overview of some preliminary findings Firstly, it became clear that Indigenous business owners measure success in a number of ways. Traditional measures of business success including financial profit, growth of the business through sales and revenue, customer satisfaction, longevity and generation of increasing employment emerged as important indicators of success for Indigenous business owners. While discussing entrepreneurialism and experience in business, one Indigenous business owner commented: You have to have a product and you have to make a profit from it. If you don’t have those two elements, you’re not actually a business.

(Female Indigenous entrepreneur, 30‐39 yrs old, Service industry) This illustrates the owner’s belief in traditional measures of success for business and the need to make a financial profit in order to be considered successful in what you do.

In regards to growth of the business, another Indigenous business owner commented: I work for three different contractors now, so I’ve got three different incomes coming into my business. When I first started...I only had the one (Male Indigenous entrepreneur, 47 yrs old, Building and Construction industry) Another important measure of business success for Indigenous business owners is how it relates to themselves and how it makes them ‘feel’. Lifestyle measures, knowing that a job has been well done and recognition and reputation ‐ to be valued by the community are all key considerations when measuring the success of a business to an Indigenous business owner.

In terms of lifestyle, one Indigenous business owner commented: My business is successful if I’m able to go out in the middle of the day and take my dogs to the beach (Female Indigenous entrepreneur, 30‐39 yrs old, Service industry) Another example of how business success relates personally to an Indigenous business owner includes knowing that a job has been well done. This is reflected in the statement below: I measure my success by how I feel at the end of it. If I walk out and feel “Oh my God, I’ve done something. I’ve made a difference here”, then I think I’ve done a good job (Female Indigenous entrepreneur, 40‐49 yrs old, Service industry) Recognition and reputation were also found to be key measures of business success for Indigenous business owners..

Examples of this include: The key to success is if you’re getting known and your name is recognised (Female Indigenous entrepreneur, 40‐49 yrs old, Retail industry) We’re successful in the fact that we’ve got International recognition. We are successful in the fact that we have won some awards (Male Indigenous entrepreneur, 60 plus yrs old, Tourism industry) These examples relate to recognition on a number of levels ranging from the local Indigenous community right through to the international level. In addition, reputation and cultural obligations are important within Indigenous communities and can have a positive or negative effect on the success of the business (Foley, 2003).

Success measures related to community were also found to be significant for Indigenous business owners.

Community in this sense refers to the local community where the business owner lives and includes both the mainstream Australian and Indigenous community. Success measures relevant to community include providing employment to Indigenous people, effects on people and social changes, providing evidence to community members that Indigenous people can successfully run a business and to retain cultural identity.

173

Branka Krivokapic‐Skoko et al.

Providing employment to Indigenous people was a significant indicator of business success for many business owners. A number of Indigenous business owners have Indigenous employment policies in place to ensure Indigenous employment remains a focus of the business. One business owner commented: I have a policy to employ, 75% of my employees are Indigenous. But at the moment, I’m smashing that, I’m at about 90%.

(Male Indigenous entrepreneur, 47 yrs old, Building and Construction industry) Another way Indigenous business owners conceptualise success at a community level involves providing evidence or proving to the wider community (including both the mainstream and Indigenous communities) that Indigenous people can successfully run a business. This measure is quite unique, as it signifies the Indigenous business owners desire to prove themselves as worthy business people. In addition to this, a number of Indigenous business owners mentioned that their desire for success was also related to being a positive role model for the younger Indigenous people within their community. They wanted to show the younger generation that Indigenous people can run a successful business and instil confidence in them to perhaps one day do the same.

Well, a lot of the younger ones, they come in and see what, how we’ve been going and that in business. They can see it’s a pretty well established business...they can see that we’re Aboriginal and we can do it (Male Indigenous entrepreneur, 63 yrs old, service industry) Another unique measurement of business success at a community level relates to retaining cultural identity for Indigenous business owners. This involves being able to practice and express meaningful aspects of Australian Indigenous culture, which has become significantly eroded since colonisation (Foley, 2006). One Indigenous business owner expressed: We’re successful in the fact that our kids are seeing old traditions continued in a contemporary way and we are actually gaining some financial benefits from that. So that’s a big success (Male Indigenous entrepreneur, 60 plus yrs old, Tourism industry) Regarding the constraints to business success Indigenous entrepreneurs singles out the need for training in multiple areas for Indigenous business owners, staff related challenges and cultural challenges such as separating the business from family and the acceptance of Aboriginal culture in the mainstream Australian community. Participants also mentioned marketing (including promotion and sales), financial management and employee training as the most necessary. One Indigenous business owner commented: I suppose my problem was accounting, first of all. Learning all of that type of thing...just knowledge of what is involved in running a business (Male Indigenous entrepreneur, 30‐39 yrs old, Information Technology industry) This lack of knowledge in the financial management and administration of the business was noted amongst several Indigenous business owners and some also commented on their inability to access assistance in this area: I’m not 100% happy with the business support I’m getting, or the lack of business support...I don’t know where to go, or who to ask (Male Indigenous entrepreneur, 30‐39 yrs old, retail industry) Another key constraint to business success to emerge from our qualitative findings included staff related challenges. This challenge incorporated the Indigenous business owner’s inability to find and retain skilled staff, issues surrounding transient staff as well as high staff turnover for some business owners. These issues however effect business owners worldwide and therefore are not unique to Australian Indigenous business owners.

One aspect of the constraints to business success that may be deemed unique to Australian Indigenous business owners is the cultural challenges they face in operating their businesses. These challenges include their struggle to separate business from family. This involves family members wanting or demanding benefits from the business which are largely financially related. Indigenous business owners commented that their cultural obligations compel them to ‘give back’ to their community, in particular their family regardless of their

174

Branka Krivokapic‐Skoko et al.

businesses financial standing. This can often cause conflict between the Indigenous business owner and their community or family and contribute additional strain on the business.

Acceptance of Australian Indigenous culture within the mainstream community is also a challenge for some Indigenous business owners. Some (not all) have experienced prejudice behaviour towards themselves or their business due to their ethnicity. This can be seen in the below comment by one Indigenous business owner: I can’t get it into shops, normal shops, because as soon as they see Indigenous, they’re just not interested. That’s been the biggest let down (Male Indigenous entrepreneur, 30‐39 yrs old, retail industry) Other barriers to success for Indigenous business owners included difficulties with running the business from home, business location issues, fierce competition and problems with partners in a partnership business. One Indigenous business owner commented: She [the business partner] sort of took over, without putting the money in that she promised. She only had like fifteen percent of the business but was taking over (Male Indigenous entrepreneur, 30‐39 yrs old, retail industry) Again, this challenge is not unique to Australian Indigenous business owners, but is important to note to ensure sufficient training and education programs can be developed to improve the situation for Indigenous business owners.

However, the research team also found that some of the responses regarding the entrepreneurship and strength of Indigenous identity became expressed in what may be described a story telling or ‘yarn’ like the following one of a young female entrepreneur who is designing and printing the textile making a range of prints based around her Indigenous identity. In her story she was telling in a very amazing way and remarkable voice how she creates a range of prints based around her identity and her family and where she comes from and who she is, but yet enfolded as a contemporary expression of Aboriginal identity. “ ..for me, the essence is on the story, you know, and being able to share those stories and loving to tell the stories and remembering family. And also, the lovely thing about it is that I’m able to continue learning and talking to people here, and hearing new stories. People come to me, I’ve got a few things to tell you, you know? Um, that’s a really beautiful component of it too, and you know, like I said, I’m outside of country, but I’ve got that constant connection to it in these prints and in these fabrics and in these stories. Which is really lovely, and learning the new things. One of my prints, / Buraworrimbool, which is my latest one, my sister told me a story about the Milky Way and she said, for us Gamillaroy people, the stars actually represent campfires. And sitting around those campfires are our Elders and our loved ones and our families and ancestors that have passed on, and they’re cooking up river mussels on the fire, big feasts. So the smoke and the haze of the cooking mussels is the haze of the Milky Way. So the reason why they’re cooking that feast is so that when we join them, we can celebrate and share in it with them. And that’s something that I’ve learnt in the last couple of years, that goes back to the beginning of time for us, you know, that we’ve always known. And know it’s something that I can share with my daughter, I can tell the other Gamillaroy people in my family, and in that way, being able to immerse myself in culture, old and new, you know? New and old traditions too, are something that keeps my Spirit alive. Yeah”.

“And so with my textiles, it’s about creating a visual connection to those people and those places and those stories, keeping them alive. .... So, making stories or retelling these stories through image, and the stories that I tell are old. You know, they’re creation stories, but they’re also family stories, you know, about going out and picking napan’s or / with Nan and Aunties, or fishing on the river. It’s a real variety of things, so it’s in my way of thinking, a really good expression of Aboriginal culture, contemporary or whatever, you know, just different ways of telling stories. Old and new both as important as each other”.

5. Reflections on doing research in Indigenous context The literature on Indigenous enterprises in Australia suggests that Indigenous businesses face a wide range of barriers and problems in establishing and developing business enterprises. Scholars, like Foley (2006) have

175

Branka Krivokapic‐Skoko et al. suggested that Indigenous businesses face both racial and gender discrimination from mainstream society (as well as discrimination from Indigenous communities, emerging from a lack of Indigenous understanding of entrepreneurs) . These have been identified by some scholars as a colonial impact, resulting in the underdevelopment of Indigenous social and human capital (Foley, 2010). Low levels of human and social capital have lead to other social and economic problems, including an undermining of the incentive for individuals to engage in entrepreneurial activity, welfare dependence and substance abuse (Baguley, 2007).

Others problems attendant with low levels of social capital include low self‐esteem, lack of confidence, as well as limited Indigenous role models and/or Indigenous business networks, ensuring that the Indigenous nascent entrepreneur is forced to operate their enterprise within the business structure of the dominant culture, that is settler society (Flamsteed and Golding, 2005; Foley, 2010).

In carrying out this project we were interested in exploring the extent to which, and the way in which, the activities of Indigenous enterprises are linked to Indigenous culture by providing goods and services that are unique to Indigenous culture – such as Indigenous art, tourism, food and artefacts – or by providing general goods and services to Indigenous communities themselves. We were are also interested in exploring how Indigenous social capital (including family, tribal and broader social networks as well as leadership and mentoring capacity) provide a basis to sustain (and to sometime constrain) the formation of, and survival of, Indigenous enterprises. A further issue when considering the factors influencing Indigenous business success is that success is often evaluated differently from the perspective of the Indigenous entrepreneur. For many Indigenous people and communities, profit is not seen as the definitive measure of business success, and other measures such as securing employment of community members have greater importance.

One significant observation identified by the researchers was the strong sense of community amongst the Indigenous entrepreneurs. Many entrepreneurs were part of significant formal and informal networks and were often volunteers or members of community groups and organisations. Mutual sharing was also noteworthy, particularly with the local Indigenous community. Many Indigenous entrepreneurs provided financial assistance, employment opportunities and sought to aid the Indigenous community in whatever way they could via their business.

While carrying out a thematic analysis of the data it became clear that the narratives were central to how Indigenous entrepreneurs recall, make sense of their experiencing in start up and running the businesses. The accounts gathered from the narrative case studies also revealed a tension between self‐identity and the constraints the identity place on business operations. Furthermore, the perception of the success was largely determined by self‐identify and particular context of Indigenous culture and community. Sharing personal experience can enrich learning by enabling knowledge and practices to be reviewed and evaluated. During this research with Indigenous entrepreneurs, personal observations were noted by researchers to enable more in‐ depth interpretations of the data gathered and also to create a greater learning experience for fellow researchers in the field of Indigenous entrepreneurship. The researchers’ own personal reflections on their observations, interpretations and impressions experienced before, during and after fieldwork can be summarised within the themes of sensitivity, intrusion, the need for community engagement; mutual exchange and recognition. Researchers on this project were often expected or encouraged to share personal experiences during the interview process. This often resulted in the Indigenous entrepreneur feeling more at ease and comfortable in the research setting. Feelings of intrusion was also a sensitive issue and one experienced particularly by the non‐Indigenous researchers of the group. A few Indigenous participants interviewed expressed extreme discomfort at the use of an audio recorder and many did not feel comfortable with the direct line of questioning involved with structured interviews. Building a rapport and developing a sense of mutual trust between participant and researcher often enabled the process to progress in a more efficient and relaxed manner.

References Aiatsis. (2011). Guidelines for Ethical Research in Australian Indigenous Studies, 2nd Edn. Canberra Australian Institute of Aboriginal and Torres Strait Islander Studies (AIATSIS). [Cited 17 December 2012.] Available from URL: http://www.aiatsis.gov.au/research/docs/ethics.pdf Atkinson, J. (2002) Trauma trails, recreating song lines: the transgenerational effects of trauma in Indigenous Australia, Spinifex Press, North Melbourne.

Altman, J.C. (2001) Indigenous Communities and Business: Three Perspectives, 1998‐2000. Working paper, No. 9/2001, The Centre for Aboriginal Economic Policy Research (CAEPR), Australian National University, Canberra

176

Branka Krivokapic‐Skoko et al.

Baguley, D. (2007) “Indigenous Business”. Management Today (August) pp. 22‐25 Brady, W. (1998) “The stories we tell. Storytelling – a passport across borderlines”, paper presented at the Winds of Change: Women & The Culture of Universities International conference, University of Technology Sydney, Sydney, 13–17 July.

Collins, J. (2004) “Koori Capitalism: Indigenous Entrepreneurs in Australia”, Regional Frontiers of Entrepreneurship Research 2004: Proceedings of the First Annual Entrepreneurship Research Exchange, Australian Graduate School of Entrepreneurship, Swinburne University of Technology, Hawthorne, pp 69‐82 Cope, J. (2005) “Researching Entrepreneurship through Phenomenological Inquiry: Philosophical and Methodological Issues”. International Small Business Journal, 23 (2), 163‐189 Dana, L. P., and Anderson, R. B. (2007). International Handbook of Research on Indigenous Entrepreneurship. UK and USA: Edward Elgar Publishing LTD. De Fina, A. (2009) “Narratives in Interview ‐ The Case of Accounts. For an Interactional Approach to Narrative Genres”.

Narrative Inquiry, 19 (2), 233‐258 Flamsteed, K, Golding, B. (2005) Learning through Indigenous Business – The Role of Vocational Education and Training in Indigenous Enterprise and Community Development. National Centre for Vocational Education Research (NCVER).

ISBN‐1‐9208‐9655‐4 Foley, D. (2003) “An Examination of Indigenous Australian Entrepreneurs”. Journal of Developmental Entrepreneurship, 8(2), 133‐152.

Foley, D. (2006) “Does Business Success Make You Any Less Indigenous?” Proceedings, Regional Footsteps of Entrepreneurship Research 2006: 3rd International Australian Graduate School of Entrepreneurship (AGSE) Entrepreneurship Research Exchange, 241‐257.

Foley, D. (2008a).“An Indigenous Standpoint Theory” in Gunstone, A. (ed) History, Politics and Knowledge: Essays in Australian Indigenous Knowledge. North Melbourne: Australian Scholarly Publishing, pp.113‐133.

Foley, D. (2008b) “Indigenous (Australian) Entrepreneurship?” International Journal of Business and Globalisation, 2(4), 419‐436. th Foley, D. (2010) How Does the Relationship between Human and Social Capital Impact on Indigenous Entrepreneurs. 24 USASBE (United States of America Small Business & Enterprise) Conference, Nashville, TN January 14‐17.

Lavallee, L. F., and Poole, J. M. (2010) “Beyond Recovery: Colonization, Health and Healing for Indigenous People in Canada”. International Journal of Mental Health and Addiction, 8(2), 271‐281 Mapunda, G. (2005) “Traditional Societies and Entrepreneurship: An Analysis of Australian and Tanzanian Businesses”.

Journal of Asia Entrepreneurship and Sustainability, 1(2), 1‐23.

Nakata, M. (1998). Anthropological Texts and Indigenous standpoints. Australian Aboriginal Studies, 2, 3‐12.

National Health and Medical Research Council. (2003). Values and Ethics: Guidelines for Ethical Conduct in Aboriginal and Torres Strait Islander Health Research. Canberra: NHMRC. Pearce, S.(2013). Indigenous Female Entrepreneurship and Gender in Australia. Paper presented to UTS Cosmopolitan and Civil Societies Research Centre, University of Technology, Sydney.

Pearce, S and Cox, E. (2009) Yarnin’ Research—Inner City Aboriginal Women and Domestic Violence, Mudgin‐Gal Aboriginal Women’s Corporation, Tiger Press, Redfern, NSW. Riessman, C. K. (2008) Narrative Methods for the Human Sciences. SAGE Publications, Thousands Oaks, CA Rigney, L. I. (1999) “Internationalization of an Indigenous Anticolonial Cultural Critique of Research Methodologies: A Guide to Indigenist Research Methodology and Its Principles”. Wicazo Sa Review, 14(2), 109‐121.

Robertson, B, Demosthenous, C and Demosthenous, H. (2005) ‘Stories from the Aboriginal Women of the Yarning Circle: When Cultures Collide’, Hecate Journal, vol. 2, pp. 35–44, viewed 26 March 2013, http://www98.griffith.edu.au/dspace/bitstream/10072/4159/1/29166.pdf Smith, L. T. (1999) Decolonizing methodologies: Research and Indigenous Peoples. London and New York: Zed Books Ltd. West, R., Stewart, L., Foster, K., and Usher, K. (2012) “Through a Critical Lens: Indigenist Research and the Dadirri Method.

Qualitative Health Research, 22(11), 1582‐1590.

Wood, G. J., and Davidson, M. J., (2011) “A review of male and female Australian indigenous entrepreneurs: Disadvantaged past–promising future?” Gender in Management: An International Journal, 26(4), 311‐326.

177

The Research Journey: Why it is Important to Think About it and Record it, as a Vehicle to Becoming a More Competent and Effective Researcher David Lamb Edith Cowan University, Joondalup, Perth, Western Australia d.lamb@ecu.edu.au

Abstract: In this paper, I draw on my personal experience of compiling a research journal to reflect on the research processes I was involved with as part of a study on family life and leisure in New Zealand. The research design for this study was based on phenomenology, where the experience of the subjects being studied was significant and involved developing an understanding of the lived experiences of pre‐birth and post‐birth couples, where the way they acted was dependent upon their understanding and meaning of their behavior (Waters, 1994). This study focused on two key life stages, first‐time pregnancy and first‐time parenthood, which is an under developed area of research in family leisure studies, as previous approaches have tended to concentrate on all phases of family life (Beck & Arnold 2008). The first stage of the study involved two focus group studies with a pre‐birth group and a post‐birth group and the second stage involved interviews with male and female partners at the pre‐birth and post‐birth stages, individually and then together as a couple. The final stage of the research involved a number of interviews with leisure facility managers working in the study area. The first part of the paper discusses the meaning and importance of reflection as an important part of the research journey, as a means of evaluating personal experiences as a researcher. The second part of this paper provides justification for using a research journal to record experiences to develop and understand research. In the third part of the paper, a number of the key research issues (research extracts) that emerged as part of the process of recording my research experience during the study are discussed and evaluated. In concluding the paper, the benefits of reflection through journal writing are re‐affirmed and through reading the paper, I hope to encourage some of you to consider recording your own research experience in a research journal.

Keywords: reflection, research journal, research extracts, research process, research journey, qualitative research

1. Introduction In the research literature, principally within the fields of psychology and sociology (Dewey, 1933) has identified that reflective writing practices are principally concerned with giving serious thought to experiences that are unfolding. With specific regard to research, reflection involves complex processing of information which helps the researcher to examine and evaluate their experience, rather than focus on the research outcome (Murray & Kujunzic, 2005). The reflective journal is one of many tools that can be used by a researcher in reflecting and examining their experience. However, with regards to the professional development of researchers, the lack of literature that discusses and evaluates the experiences of researchers in using research journals is limited (Borg, 2001). As a result little is known about the benefits of using research journals as part of the research process and the development of researchers.

The main purpose of this paper is to examine and record my experience as a researcher in the field, through the use of a research journal to help fill the gap in the research literature. The first section of this paper explains the key concepts of ’reflection’, after which the benefits of using a research journal are highlighted.

Next, the specific research journey I embarked upon in the field of family life and leisure is described, and in order to give context to this paper a number of research extracts sourced from the research journal are introduced and discussed to highlight some of the main concerns and issues encountered during this research study. In essence as Nadin and Cassell (2006, p.211) remark the journal “provided a forum for me to record these concerns which may have otherwise been lost, or at least simply not considered”. In the final section of this paper, I draw together and summarise the main tenants of the paper.

2. The art of reflection Reflection involves an action or as Boyd and Fales (1989, p.101) assert is concerned with ”the process of creating and clarifying the meaning of experience (present or past) in terms of self (self in relation to self and self in relation to the real world)”. This requires the researcher to write ‘something’ down’ which gives reflection more purpose and power (Yinger & Clark, 1981) and “the process of reflection helps to bring the unconscious into consciousness (Ortlipp, 2008, p.698.). This involves cognitive activity, where we are able to evaluate our experiences to give what Ellis (2001) refers to as ‘a measure of perspective’. According to Schön

178

David Lamb (1991) reflection is useful in helping bridge the gap between theory and practice and enables the researcher to identify ideas and start to make connections between different aspects of the research. Reflection involves higher order thinking processes (Wertenbroche & Nabeth, 2000) to analyse, reconsider and qualify our experiences (Murray & Kujundzic, 2005).

The act of writing helps facilitate the process of reflection in expanding our thoughts onto paper or as Luidens (1997, p.141) describes as a “manifestation of thinking”. Furthermore, the art of expressing our thoughts onto paper in a research journal provides evidence from which we are then able to draw conclusions about our research (Cui, 2012). Jasper, (2005, p.250) provides a more detailed account, in that “reflective writing is, by its very nature ….written in first person and is essentially subjective [and is connected] to the experiences and perceptions of the author”. Consequently, it could be argued that journal writing lacks objectivity and there is a danger of over indulgence of the self in embellishing what is recorded. Learning about research from recording our experiences has the potential to develop a range of research related skills including critical thinking, analysis and creativity (Jasper, 2004). By writing through reflection the researcher is able to facilitate new understanding as information is first perceived then reflected upon in the act of writing. This process acts to represent the researcher’s experience of the research first‐hand and gives meaning to the research leading to “understanding and learning about practice” (Jasper, 2005, p. 248). Moreover, by the art of reflecting, a researcher provides evidence to build their research experience, which also facilitates the “researcher’s discovery [in compiling] a veritable audit trail of the research process” (Jasper, 2005, p.250). Caution is needed to be mindful of the researcher’s role in deciphering how the insider‐outsider research continuum impacts on the depth of insights that can be gained from the research participants (Cui, 2012).

3. The research journal/diary As part of my journey through the process of completing the study on family life and leisure, I started to compile a journal on the research to document and record my experience as a researcher, once the literature review for the study had been completed, but before the collection of data commenced. For this purpose, I recorded my own feelings and thoughts in note form in a reflective research journal from the time of compiling the research proposal, on which this study was based, to the completion of data collection. I used a very simple template to structure my thoughts around a number of questions that I could relate my observations to. During intensive periods of the research processes that included undertaking focus groups and interviewing couples, I recorded my thoughts on a daily basis. At other less intensive periods of the research, I recorded my thought on a weekly basis, such as times when I was undertaking the literature review or making arrangement for the focus groups or interviews. To help structure the diary, I divided each page of the diary into four main sections to firstly record what went well, secondly record what did not go so well, thirdly record what I learnt from the experience and the final section referred to evidence to substantiate my comments and provide evidence to justify my reflections. Writing the reflective journal allowed me to acknowledge the thoughts and emotions I was experiencing, and in some cases shared with the people I spoke to in the focus group studies and in the interviews.

In many ways, the reflective journal allowed an outlet for my observations that were not recorded elsewhere and encouraged critical analysis and critical thinking, both key research skills in qualitative research. Writing the journal gave me time out to think about the research and make informed decisions of what was important and critical and what was not. The practicalities of recording my thoughts on the focus group process and interviews suffered some time lag in the recording of some of my observations and I appreciate that the quality of a research journal could be influenced by the researcher’s memory. This is one of the limitations of using this method to record experiences, as the researcher needs time and space to record their observations.

Taking time out from undertaking the research, through compiling the diary was a useful exercise to analyse my own thoughts during the research in a systematic way, by recording them in a research journal. In reflecting on the research process, I was able to focus on my thinking and looking in order to understand my experiences as they unfolded to question my beliefs and expectations which Gidman (2007) asserts develops the researcher.

For example, for the interviews with couples I had originally planned to just interview them together. As a result, of reflecting on the gender dimensions of the interview process with couples in the reflective diary, I then decided to interview couples on their own which allowed for interviewees to talk more freely without interruption from their partner. The process of reflection helped in examining and evaluating the research

179

David Lamb methods I utilised, rather than focusing on the outcome of the research (the data) as through reflection I was able to gain additional insights into the lived experiences of the individuals involved in the research. This helped facilitate a deeper level of questioning and ultimately helped in understanding the lives of research participants’.

I had planned to run the pre‐birth focus group for one hour and parents had arranged this time to be available.

In reality though, this focus group ran over time as I had let the conversation run off into areas that were not relevant to the research project. This only became clear after I reflected in the diary concerning my observations on the pre‐birth focus group and as a result I kept a much tighter rein on the post‐birth focus group discussion by following the pre‐determined questions more closely. This still allowed for flexibility, but gave the focus group discussions more direction and purpose in line with the key research questions. The knock on effect was that my own skills and ability in undertaking and managing the focus group meetings was improved. Furthermore, during the focus group discussion and interviews I was able to reflect on hidden data, concerning the feelings, attitudes and emotions that research participants were expressing. Without using a reflective diary this detail would have been lost to the researcher.

Gaining insights into the world of the researcher is often problematic, but the compilation of a research journal provides a window into their experience through personal reflection and can be utilised as an important methodological tool in “contributing to the trustworthiness of a research study” (Jasper, 2005, p.

248). This has the potential to provide the researcher with information to discuss less well documented issues in research with their research supervisor and can help identify any research difficulties or problems from the researcher’s perspective that might not be evident to the supervisor. Compiling information in a research journal provides evidence and another source of data for the researcher which “becomes a tangible way to evaluate our own experience, improve and clarify one’s thinking and finally become a better scholar” (Janesick, 1998. p.24).

The data presented in a research journal can be used as another form of data and to supplement primary sources of data. Also, as (Jasper, 2005, p.256) has noted the “data contains the researcher’s interpretation within their own cultural, social and experiential parameters”, which can be very empowering to the researcher (Finlay, 2002). However, one of the limitations in terms of research outputs which all academics are subject to, in universities worldwide, accounts of reflective writing are very difficult to publish or disseminate (Finlay, 2002). Nadin and Cassell (2006, p.210) argue this is often due to “the positive hegemony that permeates many of our research outlets.

The research journal became a useful vehicle to document experience and increases the awareness of the ‘self’ in research both as an individual and a professional (Borg, 2001). The journal acts as a vehicle to improve researcher development and “enables the researcher to continuously think about their own research practices and assumptions, by recording their thoughts in a systematic way” (Nadin & Cassell, 2006, p.210). However, there is a lack of guidance on writing reflectively in the research literature. This is especially the case for novice researchers with regards to “the purposes of keeping a reflective journal from a methodological perspective and how to use their perceptions as an integral part of the research process” (Ortlipp, 2008, p.696). Clearly more work is needed in this area to give future researchers the opportunity to record their experiences in a logical and structured fashion to later utilise as a source of data for the research.

4. The research extracts In this part of the paper, I outline and discuss five of the key issues (extracts) that emerged as part of writing the reflective journal based on my experiences throughout the time of the research project. In essence, the process of recording my reflections on various aspects of the research process by the physical act of writing became another form of data. When writing up the findings, I was able to refer to the research journal and provide further evidence on the key themes that arose as part of the study. Each extract below is given a title and is related to the broader context of the study to give meaning, to provide the reader with insights into the study from the researcher’s own personal perspective, which was only made possible by the compilation of a research journal.

180

David Lamb

4.1 Extract one: Getting into the field My initial interest in family life and leisure as a viable area of study can be traced back to my own personal experience, when becoming a parent and the impact, this significant life event had on my own life and leisure.

As a ‘first‐time’ parent, I was able to easily establish rapport during the focus group studies and interviews with pre‐birth and post birth couples to establish context, which built trust and confidence, enabling this study to progress (Cresswell, 1998). In this research study, I was able to gain insight and glimpses into the lives of couples experiencing a similar life event, in what Cui (2012) has described as ‘insider research’ where the researcher has some familiarity and connection with the people they are researching. I do not claim to have uncovered the ‘real truths’ about family life and leisure, but to have contributed to the literature and debates surrounding family leisure and the experiences of parenting for couples.

4.2 Extract two: Mixed methods In this study, as proposed by Mittelstaedt (2002, p.152), I used a mixed method approach to the research design to make it, “possible to assess each method used, providing information on their limitations as well as their strengths and clarifying their presuppositions and their consequences”. By adopting a mixed method and qualitative approach to seek out the opinions of couples, I sought to add to the research methods literature and offer additional insights into the lived experiences of couples, from a male and female perspective. The focus group studies combined with the interviews from a user and management perspective facilitated a deeper understanding into the phenomenon of family leisure. The qualitative tools used provided a means of questioning at a deeper level that led to a better understanding of the lives of couples involved in the study, but I appreciate that I was only able to tell part of the story.

This story is obviously selective, as ultimately I needed to make decisions about what would be included in the final version of the study. At the conclusion of the pre‐birth and post‐birth focus group studies I took notes and tried to replicate this at the end of each interview I undertook with pre‐birth and post‐birth couples and with leisure facility managers. This was a difficult and challenging task at times, as my primary purpose during the focus groups and the interviews was to concentrate on the participant’s feedback. Taking notes immediately after the focus group discussions and after each interview was at times inconvenient and impractical.

4.3 Extract three: Accessing research participants Gaining the confidence and trust of those selected as research participants was initially problematic, in the early stages of the research. However, prior to the design of the methodology, I engaged in early meetings with both men and women at the pre‐birth and post‐birth stages Consequently, their trust and willingness to participate in this study was acquired, which is a key factor in motivating people to give up their time to take part in research studies (Kenny, 1996). Also, I had empathy with their experience as I had personally encountered many of the same challenges that expectant couples and first time parents face. Assembling the focus groups took a lot of time, effort and energy and although many participants had initially given their consent to take part, they still needed further encouragement, persuasion and reminders. This was achieved through regular email contact and a number of personal phone calls to encourage their full commitment to the study. Therefore, regular and constant communication was the key to elicit their involvement in the study.

During the recruitment stage of the research, many practical problems had to be overcome. Identifying a suitable venue, time and date to suit everybody in the focus group studies caused some difficulties and required extensive consultation and negotiation and as a result was time consuming and sometimes very frustrating. For example, many post‐birth couples involved in the focus group meetings had difficulties arranging babysitters, so that both partners could be present. Some focus group participants had to be contacted on several occasions to confirm their attendance especially, couples in the pre‐birth focus group who were at the latter stages of their pregnancy.

Both pre‐birth and post‐ birth focus group participants were concerned that they might be discussing highly personal and sensitive matters with people they might not know in the focus groups. Consequently, some individuals needed re‐assurance of the processes involved and the fact that the research findings were to be confidential. For this purpose, they were given further guidelines on how the focus group conversations would be evaluated and analysed and were reminded that their comments could not be identified in the write up of

181

David Lamb the discussions. Also, all pre‐birth and post‐birth focus group participants were sent a summary of the findings of their discussions by email and asked for comment, before the findings were confirmed in written form as part of this study.

4.4 Extract four: Pre‐birth and post‐birth interviews The justification for using interviews was based on the premise outlined by Behringer (2006) as interviews are capable of producing rich, detailed information about the meaningful dimensions of lived experience, the dimensions being family life and leisure. My objective was directed towards allowing interviewees to describe the world as they perceived it, to be described in their own terms and to highlight what ‘things’ and events meant for them.

The interviews with couples took a lot of time, negotiation and effort on the part of the researcher and were more problematic than the individual interviews. During the couple interviews, I had to be conscious of one person dominating the other and be careful not to cause offence or conflict between couples on sensitive issues that arose. Nevertheless, a number of couples did argue with each other during the interviews, but fortunately disagreements never got out of hand.

The couple interviews offered insight into the interactions and nature of the relationship between couples, but I had to be mindful that the couple interviews were more likely to produce consensual data, where they might be an attempt to generate a unified reality (Paul, 1989). Also, as a researcher I needed to be aware of possible inconsistencies in the data, in that the construction of events during individual interviews might be quite different within the couple interviews. Fortunately, these concerns did not eventuate, but during the couple interviews I had to encourage men to be more engaged in contributing to the interview discussion. This was due to women often dominating the interview conversation as they were generally more vocal in both the couple and individual interviews.

I was very grateful that both pre‐birth and post‐birth couples were generous enough to give up their time and for both partners to be there at the same time for the ‘couple’ interview and post‐birth couples arranging childcare so the interviews could go ahead. I had to be flexible in terms of times and date and not get to upset when parents cancelled at short notice. When arranging the interviews for post‐birth couples it became clear that I had to plan for contingencies, as the demands of parenting are often unpredictable (Shaw, 2008). As a result, a number of interviews had to be re‐arranged for a number of post‐birth couples at relatively short notice. Likewise, I had to re‐arrange a number of interviews for pre‐birth couples that were in the last trimester of their pregnancy, as a number were very tired, stressed or simply did not feel like talking. I quickly realised and appreciated that looking after a young child or dealing with the demands of pregnancy were a lot more pressing than undertaking a study on family life and leisure.

Nevertheless, most couples were accommodating and did their utmost to meet pre‐arranged interview dates.

Some interviews took much longer to undertake than I had at first anticipated, which impacted upon my time management plan for the rest of this research study. Fortunately, I had put into place a number of contingencies that allowed extra time for the interviews to be completed in ensuring that my research plan was flexible to meet the needs of couples.

Generally, families were very generous with their time and most spoke openly about their feelings and thoughts and most interviews became a kind of conversation, where, interviewees were able to speak for themselves (Tolich & Davidson, 1999, p. 60). The majority of the interviews took place at the home of the interviewees, where they were intimately familiar with their surroundings and where most felt relaxed and comfortable. Throughout the interviews, I was conscious of the guiding principles outlined by Tolich and Davidson (1999, p.109) were effective interviewing should be concerned with, ”asking the right questions, in the right way, while ensuring you accurately record people’s responses to these questions”. A number of interviews were undertaken at the workplace of the interviewee, as I presumed they were more comfortable with this arrangement. I really enjoyed the interviews and found involvement with pre‐birth and post‐birth couples stimulating and a source of motivation to help progress and give impetus to the research study.

After the interviews were completed on many occasions, I often stayed for some time discussing a range of different issues, not all necessarily linked to the topic of this study, but still interesting and completed the

182

David Lamb journal entries whilst on site. I was offered food and drink during the interviews and on one occasion was served a three course meal while on another occasion, offered a bed for the night, as the interview has progressed into the early hours of the morning. At the end of each interview, I asked those involved to reflect on their interview experience and their feedback provided useful information to reflect upon.

The majority described their experience as relaxed, comfortable and interesting, as it became evident that and I was able to use the research journal as a learning resource to improve my moderator skills in the focus groups and my confidence grew in conducting interviews as the study advanced. During the solo interviews, individuals remarked that they had enjoyed their interviews, especially the one‐on‐one interaction with the interviewer, as this helped them remember important details about the questions they were asked.

In particular, they appreciated the open‐ended nature of the interviews they were involved with, as this gave them the flexibility and opportunity to express themselves freely in their own words. The ability to listen and later reflect in the form of writing was important in this respect. As a result, valuable data was not lost and made visible with regard to my feelings, thoughts and opinions that would not have been recorded elsewhere.

4.5 Extract five: Interviews with leisure facility manager Completion of the manager interviews took a long time to arrange, because many of the managers were very busy and had to cancel interview appointments I had made with them on several occasions. This was frustrating at times in terms of progressing the research, but I soon realised, that I would need to be patient as managers had given up their time freely to be involved in this study. Many of them had to cancel initial interview appointments, as they had other priorities that had arisen between the time I had first contacted them, and the original date we agreed for the interview to take place. In order to plan for this contingency, I asked each manager to offer two dates for a possible interview. If the first date was cancelled, which it invariably was, I still had a second date to fall back on, which in some cases was confirmed. This risk management strategy in using contingencies seemed a good idea in practice, but in reality a number of managers had to cancel both dates. In the majority of cases, the ‘actual’ interview took place on the second or third date that we had agreed upon. However, in the case of two managers, the actual interview took place on the fourth arranged interview date. I really enjoyed the interviews, even though they were ‘hard work’ (Rothman, 1986) as I was able to gain useful insights into the key research issue, from a management perspective.

5. Conclusion In this paper, I have proposed that the writing of a research journal is an important vehicle to learn about the experience of research. Furthermore, it is critically important to the documentation of the researcher’s personal research journey, the research framework and process and an important source of data to further validate the authenticity of qualitative research. Narratives that derive from research journals have the capacity to better inform other researchers, in particular early career researchers, about the realities of research, “not available from any other sources” (Borg, 2001, p.160) via discussion and analysis of researchers’ journals. Nevertheless, it seems that there is a lack of guidance and academic debate within the management sciences on how to do it and the difficulties and practicalities involved.

In writing the research journal, I was able to communicate my thoughts, feelings and opinions to make them what Ortlipp (2008) has described as visible. The writing of a research journal is one of a number of methods of reflection that a researcher can utilise to record their experiences in a simple and effective manner. I would encourage all those involved in the disciplines of business and management to consider adopting this strategy as part of their research plan, as it is apparent that reflecting on research is now a key issue for qualitative researchers in the management field (Nadin & Cassell, 2006). Although, business and management research has been dominated by the use of quantitative methods to help justify their claims as such methods produce quantifiable evidence (Mittelstaedt, 2002). In business/management research using research journals are justifiable for a number of reasons, alluded to earlier.

Although, the research project (family leisure) I relate to in this article is not within the business/management research field, the experiences outlined within this article are relevant to all researchers considering ways in which they can be more reflexive. In this paper, my major objective has been to describe the use of a research journal in a specific context (family leisure research). I would argue that the context is useful and of interest to

183

David Lamb those involved in business/management based research as the issues and concerns I have raised in this paper are evident in all types of social science research. I hope to have provided a real‐life insight concerning the practicalities and challenges of writing a research journal to encourage business and management researchers to seriously consider using this method to record important data, which is often neglected in mainstream research methods (Ortlipp (2008). Taking time out from the ‘doing’ of the research and finding space to compile a research journal provides the opportunity for the researcher to put reflection into practice by creating and clarifying the meaning of their own experiences “in terms of....self in relation to self and self in relation to the world” (Boyd & Fales, 1983, p.101).

References Beck, M. E., & Arnold, J. E. (2008). “Gendered time use in the home: An ethnographic examination of leisure time in middle class families”, Leisure Studies, Vol. 28, No.2, pp 121‐142.

Behringer, A. E. (2006). Making sense of husbands and wives. In J. F. Gubrium & J. A. Holstein (eds.), Couples, Kids and Family Life. (pp. 21‐43). Oxford University Press, Oxford, UK, Borg, S. (2001). “The research journal: A tool for promoting and understanding researcher development:” Language Teaching Research, Vol. 5, No. 2, pp.156‐177.

Boud, D., Keogh, R. and Walker, D. (1994) Reflection: Turning Experience into Learning. Kogan Page, London.

Boyd, E.M., & Fales, E. N. (1983). “Reflective learning: Key to learning from experience”, Journal of Humanistic Psychology, Vol. 23, No. 2, pp 99‐117 Creswell, J. W. (1998). Qualitative Inquiry and Research Design: Choosing among Five Traditions. Sage Publications, California.

Cui, K. (2012) “Substantiate the Reflexivity: The Insider‐Outsider Role of an Ethnographic Researcher”, Proceedings of the11th European Conference on Research Methods University of Bolton, UK, 28‐29 June 2012 Dewey, J. (1933). How we Think, Heath and Co, Boston, MA. Ellis, A.K. (2001). Teaching, learning and assessment: The reflective classroom, Eye on Education Inc, .New York, USA: Finlay, L. (2002). “Negotiating the swamp: The opportunity and change of reflexivity in research practice”, Qualitative Research, Vol. 2, No. 2, pp 209 230.

Gidman, J. (2007) Reflecting on reflection, In J. Woodhouse (ed.) Strategies for Healthcare Education: How to Teach in the 21st Century. Radcliffe Publishing Ltd, Abingdon, Oxon.

Janesick, V.J. (1998). Journal writing as qualitative research technique: History issues and reflections. Paper presented at the Annual meeting of the American Education Research association, San Diego, C.A. April 13‐17, 1998.

Jasper, M. (2004). Using journals and diaries within reflective practice. In Bulman, C., Schulz, D.S. (eds.) Reflective Practice in Nursing, Blackwell Publishing, Oxford Jasper, M. (2005). “Using reflective writing within research”, Journal of Research in Nursing, Vol. 10, No.3, pp 247‐260.

Kenny, D. A. (1996). “Models of non independence in dyadic research”, Journal of Social and Personal Relationships, Vol. 13, No. 2, pp 270‐294.

Luidens, P.M. (1997). Paper thinking: The process of writing, in Costa, A.L.., & Liebermann, R.M. (ed.), Envisioning process as content: Towards a renaissance curriculum. Corwin Press, USA. st Mittelstaedt, R. D. (2002). “Measuring leisure in the 21 Century: The strengths and limitations of three techniques”, Leisure/Loisir, Vol. 26, No.3/4, pp 147‐190.

Murray, M., & Kujundzic, N. (2005) Critical reflection: A textbook for critical thinking, McGill‐Queen’s University Press Quebec, Canada.

Nadin, S., & Cassell, C. (2006). “The use of a research diary as a tool for reflective practice: Some reflections from management research”, Qualitative Research in Management & Accounting, Vol. 3, No. 3, pp 208‐217.

Ortlipp, M. (2008). “Keeping and using reflective journals in the qualitative research process”, The Qualitative Report, Vol. 13, No. 4, pp. 695‐705 Paul, J. (1989). Money and Marriage, Macmillan, Basingstoke.

Rothman, B.K. (1986). “Reflections on hard work”, Qualitative Sociology, Vol. 9, No. 1, pp 48‐53.

Schön, D. (1991) The Reflective Practitioner: How practitioners think in action, Avebury Academic Publishing, Aldershot.

Shaw, S. M. (2008). “Family leisure and changing ideologies of parenthood”, Sociology Compass, Vol 2, No. 2, pp 688‐703.

Tolich, M., & Davidson, C. (1999). Starting Fieldwork: An Introduction to Qualitative Research in New Zealand, Oxford University Press, Auckland.

Waters, M. (1994). Modern Sociological Theory, Sage Publications, London Westenbach, A., & Nabeth, T. (2000). “Advanced learning approaches and technologies: The CALT perspective”, http://wwwinseadfr/CALT/publications/CALT report/caltperspectivepdf.

Yinger, R., & Clark, M. (1981). Reflective journal writing: Theory and practice. (Occasional Paper No.50), East Lansing, Mt Michigan State University. Institute for Research on Teaching.

184

The use of Storytelling as a Research Method: The Case of the Police Service of England and Wales Ron Lawson1, Gary Shaw2, Gail Sanders1 and Peter Smith1 University of Sunderland, UK 2 Serious Organised Crime Agency, UK

1

ronald.lawson@sunderland.ac.uk

Abstract: Storytelling is increasingly recognised as a research method in its own right, with the power to engage and explore issues of practice and identity. This paper focusses upon the practical use of storytelling as a research method, using a case study within the police force of England and Wales. The paper examines the way in which storytelling has been used by the authors to enable members of the police to explore their own professional identity. The police are currently undergoing the most radical leadership and management reforms of the last 70 years, and a critical spotlight is openly focused on the professional conduct and competence of its investigators. New questions are being asked in the training and education of these investigators. What does it mean to be an investigator? What assumptions underlie and define their professional identities and behaviours? And how can the dynamics of professional competence be developed and embedded in the identities and culture of the police service? There is much discourse around identifying the ‘threshold concepts’ which a practitioner needs to grasp to make sense of their professional behaviours. In the context of the professionalism of police investigators, we postulate that the threshold concept can be professional identity. To reflect on their own professional identity and how it impacts upon their investigative behaviours requires the individual to critically analyse those crucial moments in their experience when assumptions of professionalism have been challenged. In this paper we demonstrate how the application of long‐standing research in storytelling offers these detectives alternative ways to reflect on assumptions, and how these influence investigative behaviours. We argue that storytelling can be used as a research method to stimulate a creative perspective in the mindset of professional, promoting a more complex understanding of their professional identity, and thereby positively impacting on their practice.

Keywords: storytelling, practitioner research, impact, police investigation, reflective practice

1. Introduction This section introduces the main concepts on which this paper is based: the place of professionalism within the Police Force, the “profession” and professional identity, and the use of storytelling for management development.

1.1 Professionalism in the Police Force In line with the Coalition Government’s reform agenda on policing, they commissioned a review of police leadership and training within the police forces of England and Wales. The review concluded that the model in use today is essentially the same model which was designed back in the 1940’s to meet the challenges of Post War Britain. It argued that modern policing requires both a new approach and a new professional body to embrace the professionalism of the police service. This professional body would enable the transformation of the culture of learning in the police service in partnership with Higher Education (HE), moving away from in‐ house delivery of leadership programmes (Neyroud, 2011).

The review was well received by the Association of Chief Police Officers (ACPO) who recognised that raising the status of policing to that of a profession would represent a step change for the service (ACPO, 2012). Home Secretary Theresa May said it would “open up the closed system of leadership within the police service, harness greater diversity and experience at senior level, and equip the service with the skills it needs in a leaner and more accountable environment” (May, 2011).

The performance of the police service, and in particular police investigations, is constantly under scrutiny by the Government and the Criminal Justice System. The Police Reform Act (2002) and the Review of Police Leadership and Training (Neyroud, 2011) emphasised this need for improvement, to provide a modernised, consistent and more professional level of service. The Home Office, at the forefront of this drive, introduced the first element of initiative, the Professionalising Investigations Programme (PIP) (Public Service Review – Home Affairs, 2012). PIP is a major change programme for the Police Service of England and Wales designed to improve the professional competence of all police officers and staff who conduct investigations.

185

Ron Lawson et al.

PIP is expected to impact on the culture and philosophy of policing by fundamentally altering the police service response to investigation (Public Service Review – Home Affairs, 2012) and as such this programme is one of cultural change. ACPO and the National Police Improvement Agency (NPIA) have gone a long way to establish structural support to this change programme in the creation of the Implementation Support Plan for the PIP (ACPO Centrex, 2006). The plan details the benefits, guidance, language, assets, resources and processes involved, establishing the architectural codes needed for the change. However, little is mentioned of the cultural codes, which will also affect the change process. The cultural codes represent the interdependent sets of values, beliefs and behaviours that are common, and tend to perpetuate over a long period of time (Kotter and Hesketh, 1992). The development of the professional identity literature is linked with that of reflective practice (Schön, 1983). For a starting‐point definition we can look to Schein (1978) who offered a relatively fixed notion of professional identity as: “The relatively stable and enduring constellation of attributes, beliefs, values, motives and experiences in terms of which people define themselves in a professional role” Schein (1978).

In simplistic terms, the operational culture is the established “How things are done around here” (Drennan, 1992), the way in which the community of practice sees itself, creating an identity, a way of being that influences the way of doing. The complex nature of roles within the Police Service also provides further sub‐ cultures (Salaman, 1979), whereby the nature of the investigation carried out by officers such as traffic officers with road accidents or detectives investigating criminal offences will be affected by the way those particular groups of officers perceive their professional identity, which in turn influences their actions. This what Rogers (2004) refers to as “being” (the professional identity) and “doing” (actions). The understanding of professional identity in the police cultural context and its influence on investigation is crucial to the success of the PIP. Next we will discuss the concept of the profession and professional identity in more detail.

1.2 Professional identity The concepts of profession have emerged over time. Stan Lester (2010) classified the profession in terms of four models: classical; trade; technical and reflective. The classical model, which appears to have influenced Neyroud’s review on police leadership and training, is founded on a broad base of learning and culture. This is the archetypical model for ancient professions with an expectation that professional practitioners will have a broad general education as well as specific expertise in the areas of practice (Lester, 2010). The trade model grew out of medieval trade occupations, based upon practical expertise and craftsmanship. The technical model grew out of the industrial revolution, standardising training, and formal control of entry routes with specialism demarcation and expertise. The reflective model has an emphasis on learning through action and st reflection (Schon, 1987), making judgements in uncertain contexts, and is perhaps more fitting for 21 century policing. Lester generalises that the newer models have been overlaid on earlier ones as perceptions of professional activity have changed. The reflective concept of profession offers a portfolio of learning activity individual to the practitioner, integrated by personal identity, perspectives, values and capability. There is much discourse around identifying the ‘threshold concepts’ which a practitioner needs to grasp to make sense of their professional behaviours (Meyer and Land, 2005). In the context of the professionalism of police investigators, these threshold concepts can be assumed to be professional identity and reflective practice.

Next we will discuss the potential for storytelling for management development.

1.3 Storytelling for management development Storytelling is increasingly becoming recognised as an established, valid and powerful method of qualitative research. Organizational storytelling, in particular, is now often used to explore issues in the study of strategy and organizations (Du Toit, 2003 & 2007). Reissner and Du Toit (2011) propose a four‐stage model of storytelling, informed by a social constructionist view of coaching, which conceptualises the coaching process as a series of storytelling activities. They also provide empirical evidence of the role that coaching can play in team learning (Reissner and Du Toit, 2012) using as case study a university development programme for frontline family support workers. Reissner (2011) explores stories of organisational change identifying three patterns of story: stories of “the good old days”; stories of deception, taboo and silence; and stories of influence.

Combining storytelling with reflection can be used as a powerful management development tool, providing experienced professionals with tools which they can use to explore their own professional identity and thus gain a deeper understanding of the personal and professional values which drive and enable their day to day

186

Ron Lawson et al. practice. This paper explores the use of storytelling and reflection within a management development programme for police officers, and demonstrates the impact which the storytelling process can have as a research method.

2. The development programme This paper uses as a case study a Masters level development programme which has been specifically designed for the police, and aims to address the PIP agenda and the professional issues alluded to above. The programme also builds on research by colleagues in the areas of storytelling (Du Toit & Reissner, 2011), coaching (Du Toit, 2007), reflection (Sanders, 2011) and work‐based learning (Sanders, 2010).

The programme provides police officers with the opportunity to study leadership and management theories in the context of work‐based investigative practice. The Police Service is currently going through the most radical reforms in modern history in relation to the professionalism of investigative processes and practice. These reforms have brought about changes in the ways in which investigations are led, managed and carried out.

This programme aims to reflect upon current problems and future potential solutions within this professional context. The programme has been designed in response to the current sector needs in policing and the structure, content and assessment strategies have been informed through joint discussions between the university team and senior police representatives.

The objectives of the programme are to: provide an opportunity to study investigative management at Masters level in a programme offering academic rigour with a strong focus on critical application and professionalism; develop systematic knowledge and understanding of investigations in the context professional practice; build up a practically oriented and detailed knowledge of management theory that can be readily applied to solving investigation‐related problems at a managerial level; and enable participants to develop skills to improve investigative performance within their existing or future employment.

The aim is to introduce learners to the wider impact of their professional leadership and decision‐making in investigations, both for themselves and their organisations. It provides an opportunity for the participants to consider the changing nature of their professional environment, collaboration between agencies and the alternative perspectives of investigation stakeholders, in particular those of the victim. Students are introduced to various theories of professional leadership and management focusing on a process of critical reflection upon their professional identity and its impact on investigations.

This paper is particularly focussed upon the professional leadership module, which asks participants to reflectively explore crucial experiences that they have found challenging in the management of investigations.

They use storytelling as a research method to engage and explore issues of practice and identity. In so doing the police officers develop a deep understanding of the practical issues of managing and leading investigations with a view to improving current practice. The next section discusses the use of reflection in the development programme.

3. Reflection in action For many officers, the first time they are exposed to the concept and power of reflection is when it is introduced as part of the programme. Some find the concept difficult to grasp, and consequently they can be sceptical of its practical use. However, experience has shown that even the sceptics soon grasp the way in which reflection can be used to explore and improve their practice, and by the end of their studies they are converts, using reflection as a tool in their day to day activities. To illustrate, the remainder of this section is taken from a piece of work written by an officer who had recently been introduced to reflection. His words have been reproduced with his permission.

“Due to the very nature of their roles, police officers are often required to make decisions whilst under a number of competing pressures and demands. The ability and necessity to make sound, rational and informed decisions is central to successful policing and a vital skill for police officers, whose thoughts and actions are subject to unprecedented exposure and scrutiny in an environment of ever increasing media exposure and public accountability. Flawed decision making has been responsible for failed investigations and miscarriages of justice such as the Stephen Lawrence Inquiry and the Rachel Nickell murder investigation.

187

Ron Lawson et al.

Recently, the Hillsborough Enquiry has demonstrated that many years later, officers may be required to justify the decisions they made. The attitude and behaviour of police officers during the course of their duties has far reaching consequences for the professional reputation of the police service as a whole. It is a common assumption that if a police officer is ‘experienced’, his/her decision making processes, and rationale for such decisions will be derived from this.

However, the ability to learn and develop from previous experience is dependent on the quality of that experience and how well the individual or organisation is equipped to identify learning needs which may subsequently arise as a result.

Investigators rely on a set of working rules which are developed from both their own previous experience, and also that of others. The influence of colleagues in developing these working rules may lead to practices which become so familiar to the officers that they are not even aware that they are using them, and the concept that these are right because they have never been wrong is further reinforced within the organisation. To be reliant on the set of rules they have learned ‘on the job’ or from colleagues, is highly dependent on the quality and depth of that personal experience. I would argue that the mere fact that an officer has previous experience does not necessarily guarantee that he/she will benefit from it in future practice. For example, an inept officer may continue to exercise his/her duties without ever gaining insight into the fact that his/her actions are flawed.

In order to truly learn and develop from their experiences, it is vital that the individual is able to utilise that experience as a learning process. A key ingredient for successful learning outcomes from previous experience is the adoption of some form of reflective practice. Whilst reflection is an effective means of developing expertise, I would argue that many officers are not familiar with the process, in what is a purposeful activity and not merely a recollection of events. Having joined the Police service in 1986, I was subjected to the style of training that was delivered in regional training centres during that period, with learning being classroom based and by rote.

This was my first introduction to the concept that habitual behaviour is entrenched in police culture, and officers were not invited nor expected to air their views or share their thoughts, and critical reflection or reflective practices were never mentioned.

As my career progressed I was given a greater insight into the habitual behaviours and cultural norms that determined and directed the way that we conducted our business. Investigative skills were passed down from detectives whose own abilities depended on their previous investigative experiences and the lessons learned, notably when investigations had encountered problems or difficulties. Due to the dynamic environment in which police officers operate, there may be certain situations in which one may not possess prior knowledge, or the knowledge one does have may not fit the situation. For example, in my own practice I was dealing with a complex investigation which was subject to multiple changes of direction at short notice.

Having developed my skills in reflective practice, I was able to utilise my knowledge accordingly and through utilisation of this framework I made sense of the situation and led the team methodically and with clear understanding. Reflective practice provides a framework through which experience can become a learning opportunity. Whilst the concept of reflection and reflective practice has been introduced through Police training in recent years, it is a phenomenon that remains alien to many officers who are senior both in terms of rank and length of service, and who are important decisions makers. There is a significant need for the organisation to promote reflective practice to all officers and provide encouragement for officers to embrace this concept as a means of consolidating experience. The process should be embedded within the organisational culture, with a supporting framework that ensures that all opportunities for learning and development are fully maximised.”

The above quotation shows how an officer, who was initially sceptical of the practical use of reflection, began to use it to explore his own professional experiences and make sense of them. He also began to realise that there were different approaches to learning and development than those which he had been exposed to in his early career. Further he began to see the power of reflection, and became firmly convinced that it should be used not only as a training and development tool, but also that is should be embedded into day to day policing practice. The next section discusses the storytelling workshop which has been introduced into the management development programme.

188

Ron Lawson et al.

4. The storytelling workshop Storytelling is increasingly recognised as a research method in its own right, with the power to engage and explore issues of practice and identity (Ibarra, 2010), and has been developed as a practical teaching method on the above mentioned development programme. Even though reflective practice has become an effective learning mechanism in police education, for both pre and post probationary officers (Copley, 2011), as discussed above this reflective process can prove challenging, particularly for those mid‐to‐late career detectives whose thinking and writing skills have been habitually honed only in ‘matters‐of‐fact’ or who’s ‘crucial moments’ are so unpleasant that revisiting them is psychologically resisted. To reflect on their own professional identity and how it impacts upon their investigative behaviours, requires the individual to critically analyse those crucial moments in their experience when assumptions of professionalism have been challenged. Acknowledging professional identity as a threshold concept that requires deep individual contemplation and reflection calls into question the traditional generalised classroom taught approach. It is argued here that an alternative approach is needed for the exploration of professional identity. We argue that reflective practice aligned with storytelling is employed to provoke and explore the central assumptions that influence professional identity and practice.

The approach that has been developed over several years is a storytelling workshop. The workshop grew out of teaching ‘extending professional competence’ as a subject to police investigators studying on an undergraduate degree course. The format of the workshop is grounded in the four phases of the experiential learning cycle (Kolb, 1984) and is conducted as a half‐day session. First the students are asked to remember a crucial professional challenge that they have personally experienced in their career as an investigator, and simply tell the story to the group. Having set the scenes, the students are then introduced to the underpinning theoretical concepts of professional identity, reflective practice, triple‐loop learning and storytelling as an autoethnographic research method. In the second phase, to engage in reflection on their stories, the students are asked to retell their story using a creative technique as an alternative form of communication, such as a drawing, painting, poetry, or lyrics to a song. This approach is to stimulate a creative perspective in the mindset of the student as they enter phase three of the workshop. In this abstract conceptualisation phase the students are asked to think through and question their original stories, promoting a more complex understanding of their assumptions leading into, during and after the crucial challenge. At this point the students then engage autoethnographically as they are asked to focus not just on their own version of the story but also to give imagined voice and dialogue to others involved, such as witnesses, colleagues, victims or suspects; what would their version of events be? This powerful perspective shift is finally followed by a discussion around the workshop learning and how reflective practice can be implemented in the future.

An important part of the abstract conceptualisation phase of the workshop is the questioning of the stories and the assumptions they support. Many of the police students express concern that the majority of their investigative problem solving skills have been passed down from more experienced detectives who simply attempt to replay the successful actions of the past (single‐loop learning), without questioning the thinking strategies behind those actions (double‐loop learning). In triple‐loop, transformational learning (Argyris, 1993) the final loop is to challenge the cultural beliefs and assumptions that have informed the thinking strategies. It is difficult however to question one’s own beliefs, assumptions and supporting stories that have self‐ perpetuated over the years. The exploration of alternative narratives and the use of images, or poetry etc. can provoke a change in any old habitual thought patterns and aid the questioning and testing of assumptions.

Storytelling as an autoethnographic reflective tool gives voice to the student as being both the researcher and the researched, emerging out of the need to recognise the individual in research and capture their experience (Muncey, 2010). Using the storytelling workshop to draw out the memory, truth and meaning of the individual’s experience provides a safe environment to overcome potential resistance to truly challenge assumptions. The use of fictitious names and juxtaposed alternative and imagined narratives can further be used to create truly hypothetical scenarios for exploration, experimentation and learning. The autoethnographic storytelling and writing strategies used in the workshop can be categorised as critical analytical practices (Ellis, 2008) that not only facilitate learning about the subject of the stories, but also provides the students with critical, analytical and reflective skills for use in all aspects of their professional life.

From a facilitating perspective the challenges of overcoming any initial resistance to engage are soon rewarded as a wave of collective understanding sweeps the room on the richness of shared stories. Even though each

189

Ron Lawson et al. story is unique in its own right, a common theme develops as the students recognise the impact that cultural norms and assumptions have on their stories. This contextual understanding segues into transformational learning of professional identity through reflective practice, crossing the tipping point of these threshold concepts. To quote one student “I wasn’t keen on this ‘Jackanory’ stuff, I’m not a kid I’m a Detective Inspector, I came here to be taught not tell stories! Then I realised my whole career is based around stories, proving them and disproving them, sorting the facts from the fiction. I didn’t realise how much my interpretation of stories, my own included, was influenced by who I am and how I see myself professionally”.

Such reactions are not uncommon. The officers are used to telling and writing stories. Of course, they do so all the time when making notes, taking statements, and undertaking interviews. However the combination of storytelling with reflection adds a new dimension and power to the storytelling process, and enables them to analyse and questions their own values, assumptions and norms, and thus gain a deeper understanding of their own professional identity.

5. Conclusions This paper has explored the power of storytelling as a method for enabling experienced professionals to explore their professional identity. The authors have developed techniques for combing storytelling with reflection and are using these as tools within a management development programme. A case study of a management development programme for police officers has been used to demonstrate the power of this technique which is delivered during a creative writing workshop. The police officers are initially sceptical of the technique but begin to realise its power and potential, and use it to obtain a deeper understanding of their professional identity, which they can then use to improve their practice. This work is set in the context on recent national policy, which aims to further professionalise policing. The authors believe that storytelling has much to offer as a tool to enable professionals to develop and impact upon their practice.

References Argyris, C. (1993) Knowledge for Action. San Francisco, Jossey‐Bass.

Copley, S. (2011) Reflective Practice for Policing Students, Learning Matters, Exeter Du Toit, A. (2003) "Knowledge: a sense making process shared through narrative", Journal of Knowledge Management, Vol. 7 Iss: 3, pp.27 ‐ 37 Du Toit, A. (2007) "Making sense through coaching", Journal of Management Development, Vol. 26 Iss: 3, pp.282 ‐ 291 Drennan, D. (1992) Transforming company culture. London: McGraw‐Hill.

Ellis, C. (2008) The ethnographic I: A methodological novel about autoethnography. Walnut Creek: Altamira Press Police Reform Act: Elizabeth II. (2002) London: The Stationary Office Ibarra, H. (1999) ‘Provisional selves: Experimenting with image and identity in professional adaptation,’ Administrative Science Quarterly, 44: 764‐791.

Ibarra, H. (2003) Working identity: Unconventional strategies for reinventing your career. Boston: Harvard Business School Press.

Ibarra, H. (2010) ‘Identity as narrative: Prevalence, effectiveness, and consequences of narrative identity work in macro work role transitions,’ Academy of Management Review, 35 (1): 135‐154.

Ibarra, H. (1999) ‘Provisional selves: Experimenting with image and identity in professional adaptation,’ Administrative Science Quarterly, 44: 764‐791.

Kolb D A. (1984) Experiential Learning: Experience as the Source of Learning and Development. Englewood Cliffs, NJ: Prentice Hall Kotter, J P., Hesketh, J L. (1992) Corporate culture and performance. New York: Free Press.

Lester, S. (2010) On Professions and being professional. [Online] Available at http://www.sld.demon.co.uk/publications.htm. (Accessed: 23rd June 2012) May, T. (2011) ‘Home secretary announces body to professionalise policing,’ Guardian, 15 December 2011 [Online] Available at: Muncey, T. (2010) Creating Autoethnographies. London. Sage. http//www.guardian.co.uk/uk/2011/dec/15/home‐ secretary‐police‐body (Accessed: 19th June 2012) Neyroud, P. (2011) Review of Police Leadership and Training: Volume 1, London: Home Office.

Reissner, S C. (2010) Change, meaning and identity at the workplace.Journal of Organizational Change Management, 23(3), 287‐299.

Reissner, S C. (2008) Narrative and story: New perspectives on coaching.The International Journal of Mentoring and Coaching, 6(3), 1‐5.

Reissner, S C. (2008) Narratives of organisational change and learning. Cheltenham: Edward Elgar Publishing.

Reissner, S C., Du Toit, A (2011) "Power and the tale: coaching as storytelling", Journal of Management Development, Vol. 30 Iss: 3, pp.247 ‐ 259

190

Ron Lawson et al.

Reissner, S C., Du Toit, A, (2012) "Experiences of coaching in team learning", International Journal of Mentoring and Coaching in Education, Vol. 1 Iss: 3, pp.177 ‐ 190 Reissner, S C. (2011) "Patterns of stories of organisational change", Journal of Organizational Change Management, Vol. 24 Iss: 5, pp.593 ‐ 609 Rogers, J. (2004) Coaching skills: A handbook. New York: Open University Press.

Salaman, G. (1979) Work organisations. London: Longman.

Schein, E H. (1978) Career dynamics: matching individual and organizational needs. London: Addison‐Wesley Pub. Sanders, G. (2010) Towards a Model of Multi‐organisational Work‐based Learning: developmental networks as a mechanism for tacit knowledge transfer and exploration of professional identity, Learning and Teaching in Higher Education.

Sanders G, Kuit J, Smith P, Fulton J, Curtis H. (2011) Identity, reflection and developmental networks as processes in professional doctorate development, Work Based Learning Journal Schön, D A. (1983) The reflective practitioner: How professionals think in action. London: Ashgate.

191

Application of the Structural Equation Model to Develop an Employee Satisfaction Diagnostic Model Ophillia Ledimo and Nico Martins University of South Africa, Pretoria, South Africa manetom@unisa.ac.sa martin@unisa.ac.sa

Abstract: The purpose of this paper is to explain the application of a structural equation model (SEM) for developing a diagnostic model to measure employee satisfaction. By using the model development strategy proposed by Hair, Anderson, Tatham and Black (2005) to develop a diagnostic model for employee satisfaction, one can explain the role of SEM in theory development. There is a need for employee satisfaction diagnostic models that provide empirical evidence to enable organisations and practitioners to initiative interventions aimed at addressing areas of dissatisfaction, which are identified as developmental areas, and to leverage on its strengths, which emerge as areas of satisfaction in the organisation. Based on a literature review, a theoretical model was developed as a framework in terms of which organisations can diagnose their employees’ satisfaction. It highlights dimensions such as organisational strategy, policies, processes and outcomes in order to determine employees’ contentment and fulfilment in the organisation. A quantitative study was conducted with a sample of size N = 759, consisting of employees of a water utility organisation. An Employee Satisfaction Survey (ESS) was used to measure employee satisfaction, using both online and paper‐based methods.

Following the steps of SEM analysis, the three domains and latent variables of employee satisfaction were specified as organisational strategy, policies and procedures, and outcomes. Confirmatory factor analysis of the latent variables was conducted, and the path coefficients of the latent variables of the employee satisfaction model indicated a satisfactory fit for all these variables. The goodness‐of‐fit measure of the model indicated both absolute and incremental goodness‐of‐fit; confirming the causal relationships between the latent and manifest variables. It also indicated that the latent variables, organisational strategy, policies and procedures, and outcomes, are the main indicators of employee satisfaction. The development of a diagnostic model to measure employee satisfaction confirms the advantages of using SEM as a statistical procedure for developing a theoretically justified new model for employee satisfaction.

Keywords: employee satisfaction, structural equation modelling, organisational strategy, policies and procedures, outcomes

1. Introduction In the current world of work, organisations are concerned with the implementation of programmes and interventions to improve and sustain employee satisfaction in order to have a competitive advantage. Hence, employee satisfaction surveys are intended not only to ascertain satisfaction levels, but also to determine the improvements that need to be made (Chen et al 2006: 486). While all organisations could use surveys to provide a baseline for their existing employees’ satisfaction levels, it remains uncertain whether such surveys are based on a diagnostic framework. Thus, the objective of this research was to apply a structural equation model (SEM) to develop an employee satisfaction diagnostic model that can assist organisations to identify its strengths and developmental areas.

2. Problem statement To gain understanding on employee satisfaction, organisational development theorists need to offer empirically derived evidence to substantiate the use of organisational development models. Hence, there is still a need for employee satisfaction diagnostic model that provides statistical evidence to support its development. Therefore, the purpose of this paper is thus to explain the application of SEM as statistical procedure to develop a diagnostic model for measuring employee satisfaction.

3. Structural equation modelling (SEM) According to Hair et al (1998:583), SEM refers to a “multivariate technique combining aspects of multiple regression and factor analysis to estimate a series of interrelated dependence relationships simultaneously”.

Confirmatory factor analysis is conducted to determine the validity of the domains of the model. The purpose of the SEM is to mainly allow the researcher to examine the relations between indicators and their associated latent variables representing the constructs in the theory, as represented in the confirmatory factor analysis measurement model (Kline 1998). In order to achieve the above purpose of SEM, the follow are the main stages to be followed:

192

Ophillia Ledimo and Nico Martins Stage 1: The development of a theoretical model. This initial stage involves using all available relevant theory, research and information to construct the theoretical model of the construct (Hair et al 1998).

Stage 2: Constructing a path diagram of causal relationships. It is essential to specify the relationships between the relevant variables describing the phenomenon of study; using graphs with one‐headed arrows indicating causal relationships or two‐headed arched arrows indicating mutual dependencies (correlation) (Kline 1998).

Stage 3: Converting the path diagram into a set of structural and measurement models. Hair et al. (1998:619), describe confirmatory factor analysis as “the use of multivariate techniques to test or confirm a pre‐specified relationship”. The fundamental hypothesis of the SEM is that the covariance matrix of the observed or manifest variables is a function of a set of parameters which, in this study, means the relationship between latent variables and between the latent and the observed variables (Bollen 1989).

Stage 4: Choosing the input matrix type and estimating the proposed model, and stage 5: assessing the identification of the structural model. A correlation matrix was used because the purpose of this study was to explore the pattern of interrelationships between the latent and manifest variables (Hair et al 1998). Once the structural and measurement model have been specified and the input data selected, the AMOS technique of the SPSS computer program can be used to actually estimate the model. The goodness‐of‐fit of this model is first assessed for the overall model and then for the measurement and structural model separately. Finally, the proposed model is verified and required significant modifications of the model are explored.

Stage 6: The evaluation of goodness‐of‐fit (GFI). The GFI statistics determine if the data fit the model (Bollen 1989; Hair et al 1998), focussing on the degree to which the specified indicators represent the hypothesised constructs. It also evaluates each construct to examine the indicator loadings for statistical significance and to assess the construct’s reliability and variance extracted (Hu & Bentler 1998). The following indices are used to evaluate the model:

Goodness‐of‐fit index (GFI). A value of 0 reflects no fit, while a value of 1 is a perfect fit. Values close to 0.90 reflect an acceptable fit.

Normed fit index (NFI). A value of 0 reflects no fit, while a value of 1 is a perfect fit. Values close to 0.90 reflect an acceptable fit.

Incremental fit index (IFI). A value of 0 reflects no fit, while a value of 1 is a perfect fit. Values close to 0.90 reflect an acceptable fit.

Comparative fit index (CFI). A value of 0 reflects no fit, while a value of 1 is a perfect fit. Values close to 0.90 reflect an acceptable fit.

Roots mean squared error of approximation (RMSEA). A value of 0.05 represents a close approximate fit; values between 0.05 and 0.08 suggest a reasonably approximate fit and values greater than 0.10 suggest a poor fit.

Stage 7: Interpreting and modifying the mode: This is the final stage of SEM and it entails interpreting and modifying the model. According to Hair et al (1998:654) “possible modifications to the proposed model may be indicated through examination of the normalised residuals and the modification indices”. The modification index from AMOS exists for each fixed parameter in the model and it is used to estimate or predict improvement in the model fit by setting the parameters free (Hu & Bentler 1998; Hair et al 2005).

4. Literature review on employee satisfaction Employee satisfaction has been conceptualised and defined differently by various researchers and authors in the literature. The following are some of the earliest and latest definitions:

According to Lofquist and Dawis (1969:53), satisfaction is “a function of the correspondence between the reinforcement from the work environment and the individual’s needs”.

Locke (1976:130) states that employee satisfaction can be viewed as “a pleasurable or positive emotional state resulting from the appraisal of one’s job or job experience”.

Martins and Coetzee’s (2007: 21) definition states that employee satisfaction is a “pleasurable or positive emotional state resulting from an employee’s appraisal of his or her company environment or company experience”.

193

Ophillia Ledimo and Nico Martins

Voisard (2008:6) defines employee satisfaction as the “employee’s feelings or state‐of‐mind regarding the nature of their work and conditions of employment with a particular employer”.

A critical analysis of these definitions indicates that all the definitions emphasise the fact that employee satisfaction is an emotional response to the individual’s experience or appraisal of organisational practices and processes. This view of satisfaction depends on the intrinsic and extrinsic properties of the job and the characteristics of the individual employee (McShane & Von Glinow 2005; Varkey, Karlapudi, & Hensrud 2008).

Hence, satisfaction is defined as the individual’s overall evaluation of working for a specific organisation (Vilares & Coehlo 2000; Voisard, 2008). In other words, employee satisfaction is described as employees’ feelings or state of mind about the nature of their work and conditions of employment with a particular employer.

5. Theoretical model of employee satisfaction The literature review indicates that employee satisfaction is a concept used to describe employees’ contentment and the fulfilment of their expectations and needs, resulting from an appraisal of the various aspects of the organisation (Martins & Coetzee 2007; Aamodt 2010). Thus, it is depicted and conceptualised in the following theoretical diagnostic model as the employee’s appraisal of organisational strategy, policies and processes, and outcomes in the organisation.

Employee satisfaction

Organisational strategy • Vision and mission • Values • Leadership

Processes and policies Change management Health and safety Communication Employment equity and diversity • Human resource management • Training and development • HIV/AIDS • • • •

Outcomes • Relationships • Teamwork • Job satisfaction

Figure 1: A conceptual diagnostic model of employee satisfaction Organisational strategy includes the vision, mission, values and leadership. Eskilden and Dahlgaard’s (2000) causal model of employee satisfaction indicates that strategy and leadership are a vital element when attempting to improve employee satisfaction and are the criterion whereby management sets the agenda for the organisation’s future behaviour (Aydin & Ceylan 2008). Values also help to shape the organisation’s strategic intent and direction. In addition, research has shown that employee satisfaction has positive significant correlations with organisational leadership practices (Aydin & Ceylan 2008; Varkey et al 2008).

Employees can derive satisfaction or dissatisfaction from the policies and processes implemented by their organisation. Policies and processes include change management, health and safety, communication, employment equity and diversity, human resource management, training and development, as well as HIV/AIDS initiatives. Such policies and processes are used in the diagnosis process to determine the level of satisfaction employees derive from the work environment (Küskü 2001; Spector 2006). Literature indicates that rewards and benefits are considered to be the primary source of employee satisfaction when they are

194

Ophillia Ledimo and Nico Martins equitable and congruent with their needs and expectations (Mottaz 1987; Milner 2009; Vlosky & Aguillar 2009).

Relationships, teamwork and job satisfaction are organisational outcomes or results that affect employee satisfaction. Most employees expect their work to fulfil the need for social interaction. Studies indicate that outcomes such as teamwork and positive relationships with colleagues and supervisors lead to increased employee satisfaction (Aydin & Ceylan 2008; Milner 2009; Aamodt 2010). Job satisfaction is also a significant aspect of organisational outcomes, as it indicates the satisfaction that employees derive from their job responsibilities and duties in the organisation (Balgobind 2002; Chen et al 2006; Küskü 2001).

6. Research method Quantitative research was conducted to test and confirm the theoretical model developed in the literature study. The findings of this empirical study were finally used as the basis for developing a new model.

6.1 Research design In order to achieve the purpose of this study, a survey method was used to gather quantitative data on the participants’ responses (Welman, Kruger & Mitchel 2009:95); in order to use the survey results to apply the SEM to develop a diagnostic model for measuring employee satisfaction.

6.2 Research questionnaire The Employee Satisfaction Survey (ESS) was used to measure employee satisfaction (Martins & Coetzee 2007); it measures the construct as conceptualised in section 5. The survey is relevant for the context of this study because it is a valid and reliable measure which was developed based on several studies conducted in South Africa organisations. It consists of thirteen dimensions (see table 1) and 75 items, all of which are considered to be of equal value. The scale used is a summated rating in the form of a five‐point Likert‐type scale with (1) as strong disagreement and (5) as strong agreement.

6.3 Population and sampling The empirical study was conducted among a population consisting of employees of a parastatal utility organisation based in Gauteng province, South Africa. All employees (2650) of the organisation were invited to voluntarily participate in the survey, the sample size was 759 participants who comprised of 74, 7% males and 25,3% females. They included 2,9% senior managers, 6,9% middle managers, 20,4% supervisors and 69,8% operational staff.

6.4 Survey administration Owing to their advantages, paper‐and‐pencil and online survey methods were regarded as adequate for this study. According to Evans and Mathur (2005: 207), paper‐and‐pencil “surveys have several key strengths, including personal interaction, clear instructions, question variety, flexibility and adaptability, ability to use physical stimuli, capability to observe respondents and control over the survey environment”. In order to minimise the disadvantages of paper‐and‐pencil surveys, such as geographic limitations and incomplete surveys, facilitators (industrial psychologists) were used to facilitate the group sessions.

6.5 Statistical analysis Castro and Martins (2010:7) state “research by the HSRC indicates that an average of 3.2 is a good guideline to distinguish between positive and potential negative perceptions”. Therefore, the researcher decided to use the recommended cut‐off score of 3.2 as a guideline to differentiate potential positive and negative perceptions. This implies a mean score above 3.2 indicate satisfaction while a mean score below 3.2 indicates dissatisfaction.

6.5.1 Descriptive analysis The participants obtained significantly higher mean scores, indicating satisfaction in most of the ESS dimensions. Although the participants scored high on vision and mission (m = 3,38), the researchers could infer

195

Ophillia Ledimo and Nico Martins the participants seemed to be more satisfied with dimensions health and safety ( 3,22), HIV/AIDS ( 3,90), teamwork (3,29), relationships ( 3,21) and job satisfaction (3,37).

Table 1: Reliability and mean scores of the ESS dimensions ESS dimensions

Cronbach’s alpha coefficient

Number of items

Mean

Standard deviation

Vision and mission

0,84

4

3,38

1,16

Values

0,84

5

3,13

1,10

Leadership

0,88

7

3,13

1,11

Change management

0,89

7

2,69

1,10

Health and safety

0,81

3

3,22

1,17

Employment equity and diversity

0,74

5

2,53

0,96

Human resource management Training

0,72

4

2,69

1,05

0,78

4

3,00

1,12

HIV/AIDS

0,66

3

3,90

0,96

Communication

0,84

11

2,86

0,89

Teamwork

0,70

3

3,29

1,15

Relationships

0,83

8

3,21

0,91

Job satisfaction Total

0,89 0,96

9 74

3,37 3,22

1,05 1,02

As indicated in Table 1, the Cronbach’s alpha coefficient values were high for all ESS dimensions.

6.5.2 SEM SEM was used to investigate the theoretical employee satisfaction model, focussing on manifest and latent variables. In this study, manifest variables refer to observed variables measured as dimensions of ESS (see table 1). While, latent variables are not directly observed but are rather inferred based on theory, namely organisational strategy, policies and procedures, and outcomes (see figure 1).

6.6 Confirmatory factor analysis Table 2 indicates the results of the correlation matrix of manifest and latent variables. An analysis was conducted using AMOS (version 18.0) to determine validity of the three latent variables of the model. The manifest variables had moderate to high loadings in the expected latent variables and were all statistically significant. According to Chin (1998), the recommended standardised path is at least 0,20, or above 0,30, for variables in order to be considered for interpretation. Although there were low loadings of 0,35 (health and safety with processes and policies), 0,21 (vision and mission with organisational strategy) and 0,14 (HIV/AIDS with policies and procedures), overall the coefficient paths from the latent variables to the manifest variables were positive , indicating a satisfactory fit of above 0,40 for most of the manifest variables.

Table 3 gives the results of the confirmatory factor analysis of latent variables. The path coefficients of latent variables organisational strategy, policies and processes, and outcomes indicated a satisfactory fit for all these variables. All path coefficients leading from one latent variable to another were positive and all moderate to high, ranging from 0,919 to 1,059.

196

Ophillia Ledimo and Nico Martins Table 2: Results of the correlation matrix of the manifest and latent variables Manifest variables Vision and mission

Organisational strategy 0,21

Processes and policies 0,00

Outcomes

Values

0,42

0,00

0,00

Leadership

0,63

0,00

0,00

Change management

0,00

0,59

0,00

Health and safety

0,00

0,35

0,00

Employment equity and diversity

0,00

0,51

0,00

Human resource management

0,00

0,52

0,00

Training

0,00

0,44

0,00

HIV/AIDS

0,00

0,14

0,00

Communication

0,00

0,65

0,00

Teamwork

0,00

0,00

0,49

Relationships

0,00

0,00

0,55

Job satisfaction

0,00

0,00

0,67

0,00

Table 3: Results of the correlation matrix of latent variables Latent variables

Processes and policies

Outcomes

Organisational strategy

Organisational strategy ,00

Processes and policies

,996

,00

Outcomes

1,059

,919

,00

6.6.1 Evaluation of goodness of fit The evaluation of goodness of fit of the model indices is indicated in table 4.

Table 4: Goodness‐of‐fit measures for employee satisfaction diagnostic model 1 Indices Values Goodness‐of‐fit index (GFI) 0,87 Chi‐square (CMI) 625,02 P‐value of close fit 0,000 Root mean square residual (RMR) 2,07 Root mean square error of approximation (RMSEA) 0,10 Normed fit index (NFI) 0,88 Incremental fit index (IFI) 0,89 Comparative fit index (CFI) 0,89

Note: GFI; NFI, IFI and CFI above 0.90 signify good fit

The various measures of model fit, which included absolute and incremental fit measures, were used to assess the model. Absolute measures are chi‐square, P‐value and GFI, while incremental measures include NFI, CFI and IFI. A SEM test of employee satisfaction diagnostic model 1 all measures indicated a weak fit. Based on the above results of fit measures, the first model was rejected. On the basis of the data in this research, this model did not meet the required goodness of fit for both incremental and absolute measures used. The next section discusses the modification procedures conducted in this study.

197

Ophillia Ledimo and Nico Martins 6.6.2 Modification of the model The modification index from AMOS was applied to look for improvements and indices were used with caution because, according to (Bollen 1989), a parameter link should only be relaxed if it can be interpreted substantially regarding the direction of impact, as well as the sign of the parameter. In this study, modification indices suggested that the constraints on the error terms for manifest variables should be relaxed in order to obtain a better fit of the SEM, as depicted in figure 2.

Direct causal relationship = Correlations between variables = Error between actual and predicted value = Figure 2: Employee satisfaction diagnostic model The following indices made sense and would improve the overall fit of the model:

The first pair of manifest variables with which error terms should be allowed to correlate is communication with values, vision and mission. Communication includes disseminating and sharing information in the organisation on its values, mission and vision statement (Steyn 2003). Fairhurst,

198

Ophillia Ledimo and Nico Martins Jordan, and Neuwirth (1997: 243), argue “mission statements are seen often in company newsletters, speeches, annual reports, brochures, and posters”. Because the organisational values, mission and vision are mainly aspects or contents of the communication process, it is clear that these manifest variables are related and their error terms allowed correlation.

The second pair of manifest variables that the modification indices suggested should be allowed to correlate was job satisfaction and HIV/AIDS. Voisard (2008) suggest job satisfaction entails employees’ positive appraisal of their jobs based on the conditions of their working environment or organisation.

Employees are either infected with or affected by HIV/AIDS; hence organisations need to initiate interventions to positively impact individuals, families, organisations, businesses and the economy (Matlala 1999). The results of the study conducted by Ho (1997: 187) indicate that wellness programmes have a positive impact on employee outcomes such as job satisfaction and high morale. Since these variables are positively related, the constraints have been relaxed.

The results of the goodness of fit measures for the second employee satisfaction diagnostic model are presented below, based on the modification (see table 5).

Table 5: Goodness of fit measures for employee satisfaction diagnostic model 2 Indices Values Goodness of Fit Index (GFI) 0,905 Chi‐Square (CMI) 471,869 P‐Value of close fit 0,000 Root mean square residual (RMR) 1,765 Root mean square error of approximation (RMSEA) 0,096 Normed Fit Index (NFI) 0,910 Incremental Fit Index (IFI) 0,920 Comparative Fit Index (CFI) 0,920

Note: GFI; NFI, IFI and CFI above 0.90 signify good fit

A SEM test of employee satisfaction diagnostic model 2 indicated both an incremental and absolute goodness‐ of‐fit, based on the following measures (see table 5):

In terms of GFI, the value of 0.905 met the required value of 0.90, which suggests good model fit.

NFI demonstrated a good model fit with the value of 0.910, which met the minimum requirement value of 0.90 for model fit and this improves model fit by 91%.

IFI value of 0.920 is above the required minimum value of 0.90 which indicates an acceptable model fit.

CFI value of 0.920 is also above the cut‐off point of 0.90, which indicates an acceptable model fit.

7. Comparison of the theoretical model and the SEM model A comparison of the SEM model (Figure 2) and theoretical model revealed three similarities. Firstly, the domain of organisational strategy has direct causal relationship with dimensions vision and mission, values and leadership. These confirm literature findings that organisational strategy is the criterion according to which management sets the agenda for the future behaviour of organisations, and is one of the determinants of employee satisfaction (Chen et al. 2006; Eskildsen & Dahlgaard 2000). Secondly, the domain policies and processes has direct causal relationship with dimensions change management, health and safety, employment equity and diversity, communication, human resource management, training and HIV/AIDS. The literature also suggests policies and processes are used in diagnostic processes to determine the level of satisfaction employees derive from various aspects of an organisation (Küskü 2001; Matlala 1999). Thirdly, the domain of outcomes has direct causal relationship with dimensions relationships, teamwork and job satisfaction. The literature indicates that teamwork and positive relationships with colleagues and supervisors enhance employee satisfaction (Milner 2009; Aamodt 2010).

In addition to the above similarities, the SEM model indicated two new correlations between the latent variables. The first relationship was the correlation between communication with organisational values, vision and mission statement, and the second correlation between dimensions job satisfaction and HIV/AIDS.

199

Ophillia Ledimo and Nico Martins

8. Conclusions The use of structural equation modelling enabled the researchers to propose and confirm a diagnostic model of employee satisfaction. The domains of this model should enable organisations to identify developmental areas based on employees’ dissatisfaction or areas of strengths based on employees’ satisfaction with the dimensions. The diagnostic model will also enable organisations and practitioners to initiative interventions aimed at addressing areas of dissatisfaction and to leverage on the areas of satisfaction in the organisation.

Then, organisations will be able to realise the benefits of employee satisfaction which are employee retention, customer satisfaction, improved quality, enhanced performance, life satisfaction, employee health and wellbeing.

This paper encourages the use of structural equation modelling by researchers as an important statistical procedure in theory development, in order to generate new models that will assist managers and practitioners to deal with organisational challenges.

References Aamodt, M. G. (2010) Industrial/organizational psychology: An applied approach (6th ed.), Cengage Learning, Belmont, CA. Arnold, H. J. and Feldman, D. C. (1986) Organisational behavior, McGraw‐Hill, New York.

Aydin, B. and Ceylan, A. (2008) “The employee satisfaction in metalworking manufacturing: How do organizational culture and organizational learning capacity jointly affect it?”, Journal of Industrial Engineering Management, Vol. 1, No. 2, pp 143–168.

Aydin, B. and Ceylan, A. (2009) “A research analysis on employee satisfaction in terms of organizational culture and spiritual leadership”, International Journal of Business and Management, Vol. 4, No.3, pp 159–168.

Balgobind, V. (2002) The impact of transformational leadership on subordinate job satisfaction. Unpublished master’s dissertation, University of South Africa, Pretoria.

Bentler, P. M. and Bonnet, D. G. (1980) “Significance tests and goodness of fit in the analysis of covariance structures”, Psychological Bulletin, Vol. 88, pp 588–606.

Bollen, K. A. (1989) Structural equations with latent variables, Wiley, New York.

Chen, S., Yang, C., Shiau, J. and Wang, H. (2006) “The development of an employee satisfaction model for higher education”, The TQM Magazine, Vol. 18, No.5, pp 484–500.

Chin, W. W. (1998) “Issues and opinions on structural equation modelling”, MIS Quarterly, Vol. 22, No.1, pp 716.

Castro, M. L. & Martins, N. (2010) “The relationship between organisational climate and employee satisfaction in a South African information and technology organisation”, South African Journal of Industrial Psychology, Vol 36, No.1, 1– 5.

Eskildsen, J. K. and Dahlgaard, J. J. (2000) “A causal model for employee satisfaction”, Total Quality Management and Business Excellence, Vol. 11, No. 8, pp 1081–1094.

Evans, J.R. and Mathur, A. (2005) “The value of online surveys”, Internet Research, Vol. 15, No. 2, pp 195–219.

Fairhurst, G. T., Jordan, J. M. and Neuwirth, K. (1997) “Why are we here? Managing the meaning of an organizational mission statement”, Journal of Applied Communication Research, Vol. 25, pp 243–263.

Fosam, B. E, Grimsley, F. J. M. and Wisher, J. S. (1998) “Exploring models for employee satisfaction with particular reference to a police force”, Total Quality Management, Vol 9, pp 235–247.

Hackman, J. R. and Oldham, G. R. (1976) Motivation through the design of work: Test of a theory, Organizational Behavior and Human Performance, Vol. 16, pp 250–279. th Hair, J. J, Anderson, E. R, Tatham, L. R. and Black, C. W. (1998) Multivariate data analysis (5 ed), Prentice‐Hall, Upper Saddle River, New Jersey. th Hair, J. J, Anderson, E. R, Tatham, L. R. And Black, C. W. (2005) Multivariate data analysis (6 ed), Prentice‐Hall, Upper Saddle River, New Jersey.

Ho, J. T. S. (1997) “Corporate wellness programmes in Singapore: Effect on stress, satisfaction and absenteeism”, Journal of Managerial Psychology, Vol. 12, No. 3, pp 177–189.

Hu, L. T. and Bentler, P. M. (1998) “Cut‐off criteria for fit indexes for covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modelling”, A Multidisciplinary Journal, Vol 6, pp 1–55.

Küskü, F. (2001) “Dimensions of employee satisfaction: A state university example”, METU Studies in Development, Vol. 28, No.3/4, pp 399–430.

Küskü, F. (2003) “Employee satisfaction in higher education: The case of academic and administrative staff in Turkey”, Career Development International, Vol. 8, No. 7, pp 347–356.

Locke, E. A. (1976) “The nature and causes of job satisfaction. In D. M. Dunnette (Ed.). Handbook of industrial and organizational psychology (pp. 297–349), Rand McNally, Chicago.

Lofquist, L. H. and Dawis, R.V. (1969) Adjustment to work: A psychological view of man’s problems in a work‐oriented society, Appleton Century Crofts, New York.

Maltser, K., Fuchs, M. and Schubert, A. K. (2004) “Employee satisfaction: Does Kano’s model apply?” Total Quality Management, 15 (19), pp 1179–198.

200

Ophillia Ledimo and Nico Martins Martins, N. and Coetzee, M. (2007) “Organisational culture, employee satisfaction, perceived leader emotional competency and personality type: An exploratory study in a South African engineering company”, South African Journal of Human Resource Management, Vol. 5, No. 2, pp 20–32.

Matlala, S. (1999) “Prioritising health promotion and employee wellness”, People Dynamics, Vol. 8, No. 17, pp 22–25.

McShane, L. S. and Von Glinow, A. M. (2005) Organizational behaviour (3rd ed.), McGraw‐Hill Irwin Boston.

Milner, K. (2009) Attitudes and job satisfaction, In S. P. Robbins, T. A. Judge, A. Odendaal and G. Roodt (Eds.), Organizational behaviour: Global and South African perspective (pp 65–84), Pearson Education, Cape Town.

Mottas, C. J. (1987) “Age and work satisfaction”, Work and Occupations, Vol 14, No. 3, pp 387– 409.

O’Neil, M. (2005) “Employee care, a vital antecedent to customer care in the health care industry: An exploratory investigation of the employee satisfaction construct at North East Alabama Regional Medical Center”, International Journal of Health Care Quality Assurance, Vol. 18, No. 2, pp 131–151.

Odendaal, A. And Roodt, G. (2009) What is organizational behaviour?, In S. P. Robbins, T. A. Judge, A. Odendaal and G. Roodt (Eds.), Organizational behaviour: Global and Southern African perspective (pp. 3–25), Pearson Education, Cape Town.

Porter, L. W., Lawler, E. E. and Hackman, J. R. (1975) Behaviour in organizations, McGraw‐Hill, New York.

Rad, A. M. M. and Yarmohammadian, M. H. (2006) “A Study of relationship between managers’ leadership styles and employee’s job satisfaction”, Leadership in Health Service, Vol. 19, No. 2, pp 11– 28. th Salkind, J. N. (2009) Exploring research (7 ed.), Pearson Education International, Upper Saddle River, New Jersey.

Slater, R. and Mayne, P. (2000) Strategic organization development, In M. Meyer, & E. Botha (Eds.). Organizational development and transformation in South Africa (pp. 37–65), Butterworths, Durban. th Spector, P. E. (2006) Industrial and organizational psychology: Research and practice (4 ed.), Wiley, Englewood Cliffs, New Jersey.

Steyn, B. (2003) ‘From strategy to corporate communication strategy: A conceptualization” Journal of Communication Management, Vol. 8, No. 2, pp 168–183.

Varkey, P., Karlapudi, S. and Hensrud, D. (2008) “The impact of a quality improvement program on employee satisfaction in an academic microsystem”, American Journal of Medical Quality, Vol. 23, No. 3, pp 215–221.

Vilares, M. A. and Coelho, P. S. (2000) “The employee‐customer satisfaction chain in the ECSI model”, European Journal of Marketing, Vol 37, No. 11/12, pp 1703–1722.

Vlosky, R. P. And Aguillar, F. X. (2009) A model of employee satisfaction: Gender differences in cooperative extension.

Journal of Extension, Vol 47, No. 2, pp 1–15.

Voisard, V. (2008) “Employee empowerment and employee satisfaction in the workplace”, California Sociology Journal, Vol 1, pp 1–17.

Welman, C., Kruger, F. and Mitchell, B. (2009) Research methodology, Oxford University Press, Cape Town.

201

Integrated Evaluation Model Strengthening Effective Management of Strategic Research Programmes Ludmiła Łopacińska and Adam Mazurkiewicz Institute for Sustainable Technologies – National Research Institute, Radom, Poland ludmila.lopacinska@itee.radom.pl adam.mazurkiewicz@itee.radom.pl

Abstract: In order to ensure the high level of competitiveness of the national economy, many initiatives are undertaken, among which strategic research programmes can be distinguished. Strategic research programmes are characterised by a great number of levels (e.g. levels of research projects, project groups, thematic groups, and strategic research programme as a whole), a multitude of objectives and a high budget; therefore, it is necessary to systematically evaluate them. The paper concerns an integrated evaluation model within which factual, financial and organisational aspects are taken into account. The model is dedicated to strategic research programmes in the area of technical solutions and can be implemented at their different levels. The authors present the concept and the structure of the model and focus particular attention to the crucial components used to create a model. This includes the integrated evaluation methodology, the inclusion of standard methods (e.g. interviews, document analysis or workshops), which can be used in the assessment of any individual research projects, and above all, the non‐standard methods supporting a complex assessment of technical solutions with the implementation maturity, the commercial potential and the innovativeness taken into consideration.

Selected procedures enhancing the evaluation process are also shown. Moreover, differences between the proposed model and the models identified in publications and the original features of the model are indicated. The designed model can be applied at different moments of the realisation of a programme (before the start – ex‐ante, during the realisation – on‐going, at the end of the realisation – ex‐post and some years after the termination of the programme – follow‐up).

Therefore, the model offers different procedures and actions to be taken during each phase of the evaluation process and allows simulations that can indicate what needs to be done for the efficiency of a strategic programme to be ensured.

These procedures are supported by the design of possible essential questions, criteria, methods, indicators and tools, whose application seems to be useful from the point of view of the evaluation process. Out of these procedures, particular attention is paid to the research method supporting the assessment of the commercial potential of technical solutions developed in strategic research programmes.

Keywords: evaluation, strategic management, commercial potential, strategic research programme

1. Introduction Significant changes in the structures of industry and the economy are coming due to the globalisation of the market, the destabilisation and turbulence of the environment, and the growing dynamics of the innovation processes. These anticipated changes have contributed to the design and the current development of the strategic management (Murawska 2008). The issue of strategic management is presented by some authors (Chandler 1962; Ansoff 1985; Drucker 2002; Ackoff 2002; Andrews 1980; Porter 1980). Based on a review of the literature, we can say that the main objective of strategic management is to undertake operations in a long‐term perspective, which ensure the permanence of the organisation. This type of management is easier and more efficient to realise when developing a strategy. This strategy identifies the long‐term initiatives for an organisation, e.g. strategic research programmes, which are characterised by a complex organisational structure comprising several levels of the programme’s realisation, e.g. a research project level, a project group level, a thematic group level, and a strategic programme level (Łopacińska 2012) (see Figure 1).

In order to ensure the maximum effectiveness and efficiency of the realisation of strategic research programmes, it is necessary to conduct their systematic evaluation.

2. Literature review Having analysed the state‐of‐the‐art on the evaluation of research programmes, including those of a strategic character, the authors identified several evaluation models differing in terms of their concentration. These are listed as follows:

The evaluation process (Tyler 1942; Lindquist 1951; Glaser and Strauss 1967; Eisner 1975; Sanders and Horn 1994; Kee 1995);

The programme management (Cronbach 1963; Cook 1966; Lessinger 1970);

202

Ludmiła Łopacińska and Adam Mazurkiewicz

The participation of beneficiaries in the programme evaluation – “social models” (Stake 1967; Scriven 1974; Patton 1997; House 1998; Clancy and Horner 1999; Stufflebeam 2000).

Source: Łopacińska, L. (2012) Evaluation model for strategic research programmes in the area of technical solutions (doctoral thesis under development) Figure 1: Structure of a strategic research programme Evaluation process oriented models mainly focus on the formal aspects of a programme, e.g. the verification of whether or not the objectives assumed at the beginning of the programme are achieved, whether the results are obtained, and whether the budget is spent according to the assumptions. Programme management oriented models particularly concern the development of procedures enabling the management of strategic research programmes; whereas, social models assume the inclusion of beneficiaries in the evaluation of a programme.

The models identified in publications are not, however, conformable to the common definition of the model.

According to R. Ackoff (1969), “a model is a representation of a state, object or event. It is an idealised concept as it is not as complex as the reality is and it is much easier to be applied for scientific investigations.” The models analysed are rather kinds of evaluation approaches, which mainly indicate the main objective of the evaluation and the users of the evaluation results.

3. Evaluation model for strategic research programmes in the area of technical solutions – theoretical discussion The evaluation of strategic research programmes is an intricate process. A very important factor in the process is the duration of time, which causes changes in the validity of tasks which were planned at the launch of the programme, not only from the executors’ side, but mainly from the point of view of the final users of the results.

In this case, the following problems can appear:

What steps should be undertaken to remedy the situation in the programme?

What methods can be used to verify the effectiveness of the realisation of a programme?

What criteria should be considered in order to assess individual research projects and the entire strategic research programme?

How can the efficiency of the realisation of a programme be improved?

There is no single answer to all of these questions. In order to ensure an effective and efficient management of strategic research programmes, there is a strong need to use a tool that would help to solve all, or at least many of such problems. The literature review and a high possibility of the occurrence of such problems were the inspiration for the design of the advanced evaluation model for strategic research programmes.

203

Ludmiła Łopacińska and Adam Mazurkiewicz The assumptions of the proposed model concern the following: the modelled object (1), the source of information (2), and the operational structure (3, 4, 5).

1. The model was developed to meet the needs of the strategic research programme of a multilevel character. The structure of the strategic programme is always composed of at least two levels: the level of research projects, within which research tasks are realised, and the level of the strategic programme as a whole. Depending on its size and complexity, a strategic research programme can also be built of intermediate levels including, inter alia, thematic groups or research project groups. The model is directed to the assessment of technical solutions (materials, systems, and technologies).

2. The information source of the model is constituted by a database and a knowledge base supporting the process of the evaluation of a strategic research programme. The information support is constituted by an expert system used for the collection, processing, and analysis of data obtained from the evaluation of a strategic research programme.

3. Standard and non‐standard evaluation methods (Łopacińska 2011), particularly methods for the evaluation of the technologies’ technological readiness, commercial potential, and innovativeness are incorporated into the model. The programme is evaluated by internal experts representing the R&D institution realising the programme, and external experts from business and scientific environments specialising in the research area covered in the programme.

4. The model comprises the evaluation methodology, which can be adjusted to any level of the strategic research programme.

5. The evaluation process is composed of two stages: formal and substantial. In the formal evaluation, the organisational aspects, like the verification of the advancement of tasks, or the control over planned expenditure, are assessed; whereas, in the substantial evaluation, the emphasis is put on, e.g. social or implementational aspects, depending on the requirements of the evaluation ordering party.

6. The developed model evaluation system has an open structure, which allows for the inclusion of additional elements, which might be crucial for the verification process or the practical application of the model.

The designed model is a “decision model,” and it is mainly directed at the assessment of values of a programme. The model includes the elements of the existing models, such as the decision model (Cronbach 1963), management information systems (Cook 1966), and the connoisseurship model (Eisner 1975). However, the designed model is of a complex character, and it is adjusted to the specificity of a strategic programme, with a complex organisational structure and a factual scope of the programme taken into consideration. The evaluation proposed is a contrast for the feed‐back evaluation (Royse et al. 2009), which is very similar to monitoring (Gudda 2011). This kind of evaluation enables one to indicate if the planned goals were achieved, without changes in the economy taken into account. Figure 2 presents the structure of the model, including the evaluation methodology (1), research methods (2), procedures supporting the evaluation process (3) and assessment tools (4).

The model considers a set of factors influencing the need for the evaluation, i.e. the necessity of the consideration of the results of the evaluation of a current strategic research programme, the directions of the foresight projects, and the external conditions (e.g. changes of legal rules). These factors often influence the necessity for the evaluation to be conducted.

The model comprises the evaluation methodology, which is a complex process including both standard and non‐standard methods. The non‐standard methods include the methods for the assessment of the following: technological readiness, innovativeness, and a commercial potential of technical solutions. The methodology also comprises a set of procedures including the following:

1. The procedure for the selection of evaluation stages for individual levels of a strategic research programme considering its multilevel structure; 2. The procedure for the estimation of evaluation characteristics; 3. The procedure for the evaluation of research projects; 4. The procedure for the assessment of technologies developed within a research programme considering, i.e. their technological readiness, commercial potential, or the level of their innovativeness; and,

204

Ludmiła Łopacińska and Adam Mazurkiewicz 5.

The procedure for the generation of future research directions taking into account the results of strategic programmes, national and international scientific strategies, and the results of foresight projects, all deciding upon the launch of new strategic research programmes.

Source: Authors Figure 2: Evaluation model for strategic research programmes As and example, let us present the procedure for the evaluation of research projects (3), which enables the assessment of the research projects from different perspectives, i.e. economic, technological, social, and legal.

The proposed complex evaluation approach contains two types of evaluation: formal (1st step of evaluation) and substantial (2nd step of evaluation). The basic aim of the formal evaluation is to assess the organisational aspects of a research project: goals, results, budget and management styles (e.g. the coordination style of the projects, forms of communication between research teams, etc.) (Łopacińska 2012). The aim of the substantial evaluation is to assess the projects with the aspects crucial for the final users. The basic assumption of the approach is that the project must be positively assessed both from formal and substantial perspectives.

However, there are exceptions to the rule, which take place in the on‐going evaluation, where despite the positive outcome of the formal evaluation, the final evaluation result can be negative, i.e. the project is realised correctly from the formal point of view, but changes in national science policy were introduced. In such a case, the authors decided that the final result of the evaluation must be negative, because the most important aspect of the realisation of a strategic research programme is to design the products which are needed on the market (see Figure 3, Example 2).

On the other hand, the methodology assumes that some elements of the formal evaluation can be negatively assessed and, in spite of that, the final result of the evaluation can be positive. This is the case for the example when the project may be essential from the point of view of the economy, but due to some difficulties (e.g. complicated procedures of tenders), the budget is not spent as it was planned, or not all objectives are already achieved (see Figure 4).

205

Ludmiła Łopacińska and Adam Mazurkiewicz

Source: Authors Figure 3: Examples of the evaluation of a research project, when the formal evaluation is positive

Source: Authors Figure 4: Examples of the evaluation of a research project, when the formal evaluation is negative

206

Ludmiła Łopacińska and Adam Mazurkiewicz In order to assess the projects in detail from the factual point of view, the authors propose a set of indicators to be used. The selection of indicators depends on what aspects are the most important in the evaluation for the institution commissioning the evaluation. For instance, if the commercial potential aspect is dominant (as presented in Figure 3), the authors designed a set of indicators that should be considered in the evaluation.

There are also sets of indicators designed for the assessment of other aspects, e.g. the innovativeness or the implementation maturity.

This methodological approach is a tool for supporting the decision making process on the launch, the continuation, or the termination of research projects. Its main objective is to take such steps that would ensure the highest effectiveness of the realisation of a strategic research programme.

As stated above, the designed evaluation methodology comprises a set of methods that enable the assessment of technical products from the perspective of their implementation maturity, commercial potential, and innovativeness. The methods of the implementation maturity (Mankins 1995; Defence Research and Development Canada 2006, Mazurkiewicz et al. 2010) and the innovativeness assessment (Junkar 2001; Schmeisser 2009) are commonly known, and the authors have now designed a method for commercial potential assessment.

The literature presents few examples of the commercial potential assessment methods that are applicable for the evaluation of individual research projects (Gwarda‐Gruszczyńska 2010; Lucheng et al. 2007; Hongjun et al.

2007). They enable quick or in‐depth assessments of a new solution and the indication of its possible marketability. The commercial potential method presented in this paper was developed particularly for the needs of the evaluation of technical innovations (Belina et al. 2012). It enables a detailed assessment of the commercial potential at different levels of the development of a technical solution from its concept to its commercialisation and practical implementation.

The development of the method was a three‐stage procedure. In the first phase, the authors chose the areas of assessment with the use of the expert panel method. There were formal and social requirements designed in order to select the most appropriate experts. In regards to formal requirements, the expert should have a great knowledge in a given domain, be recognised as an expert in the eyes of other experts, and have experience in the design of the documentation. The social requirements were very numerous. The expert should have a holistic look at the problem, be independent in judgements and opinions, have communication skills, and be able to take responsible actions. The experts were selected both from academic and industrial areas.

Four areas were proposed: technological; market; economic, and legal and/or organisational. Within the technological area, the authors proposed a set of criteria concerning the technical and technological aspects adequate to the solution, i.e. the level of the technological advancement of a solution, the level of the originality of a solution in comparison to solutions already existing on the market, the operational safety, the level of environmental threat, etc. The market area includes criteria enabling the assessment of the market demand for a solution, the type of production (mass, single unit), and the potential area of application of a solution. The assessment of the economic area concerns, e.g. the amount of funds needed for the development of a solution, the cost‐effectiveness of its production, the dissemination costs compared with market costs, the price of a proposed solution in relation to prices of the existing products, and the levels of income and profits gained by beneficiaries from the application of a given product. Finally, the legal & organisational area includes, but not limited to, the following criteria: the IPR protection or the possibilities of product imitation and the dependency of its implementation on changes in legal rules. For all the assessment areas, 40 criteria were proposed. In the second phase, the key criteria were selected with the use of the matrix of direct influences by means of the MIC‐MAC method. The MIC‐MAC method enabled the authors to classify the criteria, which are significant for the commercial potential, into the following groups: key, decisive, regulating, and autonomous. The key criteria indicate the activities that should be identified as a priority in the commercial potential of a product. The decisive criteria very strongly influence the commercial potential of the product, but they are very difficult to control. Knowledge about these criteria is very important in order to observe trends in the interest of beneficiaries regarding the products on the market. The regulating criteria include such criteria that can be helpful in the achievement of the commercial potential on the market, but they are not so significant for that process as a whole; whereas, the autonomous criteria are the least significant for the commercial potential. The result of the analysis was the selection of the key criteria that

207

Ludmiła Łopacińska and Adam Mazurkiewicz have the greatest impact on the commercial potential. The key criteria included the following, among others: the level of the market demand for a product; the potential number of beneficiaries who will be able to acquire the product; the number of areas in which the product can be used; the price of a product in comparison with the prices of similar products; and potential user’s benefits. In the next phase of the development of the method the criteria were verified by the experts. The third phase was to establish weights for both the key and the supplementary criteria. The weights were established with the use of the expert panel method. They considered the results from the MIC‐MAC method and set the weights depending on the level of the impact of factors on a commercial potential of a product. The weights were assigned numerical values between 5‐1, where the weight 5 is equivalent with the greatest role of a given criterion for a particular solution. The weight 5 was given to the criteria that belonged to the key criteria, according to the division made within the MIC‐MAC method.

It was assumed that the commercial potential would be assessed by the expert team, comprising specialists in the selected area and the authors of the methodology with the use of the original computer system.

The application of the designed method enables the verification of the level of the commercial potential for solutions at different stages of their development. Additionally, it enables a comparative analysis between outcomes of different evaluations conducted at different stages of programme realisation. This comparative analysis is of significance to product development, because it takes the time factor into consideration and helps to check whether it leads to any changes in the market opportunity of the product.

All the information gathered in the evaluation process is collected in the expert system, which is a tool that supports decisions on the launch, the continuation or the termination of either a single research project or the entire programme.

4. Conclusions The designed evaluation model is directed at the needs of research organisations. It can be a tool supporting the complex evaluation of strategic research programmes. Depending on the accepted criteria, the model can support decisions taken on the realisation of strategic programmes, i.e. the transfer funds between research projects, the termination of “non‐perspective” projects or thematic groups, or the launch of new cutting‐edge projects.

The need for the application of the proposed evaluation model for strategic research programmes is justified by the necessity to ensure the efficient management, implementation, and evaluation of a programme.

References Ackoff, R. (1969) Decyzje optymalne w badaniach stosowanych, PWN, Warsaw.

Ackoff, R. (2002) The Strategic Management of Intellectual Capital and Organizational Knowledge: A Collection of Readings, Oxford University Press, New York.

Andrews, K.R. (1980) The Concept of Corporate Strategy, R.D. Iwin, Homewood.

Ansoff, H.I. (1985) Zarządzanie Strategiczne, PWE, Warsaw.

Belina B., Łopacińska L. and W. Karsznia (2012) ”Evaluation of technical innovations with the commercial potential method taken into consideration”, Maintenance Problems Vol. 4/2012.

Chandler, A.D. (1962) Strategy and Structure. Chapters in History of Industrial Enterprise, MIT Press, Cambridge.

Clancy, T. and Horner, C. (1999) Every Man a Tiger, G. P. Putnam’s Sons, New York.

Cook, D. L. (1966) Program Evaluation and Review Techniques, Applications in Education, U.S. Office of Education Cooperative Monograph, Washington.

Cronbach, L. J. (1963) “Course Improvement Through Evaluation”, Teachers College Record, Vol. 64, pp.672‐692.

Defence Research and Development Canada (2006) A Technology Maturity Measurement System for the Department of National Defence. The TML System. Contract Report DRDC Atlantic CR 2005‐279.

Drucker, P.H. (2002) Myśli Przewodnie Druckera, MT Biznes, Warsaw.

Eisner, E. W. (1975) “The Perceptive Eye: Toward a Reformation of Educational Evaluation”. Invited address, Division B, Curriculum and Objectives, American Educational Research Association, Washington, DC. Glaser, B. G. and Strauss, A. L. (1967) The Discovery of Grounded Theory, Aldine, Chicago.

Gudda, P. (2011) Guide to Project Monitoring & Evaluation. The Author House. Bloomington.

Gwarda‐Gruszczyńska, E. (2010) “Quicklook™ jako metoda oceny potencjału komercyjnego innowacji i technologii”, Acta Universitatis Lodziensis Folia Oeconomica Vol. 234, pp. 265‐275.

Hongjun W., Xiaofei Z., Rui L., Guiwu W.: An Approach to Potential Evaluation of Emerging Technology Commercialization with Uncertain Linguistic Information. Journal of Convergence Information Technology, No 7/2007.

208

Ludmiła Łopacińska and Adam Mazurkiewicz House, E. R., and Howe, K. R. (1998) Deliberative Democratic Evaluation in Practice, University of Colorado, Boulder.

Junkar, M. and Levy, P. (2001) TECHNOSOPHY: Strategic Approaches to the Assessment and Management of manufacturing technology innovation, Kluwar Academic Publishers.

Kee, J. E. (1995) “Benefit‐Cost Analysis in Program Evaluation”, In J. S. Wholey, H. P. Hatry, & K. E. Newcomer, Handbook of practical program evaluation, pp. 456–488. San Francisco: Jossey‐Bass.

Lessinger, L. M. (1970) Every Kid a Winner: Accountability in Education. Simon and Schuster, New York.

Lindquist, E. F. (ed.). (1951). Educational measurement. Washington, DC: American Council on Education.

Lucheng H., Yafei L., Feifei W.: A Conceptual Framework of Identifying the Commercialization Potential of Emerging Technology Based on Subjective Judgment and Objective Fact. Management Science and Engineering, Portland, 2007. Łopacińska L. (2011) „Ponadstandardowe metody ewaluacji strategicznych programów badawczych”, Problemy Eksploatacji. Vol. 3/2011. Łopacińska L. (2012) “Multi‐level Evaluation Enhancing Effective Management of Strategic Research Programmes”, Maintenance Problems. Vol. 4/2012. Łopacińska L. (2012) Model ewaluacji strategicznych programów badawczych w obszarze innowacji technicznych (doctoral thesis under development).

Mankins, J. C. (1995) Technology Readiness Levels: A White Paper. Office of Space Access and Technology. NASA. Mazurkiewicz A., Karsznia W., Giesko T. and B. Belina (2010) Metodyka Oceny Stopnia Dojrzałości Wdrożeniowej Innowacji Technicznych. Problemy Eksploatacji Vol. 1/2010 (76), pp. 5‐21.

Murawska, M. (2008) Zarządzanie Strategiczne Niematerialnymi Zasobami Przedsiębiorstwa, Warsaw. rd Patton, M. Q. (1997) Utilization‐focused Evaluation: The New Century Text (3 ed.), Sage, Newbury Park.

Porter, M.E. (1980) Competitive Strategy, The Free Press, New York.

Royse, D., Thyer, B. and Padgett, D. (2006) Program Evaluation: An Introduction. Cengage Learning.

Sanders, W. L., and Horn, S. P. (1994) The Tennessee value‐added assessment system (TVAAS): Mixed model methodology in educational assessment. Journal of Personnel Evaluation in Education, 8(3).

Schmeisser, W. and Hartmann M. (2009) Innovation Performance Accounting: Financing Decisions and Risk Assessment, Springer.

Scriven, M. (1974) “Evaluation perspectives and procedures”. In W. J. Popham (ed.), Evaluation in education: Current applications. Berkeley, CA: McCutchan.

Stake, R. E. (1967) “The countenance of educational evaluation”, Teachers College Record, 68, pp. 523–540 Stufflebeam, D.L. (2000) Evaluation Models: Viewpoints on Education and Human Services Evaluation. Massachusetts.

Tyler, R. W. (1942) “General statement on evaluation”, Journal of Educational Research, Vol. 35, pp. 492‐501.

209

What They did and What They Said? – An Observed Experience Arminda Lopes Instituto Politécnico de Castelo Branco, Castelo Branco, Portugal aglopes@ipcb.pt

Abstract: The paper aims to present the researcher’s experience in a design environment, by observing specific aspects and behaviours along with recording events. To discover what people and groups conducted, and to monitor their dialogue and behaviour, preserving the context in which things were said and done. The research work was based on case studies where interdisciplinary art and technology teams design interactive artworks. The selected methodology was a combination of Grounded Theory Methodology and Discourse Analysis. The main contribution of this paper was the story telling approach presented, by the researcher, to expose her experience on conducting observation methods: participant and non‐participant and structured and unstructured observations, mixed at the same studies. Conversely, it gave the opportunity to describe and interpret the artefacts/artworks using the participant and non‐participant roles altogether.

Keywords: participant observation, non‐participant observation, case studies, grounded theory methodology, discourse analysis, artefacts/artworks

1. Introduction Observations were used to study a dynamic situation of which was important to preserve the interrelationship of the person and the situation. It is a suitable method for working with people for long periods of time in their natural settings. Darlington et al., (2002:74) consider observation as a very effective way of finding out what people do in particular contexts, the routines and interactional patterns of their everyday lives.

The literature refers two types of observation: participant observation discussing how to enter the context, the role of the researcher as a participant, the data collection and analysis. Participant observation is a demanding method. The researcher became participant in the culture or context being observed in order that the observations are of natural phenomenon. Dawson (2006:32) argues that in participant observation, the action is deliberate and intended to add to knowledge.

Conversely, non‐participant observation is distinguished from participant observation in different ways, as will be explained in next section; however, the key difference is that, the observer does not become a participant in the context. The researcher needs to have enough distance from the other participants to preserve the reality’s objectivity being observed. The amount of data gathered is often in a considerable quantity which brings some difficulties in the analysis if a previous strategy has not been defined in advance.

In this research, an informal strategy was used as a main technique to let the findings arose by observing specific aspects and behaviours and recording events, to discover what the individuals and teams did, and to monitor their dialogue and behaviour preserving the context in which things were said and done. A narrative format was employed. The narrative was organised around designers, multidisciplinary teams, doing an artwork.

Verbal and non‐verbal behaviours were analysed in order to reflect, interpret, give meaning to the situation in terms of the cultural values, beliefs, and traditions presented, which contributed to understand and explain the situation in analysis. To observe behaviour, the focus was on answering the questions: why do people behave the way they do? What causes different people to react differently to the same situation?

2. Observation Observation is a data collection technique within qualitative research methods. Firstly, the researcher needs to decide what he/she wants to observe and the reasons why observation will be the best method to use considering the results that he/she wants to reach.

There are two main ways in which researchers observe – direct observation and participant observation. The former involves the observation of a ‘subject’ in a certain situation.” (Dawson 2006:32). In non‐participant observation, the researcher does not get involved in the activities of the group but remain a passive observer, watching and listening to its activities and drawing conclusions from this. Iacono et al.(2009) used the term

210

Arminda Lopes direct observation, which suggests a more detached perspective. Direct observation tends to be more focused than participant observation. The researcher is observing situations or people rather than trying to become immersed in the entire context. This type of observation allows looking especially for the nonverbal cues being used.

Participant observation‘s type characterizes most ethnographic research. It combined participation in the lives of the people under study whilst maintaining a professional distance that allows adequate observation and recording of data. In participant observation the researcher participate in the activities of the group being observed in the same manner as its members, with or without their knowing that they are being observed (Kumar 2011:141). This type is a field strategy that simultaneously combines document analysis, interviewing of respondents and informants, direct participation and observation, and introspection. (Denzin 1989:157)

Observation can be structured or unstructured. Each approach has advantages and also disadvantages. In the former, the researcher needs to prepare the observation schedule, to request permission to observe, and to consider what he/she will do with the information, among other information’s organisation. In the latter, the researcher has already a clear idea of the purpose of the observation; he only needs to obtain the details (Babbie, 2010).

This method is commonly used within qualitative research in different areas and situations. For example, Shinohara and Tenenberg (2007) conducted an in‐depth examination of a blind person’s use of assistive technologies. They used a series of semi‐structured interviews to collect the observations that form the basis of the case study.

3. Research methodology and methods The research work was the study of interdisciplinary art and technology teams designing interactive artworks.

Teams were composed of participants with various professional backgrounds, for example researchers from the fields of art, design, IT, computer science, engineering, architecture, cultural studies, and media studies.

There were about twenty five institutions and about 55 artists and technologists that took part in the study.

The methodology selected was a combination of: Grounded Theory Methodology (GTM), and Discourse Analysis (DA), some of each of the principles were applied during data collection and data analysis.

3.1 Case study Case studies require multiple data collection methods, whose results hopefully converge, in order to establish construct validity. Yin (1984) and Stake (1995) identify these methods as: direct observation of activities and phenomena and their environment; indirect observation; interviews ‐ structured or unstructured; documentation, such as written, printed or electronic information; records and charts about previous use of technology relevant to the case.

Case studies served to investigate how to answer our specific research questions. The central concern of the case study method was the collection and study of multiple forms of evidence, in some detail, to achieve our understanding of this research. Different sub‐methods: interviews, observations, document and record analysis, and physical artefacts such as photographic records, sketches, mock‐ups, materials used prototypes and the final artwork were considered and analysed. Of these, observations were the method that required most developmental effort during this research.

Despite the Case Study method being a descriptive rather than explanatory method, the opportunity to the researcher’s participation in those cases permitted to obtain a detailed contextual view of design practice and to capture an in‐depth descriptive record of the interactions in multidisciplinary groups.

3.2 Data collection and analysis Concerning conversations, data was gathered about the “actual” words of people and their conversations were reproduced to the best of researcher’s ability from the transcripts and participant observations. A video‐ recorder to capture the data was used. The researcher attempted to preserve the context in which things were

211

Arminda Lopes said and done. A narrative format was employed. The narrative was organised around designers, multidisciplinary teams, doing an artwork.

Following a GTM approach all the data was collected and dated. It was organised by subject order and coded.

“Coding means naming segments of data with a label that simultaneous categorises, summarises and accounts for each piece of data” (Charmaz 2006:43). Then, the reduction of data into concepts that were designated to stand for categories was very complex. The categories were developed and integrated into a theory (Corbin, 1986). This process was achieved by coding data, writing memos, and diagramming. Code notes, memos and diagrams became progressively more detailed and sophisticated as the analysis moved through the two types of coding.

Discourse analysis was applied and since it is basically an interpretative and deconstructing reading, there weren’t specific guidelines to follow. Features like turn taking, adjacency pairs and repairs were described by collecting more data using the constant comparative method employed by GTM.

The next analytical choice was centred on local meanings, such as the meaning of words, the structures of propositions, and coherence and other relations between propositions. Local meanings were the result of the selection made by the researcher in her mental models of events and designers’ shared beliefs. Attitudes were considered as socially shared opinions between designers about the design process.

4. Reported observations Data was gathered within different environments: competitions, workshops, and teams’ meetings. A summary description of an example of each event is following.

The researcher took part of each team’s meeting to analyse, observe their behaviour and maintain conversation in order to discover the interpretation and communication of the artefacts/artworks. Each research team from the design networks produced an artefact, and the process of designing these artefacts was recorded and observed to discover how multidisciplinary teams of people worked.

The researcher used an informal strategy to begin observations. The technique used, which is the most common, was judgement sampling – that is, the researcher relied on their judgement to select the most appropriate members of the sub culture on the research question. There were two approaches to this decision. Firstly, those who were not to study were chosen, secondary those who were to study were selected.

This approach was quite natural, requiring asking very simple, direct questions about what people did.

As a participant observer the researcher presented herself, who she was, where she was from, what she was trying to do and to find out. This allowed her to create the notion of trust and confidence in herself. She started by observing the setting, the people, activities events and some apparent feelings. Then, gradually she focused in on those elements which were particularly related to the research aims.

4.1 Competitions A competition was set up within the Leonardo‐network, Workshop III, in Lancaster University consisting on a challenge specific to Human Computer Interaction (HCI) and covering different themes. The concept of this experiment derived from previous Scrapheap events held at Lancaster since 2003. Scrapheap Challenge is an engineering game show produced by RDF Media and broadcast on Channel 4 in the UK. In the show, teams of contestants have 10 hours to build a working machine to perform a specific task, using materials available from a scrapheap. The programme of this event was composed of about twenty multidisciplinary researchers.

One day was given (about five hours) to each team absorbed in the Scrapheap spirit, to build a solution to the challenge using either the materials and equipment provided or material that each team brought to the competition. Afterwards, the participants were divided into teams that were made up from various clusters and placed in separate rooms.

One of the groups reflected on the HCI themes and the proposed tenets. Finally, they accepted, mobile phones and the material they needed for the artwork was defined: kitchen foil, a piece of carpet, transformer, among others.

212

Arminda Lopes After the final phase, the artefact was presented by the group. The idea behind was, with the topic of mobile phones, connecting people in the wild, which had generated ideas about wind up phones and chargers. The clue was to use a static generator, using carpet, to charge the mobile phone just enough to give one second of connectivity.

Figure 1: A static mobile phone charger

4.2 Workshops Another case study, with different characteristics to the previous one, as the result of the Third Workshop within the scope of the Leonardo – network which took place in London at University College of London was a source of data observation. The researcher was a direct observer. The objectives were different and the time duration was shorter. In this case, participants were not expected to build an artefact but to think and to materialise the design thinking process with drawings and words only.

People were divided into four groups of three persons per group. The way each team chose their participants was not understood: probably they split according to their position near each table and in some cases by connections between people. They created project’s ideas in response to three factors set by the facilitator.

Some exercises with sketching occurred initially, which was followed by a stage of design thinking, then the realisation of the challenge stage followed, and finally, the feedback presentation by each team occurred.

In Figure 2 the social context was composed of the groups who were challenged to identify interactive technologies that in some way could respond to a problem set by another team. Proposals were presented back to the challenging team and the other groups for comment and evaluation.

One of the themes that emerged from the workshop was Virtual Worlds for Real Communities – Virtual worlds that facilitate local communities to build a collective vision of what they wanted to achieve in the real world.

The collective virtual vision could then be manifested in the real world.

The two most visually important elements in the text are the square blocks on the top left hand side and the central block, in the middle with a feminine figure, a tree and two cameras. Amorphous shapes and lines are evenly distributed throughout the drawing. They are smaller and more complex in the centre than on the edges. The third block is on the top right hand side with a drawing specified by the label on the rectangle – “Nesco” – signifying a supermarket. A sketch indicating reality TV is placed in the top left hand side designed in squares each square representing a channel or a pool of programmes. A singer is drawn in the centre of the paper side by side to a camera. Above, there is a tree containing musical notes which is connoted with bird sound. More cameras are scattered along the right hand side of the text connecting to “Nesco“ supermarket.

The overall aspect of the narrative was creating an ambience that tends to put people to sleep.

213

Arminda Lopes

Figure 2: Lack of sleep, reality TV and supermarket In a case within the White Rose Network for Affective Communication in Consumer Product and Exhibition Design – a network event which took place in Sheffield Hallam University, the researcher participated as a direct observer.

The goal was to emphasise affective design and to identify spaces where there was benefit for some affective design. Participants were divided into four teams with four or five persons per team; and each team sat around a table to share their knowledge and attain the goals by writing and drawing together.

Teams were given a product and an adjective. They were asked to design a product with the attribute of the adjective. The design was followed by a discussion of each team’s choice and presentation ideas. Figure 3 shows an example.

Figure 3: Nail gun Comfortable and Nail Gun ‐ There were two approaches discussed; one considering the comfort of the user and one considering the comfort of the nail gun. The user’s comfort could be accommodated by having a comfortable grip and a non‐threatening soft looking shape. The nail gun’s self‐worth or comfort could be provoked by resting on a bed of jewels.

214

Arminda Lopes Figure 4, part a) presents sequential phases of the design process for the same proposal. The first one was made in a previous round of exercises and the second one was the visual result of that planning process.

Above the text, there are no visual images concerning drawings. These could be drawn mentally by the viewer in just a few guessing steps through the groups of similar and contrastive words presented. The story was developed from the centre of the paper written on the blue post it: “Dad waiting in a hospital for five hours with injured six year old boy”. The narrative comes in pink post‐its and the conclusions are written on the background paper inserted in ovoid shapes.

A considerable amount of emotional feelings expressed by the words meanings can be read from the narrative. Two axes are presented: time and space. Time refers both to physical and to psychological. The former concerns the time passing while waiting in the closed space of a hospital; the latter refers to the amount of feelings that appear and disappear during the physical waiting period and by the closure and openness which are represented by three different spaces: closed, opened, desired. Those can be connoted with the inside world, outside world and the desired world. The viewer can draw, through lines and shapes, both of the places.

In this text, the main message evoked through the written words is one of anxiety and frustration. A clear amount of negative feelings arose from the story – some interacting with the space and others interacting with the situation. All of these feelings belonged to the same axis: distress, boredom, displeasure, impatience, irritation, annoyance, preoccupation, sadness, pain, closure.

Figure 4: Dad waiting in a hospital with injured boy In Figure 4 part b) the story is the same but the expression is different. The story is “Dad waiting in a hospital for five hours with injured six year old boy” and the designers suggested a slogan “A waiting room that you won’t want to leave” which means a room with communication facilities: internet connections, telephones, a blue tooth or a bleeper system which would enable those waiting to be able to wander around the site without worrying that they would miss a call for the next phase of the medical process.

There are three different coding agendas: linear, shape and spatial organization. Lines are straight shapes – straight/curve, symmetrical/asymmetrical, opened/closed; positions – vertical/oblique, upright/inverted.

These contrasts reveal two contrasting parts, that which designers represented by drawings and that which they found in magazines (technology: games, toys, phones, and computers). Physical comfort was proposed by pieces of furniture and the entertainment one achieved by technology as a symbol of a consumerist society.

The objects’ drawings are in coloured ink, and only one is in blue, establishing the room’s comfort and the connection with the outside world. So, there are two kinds of spaces: inside – a closed circle given by the position of each element, while the outside is represented by the door at the top and the communicative

215

Arminda Lopes media which link to the outside. Pictures from a catalogue represent the desired objects to have in the room.

Some pieces are well defined while others are sketched.

Entertainment is represented by male associated activities expected to assist including pool, televised football, “scalextric” etc with plenty of other toys to keep the patient amused. There is a bar with a ‘blokey’ non‐ medical ambience represented by the glass and the pictures of a bar with bottles.

The notion of comfort is mainly represented by the sofa which is in a different colour and the room decoration: a painting on the wall and the desk oil lamp. This room would provide occupation and important contact with the outside world as desired.

4.3 Meetings Another source of observation was by team’s meetings. This case study was part of the Leonardo‐network, too. There were four institutions involved and four professionals with different backgrounds. In this case, the researcher took the role of participant observer.

In the first meeting the team discussed their ideas and materials to execute the artwork, making drawings, sketches and notes. First, ideas appeared in small waves of thoughts then, they became stronger as the team began to be increasingly sure of their ideas.

Further discussions between participants concerning their own motivations occurred regarding methodology, windows, manipulations, landscape or evoking qualities of environment, making connections between words and feelings about Scarborough, and about contrasts and landscape metaphors, in addition to, how techniques and structures could be used.

The second meeting ‐ planned by email ‐ with all the participants including the researcher was video recorded.

The agenda included project discussion, research, artefact, exhibition, outcomes and timetable.

Participants were introduced to the main point of the meeting, the objective of producing a blueprint for the artefact they intended to create and a summary of the position of the artefact at that moment was presented.

Later, they agreed on the production of a series of landscapes and windows, with frames in the background and the creation of an atmosphere through sound effects and that the artwork had to function.

During the design process, all the participants intervened and the environment was very informal, however, the way of presenting and discussing the ideas was organised sequentially. People were concerned with collaboration, communication and creativity. Along observations it was notice that the main story was interrupted with some short stories to complement or explain events.

The final 16 artefacts, displayed on artist’s easels, involuntarily explored notions of narrative, character development, synthetic speech and lyricism in Human Computer Interaction and interface design (Figure 5).

Figure 5: Sixteen artefacts

216

Arminda Lopes

5. Discussion Observation methods in observing behaviours are flexible and do not need to be structured around a statement about what we expect to observe. The reported observations were conducted in order to understand what designer say, how they behave and how they do an artefact/artework. Topics such as communication, understanding people, culture and leadership were the common ingredients along observations. The repeated observations and descriptions of events allowed making some cautious but useful generalisations about relationships and making a prescriptive statement. Factors that determined individual differences such as environmental, personal and psychological were understood.

The analysis of dialogue during the design process was based on conversational analysis, interpretation and context analysis of discourse. Discourse analysis was used to examine modes of speech used in dialogue, used as social narratives to explain the events. All the design process was video recorded, as stated before, accompanied by textual descriptions; attending to the conversation formulations produced by designers in their dialogue together; observation of designers talking, listening to their conversations and describing their formulations; analysis of the work plans; observation of all the activities which took place and the relationship between activities and designers.

Since a video recorder was used, non‐verbal behaviour could be analysed in detail. First, it provides extra information, which aids to understand what people were in reality, what they were thinking, feeling or meaning. The other advantage permits us to have a more successful relationship with people. Non‐verbal behaviours are easy to observe but difficult to interpret. The problem is deducing a correct meaning from what has been seen, and it cannot be generalised because of the diversity of people’s cultures. However, the gestures registered during the observations represented a demonstration of personality, and it complemented the dialogue to understand what was said and in what conditions and also how participants were motivated throughout the design process. While speaking, people commonly employ hand gestures or other hand movements to illustrate or add emphasis to words, which is common during dialogues. Non‐verbal behaviour covered a variety of different aspects including: Body Posture; Body Behaviours; Facial Expressions and Gestures; Gaze and Eye Contact and Quality of Speech. Artworks were designed to provide information, entertainment or communication; this was provided through verbal or non‐verbal process which means that visual dialogue was analysed. The tangible results of the different types of observations, stated in this paper, were presented by figures. Those figures served as some data source for the researcher to describe and interpret observations’ results.

Figure 1 and 5 were the output of two different challenges: a competition and a meeting work environment. In the former, an artefact which idea behind was connecting people in the wild, was presented. The clue was to use a static generator, using carpet, to charge the mobile phone just enough to give one second of connectivity. In the latter, an artwork was designed using Integrated Development Environment (IDE) Microsoft Visual Studio. The narrative drawn from musings on windows, tides and place, the images were from Scarborough. A sound system was hidden behind the canvas and the sound was played as spectators approached the piece. The first impression was the familiar image of the computer interface (Windows) in the context of an art gallery and on canvas. It addressed multiple themes and conversations. The artwork provided a dynamic space in which a methodology of collaboratively creating informed artistic practice was developed.

Figures 2, 3 and 4 were made within a design thinking process strategy. A direct message to the viewer is presented by figure 2 “Joint with other Insomniacs” to summarise the represented ideas. Figure 3 exemplifies one of the conclusions from this workshop which was: aiming for a single product is too narrow a concept, so designers need to look at the wider context and get away from function. Figure 4 shows the solution for the proposed story: communication with the inside and the outside world and entertainment. The designers’ message seems to be straightforward “A waiting room you don’t want to leave”. This room would provide occupation and important contact with the outside world as desired.

Observation method was considered important to understand and register behaviours in real‐life situations. In the case, groups with people of diverse backgrounds, experiences and cultures were observed in different situations and by different researcher’s positions. The observation method required discipline and concentration, even by recording and asking observed people in order to clarify concrete situations, about their verbal behaviours.

217

Arminda Lopes

6. Conclusions Observation is a suitable method to observe behaviours (verbal and non‐verbal). In this paper, several examples of video recorded case studies about designers, working during the design process, in different environments, were presented. The controlled of the work environment allowed the researcher to take different roles during the observation processes. In some cases, the researcher served as a participant in the event and at the same time, observing the actions and outcomes. Conversely, a non‐participant observation was also undertaken within situations where design thinking strategy was the context.

The researcher participation brought a different interaction. Permitting asking several questions to be asked; in which, allowed to deeply understand the reasons and motivations of the participants during the design process. The participation was not understood as an intrusion on the environment since meetings were video recorded which allowed the data analysis kept the event natural. In this position, the observation type was unstructured, since the researcher wanted to get involved in direct interactions with each group of designers.

Unstructured observations happened within meetings: the researcher already knew the participant, although the environment and work context was unfamiliar. However, some initial guiding behaviour was designed. As a result of this type of observation, the researcher felt that group’s behaviour was more informal and the data gathered was richer, since people behaved freely without the preoccupation of the video recording environment.

In non‐participant observation’s position, the researcher was able to prepare a guidance notes in order to complement the data gathered in other situations. Structured observation was specially prepared for workshops and competitions since the environment and participants were unknown. The experience in this role was very fruitful. The researcher observed the designers and recorded as much of their behaviours as possible. The obtained results permitted to analyse and understand people’s non‐verbal behaviours and focus on the design process’ detail.

The reported observations, presented in this paper, were the result of three levels of data analysis: descriptive analysis – no inference making on the part of the researcher. The transcriptions were rendered as they were; inferential analysis – the researcher assumed her own interpretations, especially within non‐verbal behaviours’ observation; and evaluative analysis – occurred, sometimes, in figures’ description.

To conclude, the main contribution of this paper was the story telling approach presented by the researcher to expose her experience on conducting observation methods: participant and non‐participant and structured and unstructured observations, mixed at the same research. Conversely, it gave the opportunity to describe and interpret the artefacts/artworks using the participant and non‐participant roles altogether.

References Babbie, E. (2010) The Practice of Social Research, Cengage Learning.

Charmaz, Kathy (2006), Constructing Grounded Theory, A Practical Guide Through Qualitative Analysis, SAGE Publications, London, Thousand Oaks, New Delhi.

Chatterton, Hilary, Ewan, Louise, Kinmond, Kathryn, Haire, Antonita, Smith, Nickolas, Holmes, Paul S., (2008) Observation of Meaningful Activities: A Case Study of a Personalized Intervention on Poststroke Functional State, Journal of Neurologic Physical Therapy: ‐ Volume 32 ‐ Issue 2 ‐ pp 97‐102 Corbin, J. (1986). Qualitative data analysis for grounded theory. Dans W. C.Chenitz & J. M.Swanson (Eds.), From practice to grounded theory, Qualitative research in nursing (p. 91‐101). Menlo Park, CA : Addison‐ Wesley.

Darlington, Y. and Scott, D. (2002) Qualitative Research in Practice: Stories from the Field, Maidenhead: Open University Press.

Dawson C. (2006) A Practical Guide to Research Methods: A User‐Friendly Guide to Mastering Research Tecnhiques and nd Projects, 2 edition, Oxford: How to Books Ltd. Denzin, Norman K (1989), A Theoretical Introduction to Sociological Methods, Prentice Hall, USA. Iacono, J. Brown, A. and Holtham, C. (2009) “Research Methods – a Case Example of Participant Observation.” The Electronic Journal of Business Research Methods Volume 7 Issue 1, (pp.39 ‐ 46), available online at www.ejbrm.com rd Kumar, Ranjit, (2011) Research Methodology – a step‐by‐step guide for beginners, 3 Edition, Sage publications.

Shinohara, K.and Tenenberg, J. (2007) Observing Sara: A case study of a blind person’s interactions with technology.

Proceedings of the ACM Conference on Assistive Technology (ASSETS), 171‐178.

Stake, R. (1995), The Art of Case Research, Newbury Park, CA: Sage Publications.

Yin, R. (1994), Case Study Research: Design and Methods (2nd ed.). Thousand Oaks, CA: Sage Publishing.

218

Applying Action Research in the Adoption of Information Systems Security Policies Isabel Lopes¹ and Filipe de Sá‐Soares² ¹Departamento de Informática e Comunicações, Instituto Politécnico de Bragança, Bragança, Portugal ²Centro Algoritmi, Universidade do Minho, Guimarães, Portugal isalopes@ipb.pt fss@dsi.uminho.pt

Abstract: Information Systems Security (ISS) is a critical issue for a wide range of organizations. This paper focuses on organizations belonging to a particular sector, namely Local Public Administration, where public and personal information must be protected by those in charge, and where there must be a concern to view security as a priority. There are several measures which can be implemented in order to ensure the effective protection of information assets, among which stands out the adoption of ISS policies. A recent census concluded that among the 308 Town Councils in Portugal, only 38 indicated to have an ISS policy. The conclusion drawn from that study was that the adoption of ISS policies has not become a reality yet. As an attempt to mitigate this fact, an academic‐practitioner collaboration effort was established regarding the implementation of ISS policies in three Town Councils. These interventions were conceived as Action Research projects.

This article aims to constitute an empirical study on the applicability of the Action Research method in information systems, more specifically through the implementation of an ISS policy in Town Councils where previous attempts to adopt a policy have failed. The research question we intend to answer is to what extent this research method is adequate to reach the proposed goal. The results of the study suggest that Action Research is a promising means for the institutionalization of ISS policies adoption. It can both act as a research method, improving the understanding among researchers about the issues that hinder such adoption, and as a change method, assisting practitioners to overcome barriers that have prevented the implementation of ISS policies.

Keywords: action research, information systems security policies, information systems security policy adoption, information security

1. Adoption of information systems security policies Nowadays, Information Systems Security (ISS) is a critical issue for a wide range of organizations. The centrality of information in the operations and management of organizations raises concerns regarding the protection of information systems’ (IS) assets, including hardware, software, data, processes, and people.

In order to ensure the effective protection of IS, organizations implement several different security measures.

Among these measures, ISS policies stand out. These are “documents which guide or regulate people or systems actions in the domain of information systems security” (de Sá‐Soares 2005, p. 56). The importance of ISS policies is stressed by several authors, such as Peltier (2002, p. 21), who classifies them as the “cornerstone of an effective information security architecture”.

In order to adopt an ISS policy, an organization must follow a sequence of steps, beginning by writing the policy, followed by its implementation, and then, at predefined moments or when circumstances require it, by reviewing its provisions, which may prompt modifications in the policy. Indeed, this sequence of steps may be viewed as a cycle of formulation – implementation – revision of the policy.

Although there is a considerable agreement in the literature regarding the main role played by ISS policies, there is evidence that organizations often fail in the adoption of this security control. Focusing their attention in a particular type of organizations, namely Local Public Administration, Lopes and de Sá‐Soares (2010) surveyed the 308 Town Councils in Portugal to find out that only 38 (12%) indicated to have an ISS policy.

However, it was also found that 177 (66%) of the respondents had thought or were considering formulating an ISS policy, but were not yet able to reach the state of having adopted that security measure. The conclusion drawn from the study was that the adoption of ISS policies has not become a reality yet, suggesting there is still a long way to go before the institutionalization of ISS policies measure that group of organizations.

This state of affairs promptly raised several questions to the researchers, such as the reasons for such a low level of adoption and the obstacles that have prevented the Town Councils to successfully apply ISS policies.

219

Isabel Lopes and Filipe de Sá‐Soares Shortly after the conclusion of that survey, the heads of the IT departments of several municipalities that still hadn’t adopted an ISS policy contacted the first author of this paper requesting assistance for the implementation of an ISS policy. Although the specialized literature provided general guidelines regarding the content for the policy documents as well as several recommendations for writing, implementing, and reviewing ISS policies, the authors were faced with a methodological decision, i.e., how to do it. After considering several alternatives, such as promoting workshops or just plain consultation work, a decision was made to propose the Town Councils an Action Research (AR) intervention.

This article aims to constitute an empirical study on the applicability of the Action Research method in the field of IS, more specifically analyzing the implementation of ISS policies in Town Councils where previous attempts to adopt a policy had failed, according to the tenets advocated by AR. Hence, the research question that guided this work was to answer to what extent AR methodology is adequate to support the process leading to the adoption of ISS policies.

Structurally, this paper is organized as follows. After this contextualization of the subject, we review the main tenets and characteristics of AR, in general and in the field of IS. Then, we describe the collaborative efforts that were promoted to adopt ISS policies in three Town Councils, followed by a discussion. Finally, we enumerate the papers’ main contribution, limitations, and suggestions for future work.

2. Perspectives on action research The description of a research method application, as well as the lessons learned from that application, benefit from several previous clarifications. Among them are the way researchers understand the research method, the indication of the method’s main characteristics, and the explanation of how the method applies to the targeted practice context.

AR method can be viewed in various ways, and there are probably as many views of it as the number of authors who address the topic (Jӧnsson 1991). As an example, for Rapoport (1970, p. 499), AR “aims to contribute both to the practical concerns of people in an immediate problematic situation and to the goals of social science by joint collaboration within a mutually acceptable ethical framework”. Kemmis and McTaggart (1988, p. 125) see AR as “a kind of collective and self‐reflective survey which the participants in social situations use for the rationality and justice of their own educational actions and for their understanding of those actions as well as the situations in which they undertake them”.

Although different authors may have different perspectives concerning the application of AR, there is consensus with respect to the method general architecture. Briefly, AR starts with the detection of a problem, from which changes are projected in order to solve the problem. This process has a cyclic nature and, once it is applied to organizations or other social groups, it will hardly be seen as definitely solved. It will rather suffer changes and require new interventions. As a result, AR is considered a change‐oriented methodological approach: it is not restricted simply to the understanding of phenomena but it deliberately aims at changing those phenomena.

Although the exact characterization of AR varies with the authors, Dick (2000) isolated a set of aspects which seem to be consensual among authors:

It acts on an existing situation with the dual aim of improving it and expanding the knowledge on the subject.

It possesses a cyclic nature: a number of steps are performed repeatedly. The cycle varies with the author but, at least, it includes the steps: Planning – Action – Reflection.

It admits the participation of the research subjects, although this condition is not unanimously considered as mandatory.

It possesses a reflexive nature: a critical reflection on the research process itself as well as on the results obtained is an important part of each cycle.

It is predominantly qualitative, although quantifications are possible in some situations.

220

Isabel Lopes and Filipe de Sá‐Soares The AR method completes an interactive cycle made up of a series of stages whose number and designation depend on the author. Considering the review of literature carried out, three illustrative models were identified, varying in terms of structural complexity.

Cunha and Figueiredo (2002) present a model adapted from Dick (1992), that includes three stages: Planning, Action and Reflection, as shown in Figure 1.

Source: Cunha and Figueiredo (2002) Figure 1: Three steps AR cycle Based on these three building blocks, those authors point out the philosophy underlying AR: “An intervention is planned (Planning); the corresponding action is taken (Action), causing a change which will hopefully lead to development; finally, a critical analysis of the results is made, which should lead to a better knowledge of the situation, which, in turn, enables possible adjustments that lead to new cycles (Reflection)”.

Tripp (2005) conceives the execution of AR in four phases: Planning, Acting, Describing, and Evaluating, as represented in Figure 2. In AR a change is planned, described and evaluated viewing the improvement of an action. Throughout the process, further learning takes place, both concerning the action and the research itself.

Source: Tripp (2005) Figure 2: Four steps AR cycle A more complex model was proposed by Susman and Evered (1978), who claim that the approach initially requires the definition of a “Client‐System” infrastructure, that is to say, a research environment, followed by a cycle made up of five stages, as shown in Figure 3.

Associated with each of the stages included in this model are the following goals:

Diagnosing – Identification of a problematic situation, related to the need of change of a certain organization;

Action Planning – Specification of the organizational actions which must be undertaken in order to solve the problems identified in the diagnostic;

Action Taking – Implementation of the actions previously planned which will supposedly lead to changes;

Evaluating – Assessment of the intended goals achievement and solution;

Specifying Learning – Specification of the knowledge acquired with the introduced change. Although this stage appears as the last in the scheme, it consists of a permanent process.

221

Isabel Lopes and Filipe de Sá‐Soares

Source: Susman and Evered (1978) Figure 3: Five steps AR cycle

3. Action research applied to information systems In IS the nature of knowledge is different from the nature of knowledge in traditional Sciences (empirical and formal). For this reason, traditional research methods are not always appropriate to guide the inquiry in the field of IS. Actually, some authors classify the traditional approaches as impracticable in IS, for not being realistic (Cunha and Figueiredo 2002). This understanding may lead to the option for qualitative research methods to study IS related phenomena. One of the main reasons given to justify the use of such methods is the fact that IS include the human element as a variable or consider it as a determinant research factor. Here probably lies one of the reasons for using AR in IS studies, as it is “one of the few research approaches that we can legitimately apply to study the effects of specific changes in the methods of systems development in human organizations” (Baskerville and Wood‐Harper 1996).

According to Baskerville (1999), AR was explicitly introduced in the IS community as a pure research method

by Wood‐Harper (1985). Reviewing the uses of AR in IS, Baskerville and Wood‐Harper (1998) were able to identify ten forms of AR in IS, differing in terms of several characteristics, which were organized into four groups: Process model; Structure; Typical involvement; and Primary goals. Table 1 shows these forms and characteristics.

Table 1: IS Action research forms and characteristics Adapted from Baskerville and Wood‐Harper (1998)

222

Isabel Lopes and Filipe de Sá‐Soares From the exposed structures on works carried out in the field of IS using AR, we can see the variety of practices intervened, as well as the different types these interventions have assumed from the methodological point of view.

In the context of qualitative research in IS, Estay and Pastor (2000) consider AR operates over two realities, a scientific/academic one and a practical one. Thus, two different main types of AR cycles can be identified:

Cycles looking to solve problems in IS projects: These projects frequently consist of developing an IT artifact, with the researcher focusing on the solution of specific IS development problems. In this case, the purpose of AR is the creation of knowledge useful to the subjects and the improvement of a certain practice in which they are involved. The method is applied to build models, theories and knowledge, but in a way that is informed and biased by the reality upon which it is intended to act. In this cycle, the interest in solving a specific problem generates interest in researching the practice associated with that problem.

Cycles looking to inquiry in research projects: These projects are intentional research efforts in search of a result, in which AR acts as a structuring working method and as a reason for approaching a certain reality with the aim of testing a theory or hypothesis. In this case, the primary intention is to produce new knowledge in the field of IS, enabling the improvement of the researchers themselves. In this cycle, the interest in researching generates the interest in solving specific problems.

4. Action research applied to the adoption of information systems security policies The option for AR as the fundamental methodological guidance for the ISS policy adoption process resulted from the assumption of a set of propositions, partly supported in the literature and partly stemming from the results of the survey previously mentioned.

Given the reported difficulties of formulating a policy, as well as the evidence regarding the resistance of users on observing the policy, a joint, collaborative effort was the preferred way to move forward. By involving researchers and practitioners in a dialogue, we hoped to be able to transfer some best practices and theoretical knowledge to the users, while users explained the context factors that may facilitate or inhibit the success of the ISS policy and elaborate on their specific requirements in terms of IS protection.

It was also hoped that the cyclical structure of AR could better capture the advocated steps for the adoption of ISS policies, from formulation, to implementation, and then to revision. It would be easy to make that sequence of steps as a natural progression, wherein after its culmination, a new cycle of formulation, implementation and revision of ISS policies could be triggered. Underlying this cycle would be a learning process, where users and researchers could enhance the chances of learning what was working as expected, and what fell short or was counterproductive.

As the cycle of AR starts with the detection of a problem, the perception of such problem was clear in this study, namely the low level of ISS policies adoption by Portuguese City Councils.

After detecting this problem, intervention projects were started in three City Councils, aiming the introduction of changes towards the adoption of ISS policies. The whole process was structured according to the model proposed by Susman and Evered (1978) (cf. Figure 3).

In the first stage – Diagnosing – a problematic situation was identified, namely the non‐adoption of an ISS policy by the City Council. This situation was made worse by the fact that the problem had been isolated previously and the head of the IT department had not been able to invert that situation. In other words, although the problem was known and assumed, the organizations had not been able to create the context to change the situation. This finding reinforced the conviction that AR might prove to be particularly appropriate to change the ongoing practice.

The first author came into contact with the reality of the three City Councils, starting her intervention by meeting the head of the IT department, and immediately trying to identify the reasons for not having managed to implement an ISS policy previously.

In one of the cases, the main reason was that they had not found any ISS policy model that they could adapt to the City Council reality. In another case, there had been some resistance from a council executive regarding

223

Isabel Lopes and Filipe de Sá‐Soares the adoption of an ISS policy. In the third case, it was due to the fact that the ISS policy document had been made available on the Council intranet by the IS function, without being approved by the executive and therefore, the implementation consisted only on making the document available online without any other type of contract with the users of the City Council IS.

Besides the identification of the problem and the reasons inherent to the previous adoption failures, it was also during this stage that the real need for an ISS policy in the City Council was assessed. It was consensual that City Councils must stop worrying only about crackers’ attacks or about the implementation of firewalls or anti‐virus, and start focusing on the creation of an ISS policy which can promote not only the confidentiality, integrity, and availability of information, but also the responsibility, integrity, trust, and ethics towards information.

In the second stage – Action Planning – the organizational actions which must be executed to solve the problems identified in the diagnostic were specified. This process started by drawing the ISS policy document.

The first author and the City Council IT Department Head started by assessing whether one policy would be enough or more than one would have to be drawn. We studied the possibility of drawing two policies, one aimed at the IT technicians and another at the users. However, bearing in mind that technicians are also users, although with different specifications, we chose to write only one broader policy document. We planned to draw the policy based on a model proposed by the first author and adapted to each City Council following the indications of elements from the IT department.

After drawing the security policy, we planned its implementation, which depended directly on the guidelines addressed in it. After talking to the City Council entities, two essential factors were isolated for the success of policy implementation. Firstly, the policy would need the approval of higher entities in order to have the necessary “authority” recognized by all the users. Besides this, its right promotion would be necessary among the organization human resources and users of the addressed IS in general. The way to promote the policy was also taken into account, and we planned that, after its approval, it would be clearly explained to the heads of all departments and heads of all divisions so that these could deliver and explain the document to all IS users in their sections.

In the third stage – Action Taking – the planned actions were implemented, in the hope that these would lead to a change in the organization. In the face of the risk that ISS policies may not respond to the ISS requirements of an organization if they become obsolete due to changes in the business or threats to which the organization is submitted, some factors, such as auditing, were included in the implementation stage, in order to allow an assessment of the conformity with what was defined in the policy. The implementation also considered the management of incidents which, besides treating ISS incidents, enables to verify whether the policy manages to respond to the incidents or on the contrary, it does not include some important aspect, thus resulting in the need to implement the policy again or review its formulation. Depending on the importance or severity of the incidents or unconformities detected, relevant elements would be available for an eventual reformulation. To a certain degree, it is possible to draw a parallel between the integration of these audit and incident management tools and the subsequent sages of AR, as they enable an easier evaluation of the implemented actions, and might be useful to launch new AR cycles viewing the practical improvement of the implemented ISS policies.

In the fourth stage – Evaluating – we assessed the achievement of the intended goals of the ISS policy implementation. This evaluation required a review of the policy, which must take place periodically and especially whenever significant changes occur, in order to guarantee that the policy continues to meet the goals for which it was adopted. The evaluation was carried out by assessing the users’ compliance with the rules set by the policy. The subsequent modification of the policy was not found necessary for the time being.

The last stage – Specifying Learning – concludes the cycle, although in fact, this stage accompanies the whole process cycle of AR. The learning which took place throughout the whole cycle worked as a starting point to a new planning and, therefore, to the beginning of a new cycle sequence.

224

Isabel Lopes and Filipe de Sá‐Soares

5. Discussion The implementation of an ISS policy following the AR method was aimed at the construction of a solution to generate new knowledge, which was useful to the participants, on how to implement an ISS policy and improve its practice through successive evaluations and associated changes when necessary. At the same time that researchers cooperate in that process, they also aimed to add to accumulated knowledge, trying to understand the hindrances faced by organizations in the process of ISS policy adoption and to investigate the effectiveness of initiatives put on practice to overcome those difficulties. By participating in several of those processes, the research team collected evidence that may prove useful on projecting future interventions in other organizations of the same type. This dual interest of researchers – helping to change the specific context of practice (Action) and adding to the general knowledge of the ISS policy adoption process (Research) – raises some questions. Since the intervention is based on a cooperative structure, and since the control of the intervention by researchers is limited, the clear articulation and negotiation of the goals, views, and interests of the two groups of participants is particularly relevant.

In the present application of AR, these aspects were born in mind so as to guarantee higher accuracy and validity as well as lower limitations concerning the conclusions obtained in general. There was an effort to not manipulate or control, but to present users with alternative solutions, to draw their attention to issues that may go unnoticed or that although problematic for the users, should be addressed. Similarly, particular attention was devoted to the situational factors that characterize the context of practice, both in terms of work routines and of security actions that users have to counterbalance.

Given the collaborative nature of this study, the insights of the participating researcher were often debated and brought to reflection in order to produce a shared understanding that led to the change. Indeed, it was not intended that the researcher would unilaterally propose a change plan, but to build such a plan with the other actors involved in the transformation, namely the Town Council IT Departments.

The organizational culture of the Town Council and the level of training of its IT Department technicians play an important role in the implementation of an ISS policy, both in terms of awareness and training sessions required and in terms of users’ resistance to the provisions of the policy. Also, the size of the Town Council dictated how the policy document was disseminated among IS users.

The most critical aspect in the adoption of an ISS policy by a Town Council is the ISS awareness level of its executives. This is a paramount factor for explaining delays or blockages in the adoption, as well as processes that lead to a quick adoption of a policy.

In all three interventions, the actors believed that having an ISS policy model they could adapt to their reality increased the chances of successfully implementing an ISS policy.

Among the cases of application studied, we found evidence that the adoption of ISS measures, namely policies, must go beyond the implementation of hardware or software devices which protect what is stored in the organization databases and files and which, quite often, do not offer the necessary or expected security due to functioning, parameterization or installation flaws (Peltier 2002). Besides the technological component, the human element constitutes the core of ISS. The difficulty in managing that element and in making it the main responsible for an effective protection of information assets is what makes ISS one of the most difficult and arduous aspects of many organizations management.

The institutionalization of ISS policies implies that the users observe the provisions of these policies on a daily basis, or, not of less importance, that they identify the aspects of the policy which lead to a lower protection level. By contemplating the specificities of each organization and by promoting the cooperation among researchers and users regarding the projection of actions which will affect them, AR acts both as a research and change method particularly promising for the adoption of ISS policies. On the one hand, it helps researchers understand the usefulness and limitations of the existing knowledge, opening new avenues to a better understanding of the ISS policies adoption phenomenon. On the other hand, and as a change method, it enhances the sense of property and co‐responsibility of those who need to put into practice or review the procedures set in the ISS policies on a daily basis.

225

Isabel Lopes and Filipe de Sá‐Soares Situating the interventions according to the classification presented in Table 1, the studies configure canonical AR projects, grounded in an iterative process model guided by a rigorous structure, with the participating researcher playing a facilitative role, and having organizational development as their primary goal in the form of adopted ISS policies.

6. Conclusion This study involved three City Councils through direct contact with the correspondent IT departments and indirect contact with the municipal executive as well as the users of the municipality IS. This work reports on the use and appropriateness of AR applied to the adoption of ISS policies, thus contributing as an empirical study on the application of that method in the field of IS. This research work presents limitations, namely with respect to the number of City Councils involved. Although we believe that the study carried out in the three City Councils generated enough data to serve the goal of the work, we also believe that a larger number might result in a more sustained set of data. Nevertheless, we highlight that the application of the action research method requires the researcher’s direct involvement, thus requiring a substantial amount of time. Another limitation of this work is related to the delimitation of the study within an organizational sector and a specific national reality. Among the works which might be carried out in the future, we highlight the proposal of an ISS policy model, thought up for the national municipal reality, and which may work as a starting point to the adoption of ISS policies by the City Councils, so as to invert the reduced number of policies existent in the Portuguese City Councils. The provision of that document by the City Councils and the use of AR as a method for planning and promoting change, in which researchers and practitioners project actions, implement them, and evaluate their impacts, may prove to be two important tools for the institutionalization of ISS policies in organizations.

Acknowledgments This work is funded by FEDER funds through Programa Operacional Fatores de Competitividade – COMPETE and National funds by FCT – Fundação para a Ciência e Tecnologia under Project FCOMP‐01‐0124‐FEDER‐ 022674.

References Baskerville, R. (1999) “Investigating Information Systems with Action Research”, Communications of the AIS, Vol 2, No. 19, pp 1‐32.

Baskerville, R. and Wood‐Harper, A.T. (1998) “Diversity in Information Systems Action Research Methods”, European Journal of Information Systems, Vol 7, No. 2, pp 90‐107.

Baskerville, R. and Wood‐Harper, A.T (1996) “A Critical Perspective on Action Research as a Method for Information Systems Research”, Journal of Information Technology, Vol 3, No. 11, pp 235‐246.

Cunha, P.R. and Figueiredo, A.D. (2002) “Action Research and Critical Rationalism: a Virtuous Marriage”, Proceedings of the 10th European Conference on Information Systems, Gdansk, Poland.

Dick, B. (2000) “A beginner’s guide to action research”, [online], Southern Cross University, Australia, http://www.scu.edu.au/schools/gcm/ar/arp/guide.html.

Dick, B. (1992) “Qualitative action research: improving the rigour and economy'”, Proceedings of the Second World Congress on Action Learning, University of Queensland, Brisbane, Australia.

Estay, C. and Pastor, J. (2000) “Towards a Project Structure for Action‐Research in Information Systems”, Proceedings of the 10th Annual Business and Information Technology Conference, Manchester.

Jönsson, S. (1991) “Action Research”, in Nissen, H.‐E., Klein, H. and Hirschheim, R. (Eds) Information Systems Research: Contemporary Approaches & Emergent Traditions, North‐Holland, Amsterdam, pp 371‐396.

Kemmis, S. and McTaggart, R. (Eds) (1988) The Action Research Planner, Third edition, Deakin University Press, Victoria.

Lopes, I. and de Sá‐Soares, F. (2010) “Information Systems Security Policies: A Survey in Portuguese Public Administration”, Proceedings of the IADIS International Conference on Information Systems, Porto.

Peltier, T.R. (2002) Information Security Policies, Procedures, and Standards: Guidelines for Effective Information Security Management, Auerbach Publications, Boca Raton.

Rapoport, R.N. (1970) “Three Dilemmas in Action Research”, Human Relations, Vol 23, No. 4, pp 499‐513. de Sá‐Soares, F. (2005). A Theory of Action Interpretation of Information Systems Security, PhD Thesis, University of Minho, Guimarães.

Susman, G. and Evered, R. (1978) “An Assessment of the Scientific Merits of Action Research”, Administrative Science Quarterly, Vol 23, No. 4, pp 582‐603.

Tripp, D. (2005) “Pesquisa‐ação: uma introdução metodológica”, Educação e Pesquisa, Vol 31, No. 3, pp 443‐466.

Wood‐Harper, T. (1985) “Research Methods in Information Systems: Using Action Research”, in Mumford, E., Hirschheim, R., Fitzgerald, G. and Wood‐Harper, T. (Eds) Research Methods in Information Systems, North‐Holland, Amsterdam, pp 169‐191.

226

Why Older People Intend to Learn Computers ‐ a Grounded Theory Study Eurico Lopes, Luis Barata and Ana Barata School of Technology, Polytechnic Institute of Castelo Branco, Portugal eurico@ipcb.pt luis.barata@beiranet.pt, ana.barata@ipcb.pt

Abstract: Information and Communications Technology is geared for the modern man, which is not the standard of our society, which creates individuals who feel excluded from such interaction with technology, and as a result the interaction with others. Several organizations such as the United Nations, European Commission and World Health Organization reaffirm the need to look at how people age, given that every time there is a higher proportion of new vs. old and with the increasing of the technological society, there is a marginalization of older individuals. This paper shows that the idea of old synonym for useless fades over time, and these "new elders" are quite participatory. Thus it was researched that activity and participation in a University of Third Age, the Senior University of Castelo Branco (USALBI). Using the Grounded Theory Methodology (GTM) and Actor‐Network Theory (ANT) with open‐observation the work analyzed the motivations that lead older people to learn computers, and seeks to unveil the underlying interests. Describing how elders see computers and information systems and how they see the use of computers to integrate them into society. This interpretation also includes changes observed in the level of relational structures, especially in the way the computer helps and approach social networks of the elderly. The choice was Grounded Theory Methodology (Glaser and Strauss, 1967; Charmaz, 2006, Lopes, 2010) since it’s a research methodology, but also a process to analyze data interviews. Actor‐network theory (ANT) is a social science approach for describing and explaining social, organizational, scientific and technological structures ((Law, 1992; Callon, 1999; Law and Hassard, 1999). Thus, it´s useful for describing the context network of the learning‐ processes taken in the manner that computer helps and approach social networks of the elderly.

Keywords: social informatics; learning in old age, grounded theory methodology, actor‐network theory

1. Introduction The last five decades, in addition to all the great achievements of humanity, are marked by the achievement of life expectancy, the result of all the advances in medicine and social concern of how people get aged, being the aging of the population one of the major challenges currently facing humanity (Fontaine, 2000;Fernandes, 2008; Nazareth, 2009). Several organizations such as the United Nations, European Commission and World Health Organization reaffirm the need to look at how people age, given that every time there is a higher proportion of new vs. old, and with the increasing of the technological society, there is a marginalization of older individuals. In result of the huge development of Information Technologies and Communication, because of the low cost associated, its global usage has increased a lot, being now part of everyday life. However this technology was intended for the “modern” man, which is not the standard of our society. This fact make that some people feel excluded from such interaction with technology, and as a result, with the interaction with others.

This paper shows that the idea of “old is useless” fades over time, and these "new elders" are quite participatory. Thus, it was investigated the activity and participation in a University of Third Age, the Senior University of Castelo Branco (USALBI). Using the Grounded Theory Methodology (GTM) with open‐observation and interviews, the work analyzed the motivations that lead older people to learn computers, and seeks to unveil the underlying interests. Also describes how elders see computers and information systems, and how they see the use of computers to integrate them into society. This interpretation also includes changes observed in the level of relational structures, especially in the way the computer helps and approach social networks of the elderly.

This paper is organized to give a general understanding of the study, starting a brief justification of the methodology, organization of study participants, data collection and ethical issues involved. ANT concepts and its objects, particularly Inscription, Translation and Punctualisation are analyzed and explained the integration with GTM. Subsequent steps are described in GTM, leading to the construction of categories and the framework. The paper ends with the discussion of the obtained results and some to‐do research work.

227

Eurico Lopes, Luis Barata and Ana Barata

2. Methodology and study In this paper we intended to answer the research question “Why older people intend to learn computers?” To answer this question we need suitable research methodology to work within a context of elderly people, retired from active professional duties which offer data that may be used to answer the research question.

First of all, we want to start from a point of view, that we do not have a mind full of preconceived ideas that may distorce the study, to areas where we start asking preconceived questions to the research participants.

This implies the negation of a well‐organized questionnaire, because we don´t know what to ask at the beginning. Secondly, we have to find elderly people, retired from professional action, but using computers in their lives. This wasn´t an easy task, because there weren´t old people laying around in coffees with computers, waiting for us to ask them questions why were them using it. So, a qualitative analisys aims to collect information that describes the social reality from the outlook of the elderly people using computers, and enable the researchers could construct a classification of specific dimensions, that characterize and explain this experience.

The research methodologies can be classified as quantitative and qualitative, differentiated by their origin.

While the quantitative methods generally use statistical methods to verify a particular hypothesis, qualitative methods seek to understand the phenomena by explaining them through analysis. Qualitative research, according to Flick (2002), is a research approach that produces results, without involving quantitative or statistical procedures. Thus, it can be defined as a process of interpretation, with the aim of discovering concepts and relationships in the data collected by organizing them into a theoretical scheme with a valid meaning.

For this reason, qualitative research methods are especially used in research that seek to understand the nature of human experience, in specific situations. These methods are also used in studies that seek to research areas of undeveloped knowledge, with the aim of creating new knowledge (Strauss & Corbin, 1990).

The study of the interaction of elders with information technology and computers, is still an aspect slight studied and there is few documentation. It appears that qualitative research fit the scope of this study. We intend to research the experiences, interactions and practice with computers, complemented with a data analysis to generate new knowledge or theories about the subject.

2.1 Grounded theory (GT) The Grounded Theory has its origins in the context of sociological studies, despite being increasingly used in research on new technologies. It is characterized by constant development of the theory; through a recurring process that evolves naturally as the collection and data analysis occur simultaneously (Strauss & Corbin, 1990). Grounded Theory has the purpose to build and do not verify the theory (Flick, 2002). Therefore, using this methodology, the researcher has essentially an interpretative role, introducing throughout the investigation, the views, ideas and sensitivities of participants. This choice was due to the fact that the need for, first, studying a social and human aspects, that involves people. Hence the choice of a qualitative research method. And secondly, the need to create new knowledge, and understand a little explored area of knowledge, a fact that Grounded Theory is particularly appropriate.

This methodology is based on the idea of researching, not having or using pre‐conceived ideas; the researcher should begin to research with a clean mind in terms of aspects to research. In the process of analysis, this methodology is very time consuming, since it involves a repetition of processes: data collection, analysis, creation and validation of theories, restarting the process with new data collection in an iterative process of data assembly, creation new hypotheses, further reading and new iteration until saturation of categories. For Strauss and Corbin (1990), at an early stage the researcher must select a group where it can see evidence of the phenomena studied, and start using methods for data collection, such as observation, interviewing, documentation, audio recording, video recording or combinations of them. These decisions are based on what the researcher considers that suits best their objectives, and may at any time be adapted and redirected, if the development of the study requires so. The Grounded Theory is a research method, that doesn’t start from a hypothesis or theory. A Grounded Theory‐based research often starts from a bad structured problem, which is being continuously structured along the development of the research. It is therefore important that the researcher does not focus its research and has no preconceived ideas when collecting and analyzing data (Robinson et al., 2004).

228

Eurico Lopes, Luis Barata and Ana Barata

2.2 Participants This research focused on students at the Senior University of Castelo Branco (USALBI), enrolled in the computer classes.

Table 1: Distribution of interviews Level I

Level II

Level III

Teachers

Total

Men

Women

3

4

1

3

11

8

3

The students can be divided in three groups in what concerns the motivation to learn computing:

Those who want to demystify the computer, they know what it is, know what it does, but do not know how it works.

Those who want to understand their grandchildren and the world that surround them, not feeling completely apart of their reality.

Those who use the Internet as a means of communication with the ones that they care and with grandchildren, who are distant, seeking in the Internet the ability to bring them closer.

Figure 1: Work organization The students interviewed are in an age range from 54 to 70 years and his academic background is quite diverse, some students attended school until the old 4th grade, while others have completed higher education courses. The same heterogeneity is true regarding to their occupation before retirement: businessmen, teachers, postmen or electricians. Glaser and Strauss (1967) define this strategy as theoretical approach, which aims to “representativeness of concepts” rather than the representativeness of the sample. According to Strauss and Corbin (1990), a researcher initially choose sets of situations and people in the study area and follows the analysis using samples based on the concepts that emerge as most relevant. In this method, sampling and analysis, are processed almost simultaneously, being the sampling directed according to the results of the analysis.

2.3 Observations Complementary observations were made of classes, with the aim of giving the start to the "immersion in the study site” (Janesick, 2000) and an adaptation of students to the researcher. This methodology was applied so the results to be found were developed as a product of this research, in fact, grounded in reality (Glaser & Strauss, 1967; Strauss & Corbin, 1990), thanks to the direct contact and interaction reality of students with computer’s use. The direct contact between the researcher and the students, in their daily environment, provides an observation “in the natural context of occurrence, among the actors who would naturally be

229

Eurico Lopes, Luis Barata and Ana Barata participating in the interaction, and follows the stream of everyday life.” (Adler & Adler, 1998:81), approaching the researcher with the reality of the facts. Based on these assumptions, the process of class observation was made, so that does not create interference with the normal operation, seeking above all, to see how the students interacted with the computer and technical difficulties that were finding. The items collected were notes that were later used in forming the interpretation of the study results.

2.4 Interviews Open interviews were used in order to know how the students interact with the computer, either in class or in their everyday. It was the most effective way to understand how they view the world of computers technology. The informality of the interviews along with the environment, which tried to be relaxed, allowed students to talk about their experiences, without any limitation or prejudice. The interviews also approach the students to the research, making them feel that their opinions and ideas were central to the development of this work. Interviews were open, so there wasn´t any interview guide, since its construction and follow‐up, would influence the respondents to the ideas implicit in the questions or issues to be analyzed. Therefore, and to avoid pre‐conceived ideas, the questions asked only served to link the ideas and to explore some of the points raised.

3. Ethical and privacy issues During the fieldwork, ethical principles, such as voluntary participation and confidentiality were a guarantee.

Before each conversation and / or approach, the objectives of the research were explained to each individual.

Participants were informed that their name or personal information would not be disclosed. Each one of them received an Informed Consent Form, with the objectives of the work and the conditions of confidentiality of their participation. In the interview process, none of the participants was pressed for an answer, having been informed that they could refuse to answer any questions. This kind of situation never occurred, since interviews were conversational, which made the participants very comfortable. All interviews were conducted with the consent (and collaboration) of the institution (USALBI). In classroom observation, before the start of each lesson, the teacher explained the reason for the presence of a person outside the group, a presence that was never challenged by any of the students (in fact was much appreciated). To avoid disrupting the normal functioning of the institution, the interviews were recorded at hours that students did not have any kind of activity, and wasn´t made any kind of pressure for students to attend. It was guaranteed the privacy of respondents, given the fact that in this work there isn’t any information that identifies them. It was also mentioned that at any moment they could abandon the interview by withdrawing its consent and this wouldn’t result in any kind of prejudice in his relationship with the researcher, or the institutions involved.

4. Grounded theory analysis The Grounded Theory Method was followed to develop a grounded theory that explains the older people motivations and interests in learning computers. Following next is a description of the theoretical procedures that were adopted during data collection and analysis.

4.1 Highlight of important Ideas Based on the transcript, we proceeded to multiple readings, emphasizing in each the most important aspects relevant to the research. This analysis allowed the identification of some common points between the experiments and experiences with the computer by the students. To represent the importance of these ideas they have been marked by highlighting the text.

Figure 2: Sample annotated transcript

230

Eurico Lopes, Luis Barata and Ana Barata In order to group the ideas of the first set of interviews, the main ideas were putted into small "Post‐it's" in an easy way to organize by concepts, and then joined the common ideas of the various interviews.

Figure 3: Main Ideas of the student interview

4.2 Grouping of common ideas The most important ideas were organized at the end of the analysis of individual interviews and several concepts could be grouped.

Figure 4: Grouping of ideas from interview

4.3 Coding of interview (key concepts) In parallel, in order to verify the main ideas of each interview, it was conducted an individual encoding, using the method of Coding Key Concepts, which, as regards Rodrigues et al. (2004), allowing meaningful concepts arise from the transcripts.

231

Eurico Lopes, Luis Barata and Ana Barata

Figure 5: Coding student interview After this review process was created an elaborated coding table, containing all the key concepts found in all the interviews (Rodrigues et al., 2004).

Figure 6: Table of keywords obtain from coding The data in this table were crossed with the analysis made to each of the interviews, seeking ideas, aspects and common items.

Social Contact

Interface

Support Simplification

Quality aging

Interface

Figure 6: Review of the first interview after the process of encoding Interviews reviewing process allowed the validation of the initial analysis grouping of ideas, which together with the data by coding the key concepts were the basis for the formulation of theories from this work.

232

Eurico Lopes, Luis Barata and Ana Barata

5. Elders and computers: The grounded theory The analysis revealed eight categories, each of which is the result of the grouping and combination of several key concepts, resulting from the coding of all interviews. These are:

5.1 Communication The communication category includes all the concepts related to computer usage and ICT’s, as a way of keeping in touch with family and society outside. The computer is viewed as a form of communication and interaction with others, which helps in the process of fighting the loneliness, which increasingly takes place in the elderly, not only because of being able to communicate from home, but also by having to leave their residence to attend classes, allowing insertion of a person in a group of peers. Students use the Internet and electronic mail to exchange information with distant relatives, in a way that allows them to track their life, creating a kind of virtual or digital presence.

5.2 Information It appears that the older use the computer and the Internet as a way of keeping up to date, about various subjects. They are the ones that make the selection of what they read or access. Being the information focused on their needs and sensitivities, this is for them an advantage because they do not have to be guided by any editorial policy of a newspaper or TV newscast. Accessing information and the beginning of any internet browser interaction, generally starts from Google. Students find the simplicity of Google an advantage in the process of finding information, combined with the fact that they can quickly find the information they want.

5.3 Learning All the students want to master a given task with the computer. It can be verified that besides the need to learn to use the internet as a mean of communication, they also feel the need to be updated, in order to understand the experiences and their grandchildren. This category gathers all the concepts about how the older capture the knowledge of IT, and how this knowledge is transmitted.

5.4 Motivation This category represents the will and commitment that all individuals demonstrate on learning computers, which is essential in the ability to acquire knowledge and concepts about the interaction with computers. The motivation is the factor that triggers the need to learn more about technological artifacts. The need of being socially active (social contact) and the need to understand the world of their grandchildren, is the factor that approach elders to the new technologies, and mainly the Internet is a way to communicate with the ones that they care and as a way to overcome the distance between them.

5.5 Digital inclusion Digital inclusion promotes the ability to think, produce and share knowledge with the society, being the main reason why elders are interested in learning computers. The fact that they feel overcome creates a certain discomfort, making them feel a bit apart from the society. There is a change of roles, especially in the way that they interact with their grandchildren and technology. There is also an inversion on role of the teacher (the transmitter of knowledge) and the student (the receiver of knowledge). The role of the teacher that was given to the elders, because of their life experience, is now given to the grandchildren as they teach and help their ancestors to work with computers and deal with technology.

5.6 Simplification Simplification is a concept found in the coding of all interviews. For the students, the idea of the computer to simplify their everyday is an acquired truth. Also refer to the fact that more and more services become available over the Internet.

233

Eurico Lopes, Luis Barata and Ana Barata

5.7 Social interaction and contact This item represents communication, contact and relationship with those around them. Social contact is an important aspect with regard on aging in these days. Using computers and the Internet is one way of living in society and not falling into the scourge of loneliness, which is seen by everyone with a huge fear.

5.8 Fear This category represents two ways of fear that afflict the elderly in interaction with computers. The first relates to the fear of using the computer, which is one of the factors inhibiting the use of technologies. The second shows the fear of information accessed from the computer, mainly from the internet and widespread in the mail.

6. Elderly and computers: GTM framework This framework describes the relationships between the categories obtained. In the epicenter are the elder people that as we have seen, have the need to feel integrated into society, trying to overcome the problem of loneliness in the elderly. The need for social contact and interaction generates the motivation to upgrade their skills and knowledge and to be part of the Digital Society. Allowing elder not to feel outdated will improve their self‐esteem, and by other hand will allow a better relationship with those around him, especially his grandchildren, who are already more advanced technologically, allowing an easy approach to them. These factors make older people feel the need to dedicate themselves to the technology and computers learning, trying to develop the abilities to communicate, using the computer as a form of communication. This is also a way to access information (knowledge) of various levels (new services, new knowledge or contents) to simplify their day‐to‐day.

Communication is based in Internet access, which is the main reason that takes students to learn computer’s, as they see the internet as a way of communicating with those around him. Communication can also be seen as a way of being socially active, and as a form to transmit the knowledge acquired throughout the life. The simplification is mentioned by all respondents, whether as a result of the application of computing on the day‐ to‐day, either as a necessity, being defined as a key category for the development of emerging theories. For the same reasons fear was also considered a key category, by the fact that many students often refer concerns about access to information, especially online, with access to communication tools such as web or email.

Figure 7: Framework derived from the GT analysis

234

Eurico Lopes, Luis Barata and Ana Barata

7. Discussion 7.1 Difficulties in the use of computers It appears that initially, the greatest difficulty for students who have never had contact with computers is fine dexterity. This is the first barrier they have to overcome, especially in the use of the mouse. Interestingly, about half the students do not report the mouse as being a problem, just report difficulties at an early stage of its use. Therefore we can say that over time the mouse is no longer a barrier/obstacle for seniors to use the computer. The field work supports the idea that the use of computer resources is a benefit from a biological standpoint, since the computer enhances the stimulation of reasoning and memory, while also allowing greater flexibility in their hands, thanks to training that is done to muscles. By participating on class, students battles the scourge of loneliness, a subject that is so awake and present in our consciousness, motivated by more recent experiences of loneliness in elderly people. All this learning process leads to the use of ICT as a way of socialization, and a way to integrate individuals in active society through the use of the Internet. The use of the computer and the Internet encourage the reading, the thought and the concentration on what they read, thus promoting the interaction with everyday events.

7.2 Fear of the Use of information and communication The information and communication access generates in elders some fears that should be fought. The technophobia (fear of modern technology), was easily incorporated into the list of stereotypes associated with older people. This new stereotype is based on the belief that older people are adverse to new technologies, in a conscious and deliberate way, and not as the result of the historical path and constraints that have dictated its info exclusion. There are a number of difficulties that reduce the success that elders have in the learning of information technology:

They hesitate initially, with fear of damaging the machine;

Lack of expertise, as the result of the decrease in motor skills;

Fear of making mistakes;

Fear of failing to learn a new thing.

Besides the fear in using computers, there is the fear of the Internet. Students have the idea that Internet is a source of fraud, and they are the ideal targets, so all activities made online, are made with great fear. This fear is due to the fact that the Internet is a concept that is not visible, has no face: students do not know who is on the other side of a site or an e‐mail. This lack of personification makes the breaking of the trust relation, a situation that is very important to them.

7.3 Simplification of information and communication Another major challenge that arises is how to teach elders in the use of computers. How can the way that a technologically complex machine works, being explained to someone that during his life experience had no contact with information technology? The students reported a great difficulty in perceiving everything that surrounds the computer technology. Each one, on its relationship with the computer, is building a way of thinking and interacting with the machine, which is the result of a process of growth of knowledge over time.

The individual ends up forming his own awareness of computer technology, creating mental processes that allow them to perform actions and interact with the computer. The major difficulty in the learning process is the fact that, teachers need to forget the mental processes that we created, and try to create them in conjunction with the student. It should also adapt the contents to cover the student’s needs. A student at an early stage of their learning needs to demystify the computer, not giving much explanation about hardware or software.

This whole process of learning and demystification is very important, since it depends on the creation or elimination of phobias. In the same way that learning in old age must follow several rules, also the learning of computers must follow these. It’s important to realize that each person has their own learning speed, so the transmission of knowledge must accompany the student's needs, and not just any hard lesson plan. It was also mentioned the need for more hours per week in order to enhance learning, thus creating more training days and time with the computer. Also the amount of information that appears on screen is a difficulty for students, so it is important to explain how the information is divided on the screen, and how we interact with that view.

235

Eurico Lopes, Luis Barata and Ana Barata It is therefore important to simplify the way it communicates and accesses information. Therefore, there is an urgent need to simplify the use of computers and information technology. The simplification of computers usage, could assume the development of physical interfaces adapted. This has not occurred in the observation of the target studied, there was rather a need to simplify the process of interaction. For them, the need is not to create adapted interfaces (voice commands, no mouse, minimalist interfaces, among others), but to adapt existing ones so that they can use the computer as a tool, which enables the simplification of day‐to‐day tasks.

8. Future work This study is a beginning to research the relation between computers and elderly people. It´s an important to‐ do research work extending the study to other kind of population, such elderly people with less education, since most studies related to aging and technology is associated with older and younger people with more years of study. The future work must also identify applications that elders are more suitable to use.

Complementary research must also be done in the study information technology curricula, building a virtual environment for study and self‐study, to allow the response to several questions that students have, without a teacher around.

Acknowledgements The authors would like to thank the collaboration of all the students that participated in this research. We also acknowledge Mr. Arnaldo Brás, councilman of CMCB, for allow the research in USALBI, and finally to all the teachers.

References Adler, P. A., Adler, P. (1998). Observational techniques. In: Denzin, N. K. & Lincoln, Y. S., eds.1998. Collecting and interpreting qualitative materials. Thousand Oaks, CA: Sage, pp.47‐88.

Berger, L., Mailloux‐Poirier, D. (1995). Pessoas Idosas – Uma aborgagem global. Lisboa: Editor Lusodidacta.

Carmo, H., Ferreira, M. (2008) (2ª ed.). Metodologia da Investigação – Guia para auto‐aprendizagem. Lisboa: Universidade Aberta.

Fernandes, A. A. (2008). Questões demográficas ‐ Demografia e sociologia da população. Faculdade de Ciências Sociais e Humanas da Universidade Nova de Lisboa. Edições Colibri.

Fernandes, P. (2002) (2ª ed.). A Depressão no Idoso. Coimbra: Quarteto.

Fonseca, A. M. (2005). Desenvolvimento humano e envelhecimento. Lisboa: CLIMEPSI Editores.

Fontaine, R. (2000). Psicologia do Envelhecimento. Lisboa: Climepsi Editores.

Glaser, B., Strauss, A. (1967). The discovery of Grounded Theory. Chicago: Aldine Publishing.

Jacob, L. (2007) (3ª ed.). Animação de Idosos. Lisboa: Âmbar.

Jacob, L. (2008). O idoso e a comunidade ‐ respostas sociais. Accessed in the World Wide Web, at May, 7th 2010. Available at: http://www.portaldoavo.com.pt/artigosa.php?id=2.

Law, J. (1992a). Notes on the Theory of the Actor Network: Ordering, Strategy and Heterogeneity. Lancaster: Lancaster University, Centre for Science Studies.

Law, J., (1992b). Notes on the Theory of the Actor‐Network: Ordering, Strategy and Heterogeneity. Systems Practice 5, pp.

379‐393.

Lopes, E. (2010). A Grounded Theory of Decision‐Making Under Uncertainty and Complexity. VDM Verlag Nazareth, J. M. (2009). Crescer e envelhecer ‐ Constrangimentos e oportunidades do envelhecimento demográfico. Lisboa: Editorial Presença.

Nielsen, J. (2000). Designing Web Usabitity: The Practice of Simplicity. Indianapolis: New Riders Publishing.

Rodrigues, A. et. al. (2004). Grounded Theory: Problemas de “Alicerçagem”. Accessed in the World Wide Web, at October, 23th 2010. Available at: http://arodrigues.alfarod.net/docs/articles/ 2004_GroundedTheory.pdf.

Straus, A., Corbin, J. (1990). Basics of qualitative research ‐ Grounded Theory procedures and techniques. London: Sage.

236

Towards The Operationalisation of the Socio‐Political Utilisation of Research Findings in Business and Management Studies Hendrik Marais Department of Engineering and Technology Management, Graduate School of Technology Management, University of Pretoria, Pretoria, South Africa hc.marais@up.ac.za marais.stnetwork@gmail.com

Abstract: This paper reports the results of a project on the development of an empirically oriented methodological framework for measuring the socio‐political utilisation of research findings. The project is located at the interface of three empirically supported arguments. First, science policies increasingly emphasise the need for evidence of the value of research, also in business and management studies. A systemic approach that includes the complete life cycle of a research project, is further required for the optimisation of the utilisation process. Finally, there is a growing body of research on the refinement of utilisation indicators, e.g. bibliometric analyses as evidence of the scientific utilisation, and patents as indicators of the techno‐economic utilisation. Providing convincing evidence of the socio‐political utilisation of research findings, however, poses vexing research methodological challenges such as the lack of standardised indicators. The paper describes the development of a systemic framework for the empirical measurement of the socio‐political impact of research. The framework consists of five differentially weighted sub‐systems: the nature of knowledge production; communication; interaction profiles; characteristics of the environment and beneficiaries; and the source and form of evidence. Utilisation dimensions and associated attributes are further linked to each of the five sub‐systems as rank‐order rating scales. The framework also includes guidelines for the operationalisation of these sub‐systems and suggests how these could be presented as quantitative records of socio‐political impact. The paper argues that the application of the framework should be as cost‐effective as the average publication process required for a scholarly article and less than the average time for the registration of a patent. The paper concludes by identifying some of the implications for the institutional management of research portfolios and the incorporation of socio‐political utilisation in research reward systems (e.g. appointments and promotions). The empirical validity of the framework will be tested in the final phase of this research programme.

Keywords: project management, methodology, research utilisation, valorisation, research impact

1. Introduction All the layers of most national research systems are increasingly, directly and indirectly, required to provide evidence of commitment to society and the value of their research in that context. Examples include international and national systems (e.g. Barker, 2007; Department of S&T, 2012), research councils (e.g.

Arnold, Knee, Brown et al., 2012), disciplines and programmes (e.g. Otronen, 2004; National Research Foundation, 2012) and individual research projects funded by public research funding agencies (cf.

Netherlands Organisation for Scientific Research, 2009). The result of these developments is that researchers, in applications for funding, have to respond to a ‘new generation’ of items about the potential societal value of their projects, in addition to the standard items on research design, methodology and their own track records.

Factors responsible for this broadening of the evaluation base include the emergence of performance auditing, growing control and steering by governments, financial constraints, the internationalisation of research, and growing prominence of transdisciplinary research.

In the context of the present conference, the question becomes, ‘How do researchers in business and management studies prove – in a reliable and valid way – the societal value of their research?’ The paradox, however, is that most institutions do not have standardised systems in place to evaluate the validity of such evidence, or reward systems for researchers in this regard, that are comparable to the systems for measuring, evaluating and rewarding traditional publication outcomes. This paper offers the operationalisation of a measurement framework for generating relatively reliable and valid evidence of the socio‐political impact of research findings. The framework also represents a set of guidelines for researchers to improve the likelihood of the effective implementation of their research findings.

This paper is the fifth in a series on the utilisation of research findings. The first paper reported the results of a survey on the utilisation of dissertation research findings (Marais et al., 2010); the second introduced the concept of effective research and dealt with the importance of planning for utilisation (Marais and Pienaar‐ Marais, 2011); the third discussed the management of research to improve utilisation (Chan, Marais and

237

Hendrik Marais Gathua, 2011); and the fourth addressed the role of institutional policies for the promotion of utilisation (Marais, Chan and Gathua, 2012).

This paper covers the following topics: objectives, core arguments, key concepts, theoretical orientation, a systemic framework of the socio‐political utilisation of research findings, proposed procedure and conclusion.

2. Objectives There is growing pressure on researchers to provide evidence of the potential, intended or realised utilisation of their research. There are reliable, relatively valid and ‘objective’ indicators of scientific and techno‐ economic utilisation (return on investment), but not for socio‐political utilisation (see section 4.1.3). The objective of this paper is to develop and present, firstly, a framework for the relatively reliable, valid and ‘objective’ operationalisation of socio‐political utilisation of research findings and, secondly, to demonstrate how such a framework could be applied at institutional level to acknowledge contributions by researchers to the benefit of society. (The framework could also serve as a plan for the design of research projects with a view to optimising the likelihood of utilisation.)

3. Core arguments The paper is developed around the following arguments:

Measuring the socio‐political utilisation of research is generally more challenging than measuring other forms of utilisation, since it is a more complex and variable process.

The entire research value chain has to be accounted for in developing indices to measure the utilisation of research findings in general and socio‐political utilisation in particular.

The measurement of socio‐political utilisation is not necessarily more time‐consuming than the production process leading to evidence of scientific and techno‐economic utilisation; the framework could also serve as a plan for designing institutional capacity for monitoring and evaluating research utilisation.

4. Key concepts A relatively wide range of concepts are used to refer to the uptake of research findings, e.g., utilisation, implementation, exploitation, effectiveness, valorisation and impact; brief descriptions follow:

4.1 Utilisation or implementation of research findings The concepts of utilisation and implementation refer to the use of research findings by users, beneficiaries or stakeholders, whether planned or unplanned, and directly or indirectly (cf. the ‘accidentals’ in the typology of the European Research Council, 2012: 3), within the first few years after completion of the project (cf. Bailey and Mouton, 2005; National Advisory Council on Innovation, 2003). ‘Utilisation’ does not specify in advance the nature of the use being made of the research findings, although it does suggest a timeframe of five years for the following reasons: the assumed acceleration in research output, the diversification of media (e.g. e‐ publishing), the rapid aging of information, the mobility of researchers, and the timeframe of performance reporting expected of institutions and individual researchers. At least three forms of the utilisation of research findings are often distinguished in the literature (see Bailey and Mouton, 2005, for an overview), namely:

Scientific utilisation, i.e. the advancement of the body of scientific knowledge (e.g. a scholarly article or a citation) ‘is probably the most studied in the literature’ (Godin and Doré, 2003: 2) and is normally operationalised by means of bibliometric analyses. Often overlooked is the time lag between the completion of a project and the publication of an article or book – an average of between 18 and 24 months? Of course, even more time has to be allowed if the measurement focuses on citations.

Techno‐economic utilisation, i.e. the conversion of research findings into economic benefits, including technology; a common operationalisation is the analysis of patents and licences. While recording a particular indicator, such as a patent, is obviously a rather simple measurement exercise, it should be borne in mind that five years is a reasonable estimate of the time lag between the completion of a project and the registration of a patent.

Socio‐political utilisation entails the uptake of research findings by society in modes other than scientific and techno‐economic. A few randomly chosen examples can illustrate the range: the implementation of the results of an evaluation of a particular policy; the utilisation of results of research on skills training in a

238

Hendrik Marais deep‐rural community; the implementation of the results of a project to accelerate technology transfer; research‐based contributions to the development of a new democratic constitution for a country; and the utilisation of social‐behavioural research results in the development of an anti‐HIV‐AIDS strategy.

In recent years, several international initiatives have addressed the challenge of operationalising the value of science in general, and the impact on society in particular, e.g. ERIC (Evaluating Research in Context; http://www.eric‐project.nl), SIAMPI (Social Impact and Assessment Methods for research and funding instruments through the study of Productive Interactions between science and society; www.siampi.eu/) and the European Parliament’s commissioned study on knowledge transfer from public research organisations (Arnold et al., 2012). Organisations such as the Dutch Academy of Science, the Netherlands Organisation for Scientific Research (2009), the UK Overseas Development Institute (2004) and UKAID (Grobbelaar and Kirkland, 2013) are particularly active in this field. Various international projects indirectly address this issue, e.g. the South African Human Sciences Research Council (cf. Kruss, 2012).

The broad range of examples offered earlier suggests that there would not be a relatively standardised indicator or indicators that could be used as reliable and valid evidence of such impact (cf. Spaapen and Van Drooge, 2011; Young, 2005), although a bibliometric approach has been suggested for certain types of such utilisation (cf. Nederhof and Meijer, 1995). At least four reasons can be offered for the lack of progress in the operationalisation of socio‐political impact (also see Banzi, Moja, Pistotti et al., 2011; Spaapen and Van Drooge, 2011):

Firstly, the research findings may take a complex and even unpredictable route to implementation ‐ an ambiguous relationship between research findings and the form of utilisation.

Secondly, as Godin and Doré (2003) point out, such impact ‘is diffused in space and time’ (2003: 3).

Thirdly, the range of outcomes varies.

Fourthly, the lack of agencies that undertake the operationalisation (e.g. Thompson Int.), require individual researchers, institutions or funding agencies to develop their own systems for ascertaining utilisation, if any.

It is suggested that the time required to operationalise the socio‐political impact measures developed in this paper would be comparable to the time required for the production and measurement of scientific and techno‐economic utilisation.

It should be noted that the renewed emphasis on socio‐political utilisation of research findings has been stimulated in part by the challenges posed to the social sciences and the humanities to provide evidence of the return on investment by them (see, e.g., Royal Netherlands Academy of Arts and Science, 2005).

4.2 Related concepts Exploitation is often used interchangeably with utilisation of research findings. Effectiveness of research (Marais and Marais‐Pienaar, 2011) refers to research that has succeeded in attaining the goal that the researcher originally set in the project plan. Valorisation of science, in the broader sense, is used to denote the entire value chain from the first ideas to the actual contribution to the benefit of society. Research impact is often used interchangeably with the above concepts, but in its narrow meaning refers to, or requires, the use of the research findings as a specific measureable outcome.

The relationship between these and related concepts is shown in Figure 1.

5. Theoretical orientations The process of utilising research findings is manifest in a range of theoretical and empirical orientations, including the diffusion/communication of innovation (cf. Rogers and Shoemaker, 1971), the knowledge cycle (cf. Rich, 1981) and technology transfer (Bozeman, 2000: 642–643); while the field of transdisciplinary research relies on the concept of utilisation (cf. Klein, Grossenbacher‐Mansuy, Häberli et al., 2001); and sections of doctoral research pedagogy that promote ‘translational skills’ i.e. skills ‘necessary to translate research findings into societal applications’ (Nerad, 2011: 203).

239

Hendrik Marais Effects; also uptake

Need

Research

Optimisation of conditions (cf Table 3)

Utilisation Adoption Exploitation

Impact

Appropriation Implementation

Valorisation of research Time & quantifiability

Figure 1: Relationship between concepts associated with utilisation These five and related orientations suggest the spectrum encountered in the literature – some differ fundamentally, while others differ only in minor respects or terminology. A common thread running through these orientations, however, is that utilisation is conceived as a process extending from the production of a research finding, through its conversion into what Bozeman (2000) calls a ‘transfer object’, to the eventual adoption by the intended or other beneficiaries.

Reviews of the literature, such as Bailey and Mouton (2005) and the National Advisory Council on Innovation (2003), have shown that the utilisation of research findings is, on the one hand, understood as a system involving all phases of the research process, from the initial stages of project planning through to the identification and nature of the intended or unintended beneficiaries of that research – and, of course, the social systemic dynamics of adoption itself. (The concept of ‘system’ is here understood to have properties commonly ascribed to an open system, such as interconnectivity between subsystems, feedback, equifinality [the principle that a given end state can be reached by different means], multiple causation, hierarchy, interdependence, etc. (cf. Stacey, 1996; Von Bertalanffy, 1968). A conceptual point of departure for developing a systems‐based framework of the utilisation process would be to use a model of research communication.

Bozeman’s (2000) contingency effectiveness model of technology transfer resembles the original systems model of communication in many respects (e.g. Berlo, 1977). Bozeman’s model consists of the transfer agent (sender), transfer media (medium), transfer object (message), transfer recipient (receiver), demand environment (environment or social context) and effectiveness indicators. However, from the perspective of the utilisation of research findings, the Bozeman model implies, but does not explicitly include, what could be called the antecedents of the transfer object (e.g. technology). ‘Antecedents’ refers firstly to those variables that precede or form part of the production of research findings; secondly to the execution of the research itself (these two aspects roughly equate to what Van der Meulen, 2007, called ‘insight in the dynamics of research’); and finally to the research product, namely a report reflecting the findings. There is empirical evidence that would justify the inclusion of these antecedents as variables that influence the likelihood of all three forms of effective utilisation of research findings, e.g.:

Including the topic ‘the utilisation of research findings’ in research methodology training was found to improve the likelihood of the utilisation of dissertation findings (Chan et al., 2011).

The importance of the project/disciplinary field, project planning and the origin of the project have been shown to be predictors of utilisation (National Advisory Council on Innovation, 2003).

The inclusion of recommendations in a dissertation significantly influences the likelihood of utilisation (Chan et al., 2011).

Research on the diffusion of innovation, knowledge utilisation, and technology transfer confirms the importance of the dissemination or communication process, the role of intermediaries, characteristics of the intended beneficiaries and the environment in the utilisation process.

240

Hendrik Marais Against this background, a model of the utilisation process therefore has to incorporate ‘antecedents’ to capture the dynamic and systemic nature of the process. Broadening the model of utilisation makes even more sense in considering the complex nature of the socio‐political utilisation of research.

6. A systemic framework of utilisation of research findings In Figure 2 the antecedents are reflected in the interactive orientation of the researcher(s), the project plan, the execution of the research and, partially, in the research report, while the communication of the findings and the demand environment (cf. Bozeman, 2000) are normally represented as the main components of the utilisation process.

Figure 2 represents a logical framework of the process of research culminating in the findings being utilised.

The key attributes of each variable are given in italics and elaborated in Table 3. The following notes on the utilisation process may be useful:

The internal organisational environment (e.g. facilitating institutional research policy) influences the utilisation performance of individual researchers, groups and the institution (e.g. Van Geenhuizen, 2010, who found that it accounted for 31% of the obstacles to successful valorisation).

The seniority and connectedness of researchers are important predictors of the successful utilisation of their findings, other things being equal; hence the link between researcher and intermediaries; cf.

Spaapen and Van Droogen’s ‘direct interactions’ (2011: 213; and Van Drooge and Van der Meulen, 2009: 6).

Ideally, the intention to utilise research findings should start during the conceptualisation of a research project: what is the subject and why is the project conceived, e.g. self‐initiated or commissioned, and to what end?

Internal organisational environment

Project

Execution

Researcher Seniority, networks Nature of project Subject, Origin

Project plan Planning for utilisation

Research Quality

Report & communication

External environment

Project report Transferability Utilisation object Communication of the findings Media

Beneficiaries Need Orientation Intermediaries Facilitators Brokers Utilisation Type Impact Time lag

Figure 2: A systemic framework of the process of utilising research findings

For methodological, professional, ethical and empirical (see Young, 2005) reasons, it is assumed that the utilisation of research findings should refer to research of acceptable quality.

A project report should communicate effectively, which means that the findings, immediate implications and recommendations should be appropriately converted and clearly reported (Godin and Doré, 2003; Royal Netherlands Academy of Arts and Science, 2005; Young, 2005).

Appropriate and effective communication of the findings is often neglected. Communication is more than simply sending an academic report to possible users (e.g. business CEOs, executive officers of development agencies, etc.) and expecting more than a brief acknowledgement of receipt. (The term ‘communication’ is preferred, since ‘transfer’ – see, e.g. Bozeman, 2000 and Godin and Doré, 2003 – may suggest that the research findings have been neatly packaged.)

241

Hendrik Marais

Intermediaries or information brokers are increasingly involved in the technical communication process, ranging from internet consultants to communication specialists assisting academics to prepare for television interviews on their research – or university innovation officers. Generally, they are well informed on both the production and use of knowledge and have incentives, e.g. commission for connecting producer to user.

The intended beneficiaries of research findings, e.g. members of a deep‐rural community hoping to increase their crops, are by definition most important (cf. Spaapen, Dijstelbloem and Wamelink, 2007; Young 2005).

The type of intended utilisation is an essential component of the process. A general rule: the more focused the utilisation object, the more specific the beneficiary and the lower the cost, the more likely the research findings are to be utilised.

Finally, socio‐political impact is not restricted to the outcome of the process, but is intrinsically part of the interactions, i.e. the vectored linkages in the figure, (or ‘productive interactions’, according to Spaapen and Van Drooge, 2011). In other words, these interactions between a research team and brokers or intended beneficiaries, for example, could qualify as socio‐political impact.

6.1 Operationalisation of the dimensions Table 3 offers a summary of the sub‐systems, dimensions and their attributes (Figure 2) that relevant literature has shown to enhance the likelihood of effective utilisation. This tabulated summary can be converted to a measuring instrument. For the sake of quantifiable measurement, suggested values have been allocated to each attribute as rank‐order scales. Of course, the selection of attributes and their values can be contested; there is also the chance that a piece of research that does not meet any of these criteria could, over a longer period, directly influence the socio‐political environment.

Table 3: A systems approach to measuring utilisation of research findings (URF) Sub‐systems

Knowledge production (KP)

Dimension of utilisation Researcher Project plan Research theme Origin of project Type of research Quality of research Appropriate conversion Utilisable object

Brokerage Beneficiaries Intermediaries Other Demand environment

Attribute of dimension (and value) Senior/networked (2); Inexperienced (1) Utilisation planned (2) Stakeholders involvement (2) No provision (0) Relevance/salience/prominence (5) Not (0) Self‐initiated (1) Commissioned (5) Fundamental (1) Strategic (2) Applied (3) High (2) Mediocre (1) Explicit (implementation plan) (5) Vague (3) None (0) Visible (3) ‘Simple (2) Abstract (1) Recommendations (2) Further research (0) Academic media(1) Public 2) Networks (3) Experienced (2) Not appropriate experience (1) No broker (0) Direct (4) Indirect (2) Direct (3)Indirect (1) Direct/indirect (1) Existing demand (5) Potential demand (3) New (2) Abstract (1)

Beneficiaries

Entrepreneurial (3) Traditionalists (1)

Source Realisation

Researcher opinion (1) Third party opinion (2) Documented (3) Expected (1) Realised (5)

Report

Communication (C) Inter‐ action Profiles (I) Receiver; environment (R) Evidence: Nature (S+RU)

The information of Table 3 is graphically summarised in the next figure.

The following notes elaborate on information in Table 3.

6.1.1 Systems approach The components of the utilisation system (Figure 2) have been regrouped at a higher order to produce four sub‐systems, namely knowledge production, communication of the findings, interaction profiles and characteristics of the receiver/beneficiary in the environment. A critical factor that still has to be added to this emerging framework is the substantiation of the evidence on the attributes listed in Table 3. In the case of scientific utilisation, a researcher ‘simply’ needs to submit as evidence the full reference of a published article

242

Hendrik Marais

Multiplier Source of evidence Realisation of utilisation

(plus the journal’s impact factor). And ... in the case of the validity of evidence of socio‐political utilisation? As a first approximation, it is suggested that a two‐factor measure consisting of the source of the evidence and the extent of realisation of the utilisation be included in the framework.

URF Framework

Utilisation of research findings

KP D I R

Subsystem Knowledge Prod Dissemination Interaction Recipient

2 Dimensions Dimensions 3 Dimensions 8 Dimensions 2 Dimensions

Attributes Attributes Attributes Attributes Attributes

Figure 3: Summary of the utilisation measurement system 6.1.2 Source of evidence What is the evidence on utilisation? It could be oral opinion (either that of the researcher or a third party) or documented evidence. In Table 3 the following weights were given to this factor: Researcher’s oral opinion: 1; Third‐party oral opinion: 2; Documented evidence: 3.

6.1.3 Extent of realisation of utilisation Probably as important as the source of evidence is the extent to which the findings have been utilised.

Expecting research findings to be utilised, or being sure that this will happen, is not equivalent to being able to prove that this has actually been done. In this regard, one could probably compare the researcher in an annual performance report expressing the intention to submit a manuscript to a journal as opposed to providing the bibliographic information on the article. The following weights were provisionally allocated to this factor: Unsure: 0; Expected: 1; Realised: 2.

6.1.4 Differential weights In line with the important properties of open systems, namely hierarchy and differentiation, differential weights are allocated to the different attributes of the sub‐systemic dimensions. It is argued, for instance, that documented and empirical evidence of utilisation would be ‘worth’ more than an oral opinion on whether utilisation has taken place (or even whether it is likely to take place), hence a weight of 3 for the documented evidence, 2 for oral evidence by a third party and 1 for oral evidence by the researcher.

6.1.5 Utilisation equation The argument so far suggests the following equation for calculating the evidence of socio‐political utilisation:

URF=(KP+C+I+R)x(S+RU), where

URF is the likelihood or extent of the socio‐political utilisation of the research findings.

KP represents the sum of the knowledge production processes.

C is the sum of the communication elements.

I is the sum of productive interactions.

R is the sum of the receiver or beneficiary characteristics.

The sum of S (source of the evidence) and RU (extent to which socio‐political utilisation has been realised) serve as multiplicative weights in the equation.

However, notwithstanding the simple elegance of the model it clearly suffers of the following inherent problems. Firstly, the more dimensions a subsystem has, the higher its contribution to the final number and, secondly, the multipliers (Source of Evidence and Extent of Realisation) are applied to the composite raw number, but their applicability may vary among the attributes. To address these potential problems, different

243

Hendrik Marais weightings can be allocated to the nature of evidence (i.e. Source of Evidence and Extent of Realisation) to different dimensions, yielding the following equation:

URF = (w(1)KP* + w(2)C* + w(3)I* + w(4)R*)

Where

W(i), I=1,2,3,4 are weightings that reflect the relative value attached to the contribution of the score of each subsystem in Table 3 to the composite URF;

KP*, C*, I*, and RP* are the adjusted scores of the sub‐systems after the differentiated nature of evidence multipliers have been applied to the raw values of the dimensions that make up the sub‐systems

An informal trial in the application of both versions (equations) of the framework suggested their feasibility in evaluating the extent of utilisation of research findings. The next phase of this project will consist of formally assessing the administrative requirements and empirical validity of the framework by applying it to a range of completed projects.

7. Proposed procedure The first argument in this paper has been, firstly, that an open systems approach was required for the measurement of socio‐political utilisation ranging from the project conceptualisation of a research project to the socio‐psychological characteristics of the intended beneficiaries. The second underlying argument has been that providing evidence of socio‐political utilisation does not necessarily ‘cost’ more than, e.g., bibliometric evidence of scientific utilisation, although the process would have to be driven by funding and research institutions.

In light of these two arguments, this section sets out a proposed procedure for the flow of the systemic process (Figure 2) and its attributes and values of the sub‐systems (Table 3).

There is no internationally accepted system for evaluating the socio‐political utilisation of research findings, although some countries are making progress towards developing national systems, e.g. the Netherlands (Spaapen et al., 2007). It is suggested that the proposed framework could be used as the basis for acknowledging the socio‐political utilisation of research findings in a quantifiable way.

The standard administrative procedures include an evaluative framework consisting of procedures, a moderating mechanism, a feedback system and a database for recording outcomes. Of course, the nature of the various components of the evaluation framework would be specific to the institution or agency concerned.

Notes on the application procedure, sources of evidence and moderating mechanism follow below.

7.1 Application procedure It is suggested that the assessment of the likelihood or extent of utilisation of any particular research report should be done by means of a measuring instrument that is standardised within an institution or agency and completed by the researcher. Table 3, listing the key variables and weights, could with relative ease be formatted as an application form or utilisation questionnaire. The purpose of such a questionnaire would be analogous to the methodological and publication guidelines used by the editorial boards of journals.

7.2 Sources of evidence Obtaining evidence remains one of the main challenges in ‘proving’ the socio‐political utilisation of research findings. In this regard, the US National Research Council offers a useful guideline: ‘One looks to the customers and stakeholders for an evaluation of the impact of a research program. Supporting this evaluation, an organization will put in place and use on a regular basis a systematic process of outreach to this clientele.

Polling by questionnaire and polling by interviews are alternatives’ (2012: 7). In the same vein, Spaapen and Van Drooge conclude ‘that the evaluation method includes all relevant stakeholders…’ (2011: 212).

244

Hendrik Marais

7.3 Moderating mechanism To evaluate the evidence provided by the researcher in the application procedure, a moderating mechanism akin to the review panels for funding applications or editorial boards of journals would be required. An utilisation moderating panel, consisting of a small number of knowledgeable and prominent individuals in the relevant field, would be required to apply the measurement instrument based on Table 3 and to audit the submitted evidence.

8. Concluding comments The primary purpose of developing the proposed systems orientated and evidence‐based framework was threefold: The first objective was to serve as a basis for the measurement of potential and realised utilisation of research findings. The framework consequently incorporated all the necessary – but not sufficient ‐ systemic sub‐processes involved in utilisation of research findings. This is in contrast to, for instance, the well‐argued model on types of outcomes of Godin and Doré (2003). Secondly, it can function as a plan for designing an institutional capacity for measuring socio‐political utilisation of its publicly funded researcher. Thirdly, it can also serve as a guide or check list to improving the chance of successful implementation from the time of the planning of a new project or programme.

The framework is certainly complex and would require more in‐house administration than the recording of bibliometric or patent information. However, the framework can serve an instrument to de‐mythologise the unspecified claims by some researchers of the effects of their research. The framework also affords researchers a relatively objective, reliable and valid approach to ‘proving’ the socio‐political effects of their research. It should be emphasised that the full measurement instrument obviously would not need to be applied if a researcher presents documented evidence of the realised utilisation of his/her research findings.

The framework can claim conceptual validity to the extent that substantiation is available in the literature for the role of the individual subsystems and dimensions in promoting utilisation. Empirical validation of the framework as represented in the equations will be sought in the next phase of the project by their application to a number of diverse research reports on the one hand, and practical feasibility on the other.

Acknowledgements The financial support towards this project by the Graduate School of Technology Management, University of Pretoria, is gratefully acknowledged. Stimulating discussions with the following colleagues in October 2012 helped shape my thoughts on the present subject: Jack Spaapen (Royal Netherlands Academy of Arts and Science), Stefan Kuhlmann (University of Twente), Erik Arnold (Technopolis), Phil Shapira (Manchester University and Georgia Institute of Technology), Barend van der Meulen (Rathenau Institute), Peter van Sijde and Peter van den Besselaar (Free University of Amsterdam). The contribution by Jos Grobbelaar towards the critical refinement of the utilisation equation is gratefully acknowledged. I also wish to thank Magdal Pienaar‐ Marais for her lateral inputs to my thinking on the subject. Robyn Arnold is thanked for her professional copy‐ editing of the paper.

References Arnold, E., Good, B., Ohler, F., Tiefenthalter, B. and Vermeulen, N. (2010). The quality of research, institutional funding and research evaluation in the Czech Republic and abroad. Prague: MSMT. Arnold, E. Knee, P., Brown, N., Jávorka, Z., Giarracca, F. and Sidiqui, S. (2012). Knowledge transfer from public research organisations. Strasbourg: European Parliament. (S&T Options Assessment) Bailey, T.G. and Mouton, J. (2005). A review of models of research utilisation. Stellenbosch: CREST. Banzi, R., Moja, L., Pistotti, V., Facchini, A. and Liberatti, A. (2011). Conceptual frameworks and empirical approaches used to assess the impact of health research: An overview of reviews. Health Research Policy and Systems, 9(26): 1‐10.

Barker, K. (2007) The UK research assessment exercise: The evolution of a national research evaluation system. Research evaluation, 16(1): 3–12.

Berlo, D. (1977). Communication as process: Review and commentary. In B.D. Ruben (ed.), Communication yearbook 1.

New Brunswick: Transaction Books. Pp. 11–27.

Bozeman, B. (2000). Technology transfer and public policy: A review of research and theory. Research Policy, 29: 627–655.

Chan, K‐Y., Marais, H.C. and Gathua, S. (2011). Theoretical and empirical perspectives on the management of R&D to improve the implementation of research findings. Paper presented at the annual ISEM conference, Stellenbosch, September 2011.

245

Hendrik Marais Department of Science & Technology (2012). Final report of the ministerial review committee on science, technology and innovation landscape in South Africa. Pretoria: DST. European Commission (2002).Science and society action plan. Luxembourg: European Commission (http://www.cordis.lu/science‐society>) European Research Council. (2012). Understanding and assessing the impact and outcomes of the ERC and its funding schemes (EURECIA): Final synthesis report. http://erc.europa.eu Godin, B. and Doré, C. (2003). Measuring the impact of science: Beyond the economic dimension. Montreal: Canadian Science and Innovation Indicators.

Grobbelaar, S.S. and Kirkland, J. (2013). Institutionalising research uptake in sub‐Saharan Africa universities. Paper (to be) presented at the 34th conference of the International Association of Scientific and Technological University Libraries, Cape Town, 14‐18 April.

Klein, J.T., Grossenbacher‐Mansuy, W., Häberli, R., Bill, A., Scholz, R.W. and Welti, M. (eds.) (2001). Transdisciplinarity: Joint problem solving among, science, and society. Basel: Birkhauser Verlag.

Kruss, G. (2012) Universities, innovation and inclusive development in Africa: Towards a research network. Paper presented at the 10th annual conference of SARIMA, 5‐7 September, Port Elizabeth.

Marais, H.C., Chan, K‐Y. and Gathua, S. (2012). Promoting utilisation of research findings: From policy to reality. Paper th presented at the 10 SARIMA conference, Port Elizabeth, September 2012.

Marais, H.C., Pienaar‐Marais, M. and Gathua, S. (2011). On the promotion of the utilisation of research findings by graduate students in management studies: A South African perspective. Paper presented at the 11th conference of the ECRM, July, 2011, Caen.

Marais, H.C. and Pienaar‐Marais, M. (2011). A multi‐dimensional approach to research and research training. Acta Academica, Supplementum (2): 239–263.

National Advisory Council on Innovation (NACI) (2003). Utilisation of research findings: Extent, dynamics and strategies.

Pretoria: NACI. National Research Council (2012). Best practices in assessment of research and development organizations. Washington, DC: National Research Council.

National Research Foundation (NRF) (2012). Impact report twenty12: Impacting lives, impacting South Africa. Pretoria: NRF. Nederhof, A.J. and Meijer, R.F. (1995). Development of bibliometric indicators for utility of research to users in society: Measurement of external knowledge transfer via publications in trade journals. Scientometrics, 32(1): 37–48.

Nerad, M. 2011. It takes a global village to develop the next generation of PhDs and Postdocs. Acta Academica, Supplementum(2): 198‐216.

Netherlands Organisation for Scientific Research (NWO) (2009). Standard evaluation protocol 2009–2015. Protocol for the assessment of research in the Netherlands. The Hague: NWO (www.knaw.nl/sep) Otronen, M. (2004). The evaluation of social relevance in an environmental research programme. Research Evaluation, 13(1): 43–50.

Overseas Development Institute (2004). Bridging research and policy in international development: An analytical and practical framework (www.odi.uk/rapid Rich, R.F (ed.) (1981). The knowledge cycle. Beverley Hills: Sage.

Rogers, E.M. and Shoemaker, F. (1971). The communication of innovations. New York: Free Press.

Royal Netherlands Academy of Arts and Science (RNAAS) (2005). Judging research on its merits: An Advisory report by the Council for the Humanities and the Social Science Council. Amsterdam: RNAAS. Spaapen, J. Dijstelbloem, H. and Wamelink, F. (2007). Evaluating research in context: A method for comprehensive assessment. The Hague: Consultative Committee of Sector Councils for Research and Development.

Spaapen, J. and Wamelink, F. (2007). Zichtbaar maken van maatschappelijke relevantie van kennis (‘Concretising social relevance and knowledge’). Amsterdam: RNAAS. Spaapen, J. and Van Drooge, L. (2011). Introducing ‘productive interactions’ in social impact assessment. Research Evaluation, 20(3): 211–218.

Stacey, R.D. (1996). Complexity and creativity in organizations. San Francisco: Berrett‐Koehler.

Van der Meulen, B. (2007). Onderzoeksevaluatie en maatschappelijk impact (‘Research evaluation and social impact’).

Paper presented at an ERIC seminar, Amsterdam, 12 June.

Van Drooge, L. and Van der Meulen, B. (2009). Evalueren in context: Beoordelen van de maatschappelijke kwaliteit van onderzoek (‘Evaluation in context: Assessment of social quality of research’). HO Management, December: 6–8.

Van Geenhuizen, M. (2010). Valorisation of knowledge: Preliminary results on valorization paths and obstacles in bringing th university knowledge to market. Paper presented at 18 Annual high technology small firms conference, 27–28 May, University of Twente, the Netherlands.

Von Bertalanffy, L. (1968). General system theory: Foundations, development, applications. New York: George Braziller.

Young, J. (2005). Bridging research and policy: The RAPID approach. Paper presented at the international conference, African economic research institutions and policy development: Opportunities and challenges, Dakar, Senegal.

246

The Critical Incident Technique as a Tool for Gathering Data as Part of a Qualitative Study of Information Seeking Behaviour Rita Marcella, Hayley Rowlands and Graeme Baxter Robert Gordon University, Aberdeen, UK r.c.marcella@rgu.ac.uk h.rowlands1@rgu.ac.uk g.baxter@rgu.ac.uk

Abstract: Since devised by Flanagan in 1954 as a tool to explore what people do to achieve an organisational aim, the critical incident technique (CIT) has been used in various disciplines as a method of understanding human behaviour. This paper provides an overview of the use of CIT in the specific field of information behaviour, both in large‐scale quantitative studies designed to assess the quality and impact of library and information systems and services, and in more qualitative research examining the information needs and use of particular professions or occupational groups, or of particular societal or community groups. It highlights the inconsistent application of CIT in academic research, and the quantitative versus qualitative tension that exists in discussions of the use of CIT as a data collection tool. The paper also discusses the use of CIT by the authors in a study of the information seeking behaviour of oil and gas professionals in a health and safety context, considering that project in relation to Flanagan’s five main steps in the CIT process, and in terms of the benefits and limitations of the technique identified by Flanagan and by other commentators. The authors believe that CIT has particular advantages in the study of information behaviour as a method of illuminating the ways in which the context of information need impacts on information behaviour, how participants feel, and in particular in identifying positive and negative behaviours in information seeking and use. The authors also argue that CIT must be used in a thoughtful manner and in a full recognition of its weaknesses in the design of future research.

Keywords: critical incident technique, information behaviour, qualitative techniques

1. Introduction The critical incident technique (hereafter CIT) has its roots in the Aviation Psychology Program of the US Army Air Forces during World War II, where it was used in the selection and classification of aircrews. Almost ten years after the war, one of the psychologists involved in the programme, John C. Flanagan, wrote a now famous paper on the development of the methodology and its subsequent use in a number of studies exploring the critical requirements for specific occupational groups or activities (Flanagan 1954). In his paper, Flanagan (p.327) defined CIT as consisting of “a set of procedures for collecting direct observations of human behavior in such a way as to facilitate their potential usefulness in solving practical problems and developing broad psychological principles”. For an incident to be critical, Flanagan argued (p.327), it “must occur in a situation where the purpose or intent of the act seems fairly clear to the observer and where its consequences are sufficiently definite to leave little doubt concerning its effects”.

While Flanagan’s paper placed a clear emphasis on data collection through the direct observation of human behaviour by trained observers, he acknowledged that, “if suitable precautions are taken, recalled incidents can be relied on to provide adequate data…” (p.340). Indeed, he was at pains to point out that the technique “does not consist of a single rigid set of rules governing such data collection. Rather it should be thought of as a flexible set of principles which must be modified and adapted to meet the specific situation at hand” (p.335).

Flanagan suggested that four procedures could be used in collecting recalled data in the form of critical incidents: individual interviews; group interviews; questionnaires; and/or “record forms” (i.e. where the participants record details of critical incidents in narrative form, or where they place a ‘check’ or ‘tick’ in the appropriate place on a pre‐determined list of the most likely incidents to occur).

In 2005, in a paper celebrating 50 years of CIT, Butterfield et al. (2005) championed both this flexibility and the subsequent diversity of disciplinary application. They noted that the technique has been utilised across a wide range of subject areas, including counselling, education and teaching, marketing, medicine, nursing and social work. They also concluded, however, that the CIT’s flexibility has become something of a “double‐edged sword”, as it has “encouraged the proliferation of approaches and terminology” (p. 476). For example, they catalogue several studies in which the data analysis procedures have diverged from those outlined by Flanagan (Butterfield et al. 2005, p.481); and, to highlight terminological inconsistencies, they list a number of phrases

247

Rita Marcella, Hayley Rowlands and Graeme Baxter used in studies adopting CIT, including ‘critical event technique’, ‘critical incident exercise’ and ‘critical incident reflection’ (p. 476).

This current paper discusses the use of CIT as a methodology in the domain of library and information science (LIS), and more specifically in the field of information behaviour. It will provide an overview of some of the published studies in this area, before focusing on a recent project, conducted by the authors, which used CIT to explore the information seeking behaviour of oil and gas professionals in a health and safety context.

2. CIT use in information behaviour studies As Urquhart (2001) explains, the use of the critical incident technique in information behaviour research has tended to revolve around the examination of “a brief, but memorable information seeking episode”, where the researchers have asked participants to provide one or more examples of occasions when they have sought information in order to, say, solve a problem or make a decision, to discuss the methods used in acquiring the information, and to evaluate the value and impact of the information obtained.

A number of these studies have been large‐scale, questionnaire‐based exercises, designed largely to assess the quality and impact of library and information systems and services. For example, Radford (2006) explored young people’s perceptions of public librarians and library staff in New York City, by gathering details of both “good” and “unpleasant or bad” experiences in public libraries from over 2,400 fifth and seventh grade students. Also in New York, Small and Snyder (2009), in a study of the impact of school libraries on student achievement and motivation, used an “open‐ended critical incident probe” to ask respondents to reflect on a particular event or activity in which the school library “helped or excited students about learning something new”. Tenopir, meanwhile, has been part of a team that has written a series of papers on academic faculty’s readership of scholarly articles (e.g. Tenopir, King and Bush, 2004; Tenopir et al., 2009; Tenopir, 2012). Here, they have used a variation of CIT, which they term the “incident of last reading”, in order to both assess information‐seeking and reading patterns, and demonstrate the value of libraries’ journal collections. Urquhart et al. (2003, p.76) highlight the problems associated with a self‐completion questionnaire approach, where “respondents…despite instructions supplied, had answered the questionnaire in general terms, giving details about what they usually did…rather than detailing what happened on one particular incident”. Conversely, Serenko and Turel (2010) describe the successful use of CIT in a paper‐based survey of Canadian university students’ positive and negative incidents relating to email usage, citing as benefits of CIT that novel descriptions of previously unrecorded phenomena may be generated.

Of greater interest to the present authors is the use of CIT in more qualitative information behaviour studies, where the technique has been integrated into interview instruments. Of the studies discussed in the literature, several focus on particular professions or occupational groups, frequently in the healthcare professions.

Indeed, Urquhart et al. (2003, pp. 72‐74) provide a tabular overview of CIT studies in the health sector, conducted between 1983 and 2001. More recently, the information behaviour of health professions has been the subject of CIT‐based research ranging from Musoke’s 2007 study of the accessibility and use of health information among primary healthcare providers in rural Uganda, to the investigation of the information challenges facing non‐clinical managers in rural Nova Scotia, Canada, by MacDonald et al. (2011). The academic community, too, has been the subject of a number of qualitative CIT studies, aided presumably by the relative convenience of research participants. These have included studies of academic staff, such as Jamali and Asadi’s 2010 exploration of the role of Google in the information seeking behaviour of physicists and astronomers, and Makri and Blandford’s examination, in 2012, of the ways in which interdisciplinary researchers come across information serendipitously. They have also included student‐focused projects, such as the research conducted by Kerins et al. (2004) which explored the information behaviour of engineering and law students in Ireland, where the critical incident focused on their final year project.

Elsewhere, CIT has been used as the basis for interviews in a range of occupational information behaviour studies conducted internationally, from the examination of artisan fisher folk in Uganda (Ikoja‐Odongo and Ocholla, 2003), to Zach’s 2005 study of how senior arts administrators in the US decide when they have found “enough” information to complete management tasks. And from Lambert’s 2010 study of how Southern Baptist ministers in the midwestern US obtain information for administrative or pastoral purposes, to the investigation of the ways in which Kuwaiti journalists seek information for journalistic assignments (Chaudry and Al‐Sagheer, 2011).

248

Rita Marcella, Hayley Rowlands and Graeme Baxter Away from the workplace environment, a number of information behaviour researchers have used CIT within the framework of everyday life information seeking (ELIS), which Savolainen (1995, pp 266‐7) defined as “the acquisition of various informational (both cognitive and expressive) elements which people employ to orient themselves in daily life or to solve problems not directly connected with the performance of occupational tasks”. Some of this research has attempted to look at a broad cross‐section of a particular community, including Savolainen’s own study (1995) of the residents of Tampere, Finland; and Johnson’s (2004) investigation of the role of social networks and social capital in the information seeking behaviour of the residents of Ulaanbaatar, Mongolia. Other studies have focused on particular societal groups, or on individuals at particular stages of the life cycle. For example, Jiyane and Ocholla (2004) studied the information needs and sources of women in a rural community in South Africa, while Hamer (2003) used CIT interviews with young gay men to acquire data on their information behaviour relating to coming out. Julien (1999) explored the barriers that Canadian adolescents face in accessing information helpful for career decision making; while Niemelä and Huotari (2008) investigated the information use of Finnish senior citizens.

While Davenport (2010), in a review of the use of three “confessional methods” in ELIS research, describes CIT as a “fitting technique for analyzing…confessional phenomena”, and argues that the confessional approach is particularly apposite for ELIS, she is critical of the ways in which CIT has actually been used in the ELIS field. In particular, she notes that the number of critical incidents required for “robust analysis” is rarely achieved, that few researchers comply with Flanagan’s protocols, and that his caveats are “rarely discussed, let alone heeded” (p. 539). However, this charge could equally be levelled at the use of CIT in other social science research (Butterfield et al. 2005, p.476). Urquhart et al. (2003, p. 65) are also critical of the lack of methodological detail in the published accounts of many CIT studies.

Bearing these points in mind, the following section of this paper will discuss the present authors’ use of CIT to explore information behaviour in an oil and gas, health and safety context. Although it should be emphasised here that, in designing the study, the researchers did not have Flanagan’s protocols in mind. Rather, they drew upon some of their previous, interview‐based, information behaviour research in which participants focused on recent information‐seeking incidents or episodes. For example, Marcella and Illingworth (2012) used such an approach when studying the information behaviour of UK entrepreneurs encountering business failure; while Baxter, Marcella and Illingworth (2010), in exploring organisational information behaviour during Scottish Government public consultation exercises, asked interviewees to consider the example of the most recent consultation to which they had responded. While the current authors did not, themselves, adhere rigidly to Flanagan’s guidance, they will consider their research in relation to his five main steps in the CIT process, and in terms of the benefits and limitations of CIT identified by Flanagan and by other commentators.

3. The use of CIT as a tool to gather data about information behaviour in the oil and gas industry The study which provided the basis for this paper and reflections on the use of CIT as a data collection method was conducted in 2011 and was sponsored by an engineering software provider, focusing on the oil and gas sector. The study explored the role of information systems in enhancing health, safety and emergency response in the oil and gas industry and gathered useful perspectives on the role of information systems in enhancing health and safety management. The study also uncovered insights into the information seeking behaviour of oil and gas professionals in a health and safety context. The project was particularly timely in that it took place when the critical nature of health and safety in the energy sector continued to receive considerable attention internationally, particularly following the high‐profile Deepwater Horizon disaster in 2010. Indeed, a commission appointed by President Obama to investigate the Gulf of Mexico explosion and oil spill concluded that “most, if not all, of the failures at Macondo can be traced back to underlying failures of management and communication” (National Commission on the BP Deepwater Horizon Oil Spill and Offshore Drilling, 2011, p.122). The research described here coincided with that discussed in the only other known published paper on CIT‐based information behaviour research in the energy sector, where Ibrahim and Allen (2012) explored the relationship between information sharing and trust during major incidents in the oil industry, interviewing 19 employees of a major multinational oil company in the process.

The present authors’ study was undertaken in two distinct stages, in a mix of quantitative and qualitative approaches: 1) an online questionnaire survey of over 370 individuals in the oil and gas industry, completed largely by health and safety managers, senior managers and engineers located across the globe; and 2) a series

249

Rita Marcella, Hayley Rowlands and Graeme Baxter of in‐depth interviews utilising critical incidents as a focus. It is the second stage of the project which forms the basis of the discussion below, relating this to Flanagan’s five key steps in the CIT process.

Understanding the general aims of the activity being studied: Flanagan (1954, p. 336) notes that a basic condition for any use of CIT is a “fundamental orientation in terms of the general aims of the activity” being studied. The main focus of this study was, of course, the role of information systems and information behaviour in enhancing health and safety in the oil and gas industry. The present authors have an extensive background in LIS research, and, through a number of other recent research projects commissioned by industry, have developed a sound knowledge of health and safety management in the energy sector. Of critical importance here, though, was the input of the commissioning company. All survey questions and interview schedules were designed in consultation with the research sponsors, to reflect industry perspectives. On reflection, while the process of collaborative research design can be challenging, with industry‐led research focusing on broad lessons, and academic research focusing on the ‘small detail’, such an approach was felt to be an effective exercise in ensuring that the research instruments were well designed.

Making plans and setting specifications: As the second stage of the CIT process, Flanagan (1954, pp. 337‐339) called for researchers to set precise plans and specifications. He urged that the types of situations to be explored – the critical incidents – should be relevant and clearly defined, and that those conducting the research should be familiar with the types of incident being studied and be consistent in their approach. In the current study, the critical incidents to be discussed in the interviews were all pre‐determined by the participating businesses from the oil and gas sector, with agreement from the research team. While the incidents could be chosen by the organisations, they had to meet two basic criteria: 1) that any incident be substantial enough in nature that the information aspects of dealing with the situation might be explored fully in the CIT interview; and 2) that they be significant enough to have caused potential detrimental impact to the organisation, but not sufficiently high profile to jeopardise participant anonymity. To ensure consistency of approach, the interviews were all conducted by the same member of the research team. Flanagan (1954, p.

341) also highlighted the need to “try out” questions with a small group of typical participants before being put into general use in a study. However, as Urquhart et al. (2003, p. 65) point out, many published CIT studies fail to provide any details on how, or indeed if, such piloting was conducted. In this research, to ensure the effectiveness of the interview instrument, the interviewer first conducted pilot interviews with three individuals with experience of working either in the oil and gas industry or in a health and safety role.

Collecting the data: Flanagan (1954, p. 343) noted that sample size in CIT studies is determined less by the number of participants involved, than by the number of critical incidents (and associated “critical behaviors”) observed or reported, and whether or not these adequately represent the activities being studied. While he noted that “there does not appear to be a simple answer to this question”, he discussed samples in terms of hundreds or thousands of incidents. Davenport (2010, p. 538), observes that these numbers are rarely met in information behaviour research, but fails to acknowledge the resource levels required to conduct CIT research on such a scale. Urquhart et al. (2003, p. 65), on the other hand, note that “few studies can afford [that] level of research effort”. Certainly, in industry‐sponsored projects such as that described here, researchers rarely have the luxury of the time and financial resources necessary to gather samples of that size.

The CIT element of the study discussed here was based on just four critical incidents, one identified by each of four participating companies. Enlisting organisational participants proved difficult, largely due to the subject nature of the research. Understandably, businesses were cautious about openly discussing incidents that might be considered embarrassing and commercially sensitive. This problem was, presumably, also encountered by Ibrahim and Allen (2012) in their research, as they note that they had a signed confidentiality agreement with the participating oil production company. The four businesses participating in this study were recruited via the online survey, the online professional networking site, LinkedIn, or via the researchers’ industry contacts. The four companies consisted of an operator, a contractor, a manufacturer, and a logistics company; although this range of business types was achieved more by accident than design. The four critical incidents on which the interviews were based were, respectively:

the 2010 Icelandic ash cloud and its effects on the safe transportation of staff between onshore and offshore installations, which was considered potentially detrimental to business performance;

a software failure, resulting in the loss of data with health and safety implications;

250

Rita Marcella, Hayley Rowlands and Graeme Baxter

the potential contamination of thousands of pockets of private land, due to industrial and domestic fuel container leakage; and

a chemical spillage following a traffic accident at a major city road junction.

The duration of these four events ranged from one week to around eight years, and interviews were sought from key individuals instrumental in dealing with the incidents and their aftermaths, and who were thus able to comment critically on the impact that information behaviour had had on their company’s capacity to respond. Interviews were conducted with eleven individuals from across the four companies, either face‐to‐ face or by telephone. The research team had also sought participation from employees from a range of hierarchical levels, to reflect the differing perspectives within a response team dealing with a critical incident.

As a result, the interviewees occupied a variety of roles, including investigator, supervisor, lead response, safety advisor, duty manager and data analyst.

All interviewees were given advance notice of the critical incident to be discussed, although the precise nature of the interview questions was not revealed prior to the interviews taking place. As Flanagan (1954, p. 339) rightly advises, recalled critical incidents are best recorded “while the facts are still fresh in the mind”. While three of the four incidents had occurred in the 12 months prior to the interviews taking place, the fourth incident (that of the fuel container leakage) had been a lengthy affair first identified in 2001, ten years before this research took place. Despite this time lapse, there was little evidence of any significant deviance between the basic accounts of the individuals involved in that incident. And while the researchers acknowledge that respondents from within particular companies, having had prior notification of the critical incident to be explored, would have had the opportunity to consult company documentation on the incident, or to discuss it with their colleagues, there was no obvious evidence of any ‘collusion’ having taken place in an attempt to provide consistent accounts. With one incident, the company concerned also provided a copy of their internal report on the event, further verifying their employees’ verbal accounts.

All interviewees talked freely and at length about the incidents and their associated information behaviours.

The researchers believe that the use of CIT gave focus to the interviews, allowing open discourse on a familiar topic, and enabling participants to describe information seeking behaviour without a need for a deep understanding or interest in the information domain on a conceptual level. Sharoff (2008) warns that participant embellishment might occur during CIT; however, the current authors feel that such embellishment will occur naturally and is not a factor which can be excluded from any study.

Analysing the data: The CIT interviews lasted between 40 and 120 minutes, were recorded with permission, and were subsequently transcribed verbatim. Flanagan describes the analysis of critical incident data as involving three distinct elements: determining a general frame of reference that will be useful for describing the critical incidents; inductively constructing a set of categories for the incidents; and deciding on the most appropriate level of specificity‐generality with which to report the data. In the present study, the transcripts were analysed with recurring themes being coded in an iterative process.

Interpreting and reporting the results: Flanagan (1954, p. 345) warns that the “real errors” in CIT “are made not in the collection and analysis of the data but in the failure to interpret them properly”, while both Urquhart et al. (2003) and Radford (2006) highlight the difficulty of coder interreliability. In this study, the interview transcripts were independently analysed by two members of the research team to add reliability to the interpretation of the data.

4. Conclusions This paper has provided an overview of the use of CIT in the study of information behaviour, and has discussed its specific use in an exploration of the information seeking behaviour of oil and gas professionals in a health and safety context. It has shown that the inconsistent application of the technique has been highlighted by various observers as a weakness of CIT‐based studies. However, in line with Flanagan’s belief in the importance of flexibility and freedom from constraint, the current authors believe that it is the critical insight with which the study is designed that is important, and that it is only by testing research tools in an open and exploratory way that their true contribution can be evaluated. There also remains a quantitative versus qualitative tension in discussions of the use of CIT as a data collection tool. The major challenge, however, is that researchers must position their research appropriately within a research domain and not try to apply

251

Rita Marcella, Hayley Rowlands and Graeme Baxter empiricist approaches to qualitative studies and vice versa. The present authors view their use of CIT as a technique in a classic qualitative tradition.

As with the use of any methodology, the CIT has strengths and weaknesses. However, the authors believe that it has particular advantages in the study of information behaviour as a method of illuminating the ways in which the context of information need impacts on information behaviour, how participants feel and in particular in identifying positive and negative behaviours in information seeking and use, which would merit further exploration. The authors would also argue that CIT must be used in a thoughtful manner and in a full recognition of its weaknesses in the design of future research. Flanagan’s original conception of the CIT to better understand human behaviour in the course of achieving an aim is fundamental to information behaviour research and his design of the CIT process recognised the need for flexible evolution of the tool. It is hoped that a body of work extending this understanding of its contribution to information behaviour research will continue to evolve.

References Baxter, G., Marcella, R. and Illingworth, L. (2010). Organisational information behaviour in the public consultation process in Scotland, Information Research, Vol. 15, No. 4, http://informationr.net/ir/15‐4/paper442.html Butterfield, L. D., Borgen, W. A., Amundson, N. E. and Maglio, A. T. (2005) “Fifty years of the critical incident technique: 1954‐2004 and beyond”, Qualitative Research, Vol. 5, No. 4, November, pp 475‐497.

Chaudry, A. S. and Al‐Sagheer, L. (2011) “Information behavior of journalists: analysis of critical incidents of information finding and use”, The International Information & Library Review, Vol. 43, No. 4, pp 178‐183.

Davenport, E. (2010) “Confessional methods and everyday life information seeking”, Annual Review of Information Science and Technology, Vol. 44, No. 1, pp 533‐562.

Flanagan, J. C. (1954) “The critical incident technique”, Psychological Bulletin, Vol. 51, No. 4, July, pp 327‐358.

Hamer, J. S. (2003) “Coming‐out: gay males’ information seeking”, School Libraries Worldwide, Vol. 9, No. 2, pp 73‐89.

Ibrahim, N. H. and Allen, D. (2012) “Information sharing and trust during major incidents: findings from the oil industry”, Journal of the American Society for Information Science and Technology, Vol. 63, No. 10, pp 1916‐1928.

Ikoja‐Odongo, R. and Ocholla, D. N. (2003) “Information needs and information‐seeking behavior of artisan fisher folk of Uganda”, Library & Information Science Research, Vol. 25, No. 1, pp 89‐105.

Jamali, H. R. and Asadi, S. (2010) “Google and the scholar: the role of Google in scientists’ information‐seeking behaviour”, Online Information Review, Vol. 34, No. 2, pp 282‐294.

Jiyane, V. and Ocholla, D. N. (2004) “An exploratory study of information availability and exploitation by the rural women of Melmoth, KwaZulu‐Natal”, South African Journal of Libraries & Information Science, Vol. 70, No. 1, pp 1‐9.

Johnson, C. A. (2004) “Choosing people: the role of social capital in information seeking behaviour”, Information Research, Vol. 10, No. 1, http://informationr.net/ir/10‐1/paper201.html Julien, H. E. (1999) “Barriers to adolescents’ information seeking for career decision making”, Journal of the American Society for Information Science, Vol. 50, No. 1, pp 38‐48.

Kerins, G., Madden, R. and Fulton, C. (2004). “Information seeking and students studying for professional careers: the cases of engineering and law students in Ireland”, Information Research, Vol. 10, No. 1, http://informationr.net/ir/10‐ 1/paper208.html Lambert, J. D. (2010) “The information‐seeking habits of Baptist ministers”, Journal of Religious & Theological Information, Vol. 9, No. 1‐2, pp 1‐19.

MacDonald, J., Bath, P. and Booth, A. (2011) “Information overload and information poverty: challenges for healthcare services managers?”, Journal of Documentation, Vol. 67, No. 2, pp 238‐263.

Makri, S. and Blandford, A. (2012) “Coming across information serendipitously – part 1: a process model”, Journal of Documentation, Vol. 68, No. 5, pp 684‐705.

Marcella, R. and Illingworth, L. (2012) “The impact of information behaviour on small business failure”, Information Research, Vol. 17, No. 3, http://informationr.net/ir/17‐3/paper525.html Musoke, M. G. N. (2007) “Information behaviour of primary health care providers in rural Uganda”, Journal of Documentation, Vol. 63, No. 3, pp 299‐322.

National Commission on the BP Deepwater Horizon Oil Spill and Offshore Drilling (2011). Deep water: the Gulf oil disaster and the future of offshore drilling, http://www.oilspillcommission.gov/final‐report Niemelä, R. and Huotari, M. (2008) “Information use and enactment: the perspective of senior citizens’ everyday life information behavior”, Paper read at the International Conference for the Celebration of the 20th Anniversary of Information Studies, Oulu, Finland, 23‐25 June.

Radford, M. L. (2006) “The critical incident technique and the qualitative evaluation of the Connecting Libraries and Schools Project”, Library Trends, Vol. 55, No. 1, pp 46‐64.

Savolainen, R. (1995) “Everyday life information seeking: approaching information seeking in the context of “way of life”, Library & Information Science Research, Vol. 17, No. 3, pp 259‐294.

Serenko, A. and Turel, O. (2010) “Rigor and relevance: the application of the critical incident technique to investigate email usage”, Journal of Organizational Computing and Electronic Commerce, Vol. 20, No. 2, pp 182‐207.

252

Rita Marcella, Hayley Rowlands and Graeme Baxter Sharoff, L. (2008). “Critique of the critical incident technique”, Journal of Research in Nursing, Vol. 14, No. 4, pp 301‐309.

Small, R. V. and Snyder, J. (2009) “The impact of New York’s school libraries on student achievement and motivation: phase II―in‐depth study. School Library Media Research, Vol. 12, http://www.ala.org/aasl/aaslpubsandjournals/slmrb/slmrcontents/volume12/small_phase2 Tenopir, C. (2012) “Beyond usage: measuring library outcomes and value”, Library Management, Vol. 33, No. 1/2, pp 5‐13.

Tenopir, C., King, D. W. and Bush, A. (2004) “Medical faculty’s use of print and electronic journals: changes over time and in comparison with scientists”, Journal of the Medical Library Association, Vol. 92, No. 2, pp 233‐241.

Tenopir, C., King, D. W., Edwards, S. and Wu, L. (2009) “Electronic journals and changes in scholarly article seeking and reading patterns”, Aslib Proceedings, Vol. 61, No. 1, pp 5‐32.

Urquhart, C. (2001) “Bridging information requirements and information needs assessment: do scenarios and vignettes provide a link?”, Information Research, Vol. 6, No. 2, http://informationr.net/ir/6‐2/paper102.html Urquhart, C., Light, A., Thomas, R., Barker, A., Yeoman, A., Cooper, J., Armstrong, C., Fenton, R., Lonsdale, R. and Spink, S. (2003) “Critical incident technique and explicitation interviewing in studies of information behavior”, Library & Information Science Research, Vol. 25, No. (1), pp 63‐88.

Zach, L. (2005) “When is “enough” enough? Modelling the information‐seeking and stopping behavior of senior arts administrators”, Journal of the American Society for Information Science and Technology, Vol. 56, No. 1, pp 23‐35.

253

Applying Quadrant Analysis to Determine Textbook Learning Enhancement Nico Martins Department of Industrial and Organisational Psychology, University of South Africa, Pretoria, South Africa martin@unisa.ac.za

Abstract: The purpose of this research was to determine to what extent textbook characteristics enhance learning – as perceived by academics and students. The focus of this paper is on the analysis and interpretation of the two scale items, using quadrant analysis to determine the agreement (satisfaction) of students and academics regarding the importance of textbook characteristics and whether or not these enhance learning. Academics and students from all academic institutions in South Africa were invited to participate in this survey‐based research. In total, 203 academics participated in the lecturer survey and 501 students in the student survey. A project team, consisting of a publishing company, academics and the researcher, compiled the two questionnaires. Both questionnaires consisted of biographical questions, “yes/no” questions and scale‐based questions using a four‐point scale. Twenty‐nine questions had two scales to determine the importance of agreement with textbook characteristics. These questions are the focus of this paper. The results of the research indicated a number of significant factors. Academics who embark on writing textbooks should take note of the textbook characteristics that students and academics feel do in fact enhance learning and those characteristics that do not, those characteristics with low importance and low satisfaction, and those with low importance and high satisfaction.

Publishers should also take cognisance of the research and cooperate closely with academics when planning academic textbooks.

Keywords: quadrant analysis, textbook characteristics, students, lecturers, learning enhancement

1. Introduction One of the initiatives that universities worldwide are embarking on is to investigate students’ dependence on textbooks and the strengthening of professional and teachers’ reliance on textbooks (Watt 2004, 4).

Furthermore, according to Watt (2004, 51), research literature on the use and curriculum role of materials (textbooks) in the United States has emphasised the fact that teachers and students depend on such materials.

In South Africa, there is the additional debate on the role of the African Renaissance in curriculum development on the African continent (Higgs, Van Niekerk and Van Wyk 2010, 134). According to Kim‐Prieto and D’Oriano (2011, 458), creating a new course from scratch can be time consuming. Developing the syllabus, choosing articles or textbooks, and creating exercises and assignments, in addition to developing lecture materials, all take considerable time, in spite of access to high‐quality resources and examples. In addition to the time required to create a course, the academic may also have to gain the approval of curriculum committees overseeing the development of new courses.Given the complexity of the modern world and the workload of lecturers, the task of developing resources to fit a curriculum can be daunting.

In many instances, it appears that only senior academics are involved in curriculum development. Studies by Lianli (2007, 107) indicated that lecturers involved in teaching English language classes at university are quite happy to hand over the responsibility for programme content and design to the textbook. Another area of concern is that it would appear that textbooks are not always selected according to the theoretical basis that supports the framework of the textbook. Given the above, the purpose of this research was to determine to what extent textbook characteristics enhance learning – as perceived by academics and students.

2. Context of the study The aim of the research was to include in the study academics as well as students of all South African universities, universities of technology and nursing colleges. The participating publishing company supplied the researcher with a contact list of all the academics at the various institutions. All of the academics were contacted via e‐mail and asked to complete the lecturer’s questionnaire online.

A two‐pronged approach was used to obtain information from students. First, at all the publisher’s branches throughout South Africa, students were approached by briefed staff to complete the student questionnaire.

On completion, each student received a pen for his/her time and input. Secondly, the publisher contacted key people at the institutions and obtained their consent to distribute questionnaires to their students and then

254

Nico Martins post them back. Copies of the questionnaire, detailed instructions and a pen for each participant were posted to the relevant lecturers. The purpose was to obtain a broad and representative sample of both students and lecturers at all the institutions.

Both the paper and electronic questionnaires contained the background to the study, instructions for completing the surveys and the assurance that the questionnaires would be handled in the strictest confidence and that no student’s or academic’s results would be given to the participating institutions. No indication was given of the name of the publishing company. Both questionnaires were compiled by the project team after in‐ depth discussions with a sample of lecturers from various universities to determine their perceptions of curriculum development and what they need from lecturing resources such as textbooks, study guides, articles and online resources.

In total, 203 academics participated in the survey. This is a 16% response rate from the electronic database supplied by the publishing company. Table 1 indicates that most of the academic staff were from Unisa (18.2%) and the Durban University of Technology (16.3%). Overall, 18 of the 19 South African academic institutions for higher education participated in the survey.

Table 1: Biographical information of academic participants Universities Frequency Rhodes University 2 University of Cape Town 5 University of Fort Hare 1 University of the Free State 3 University of KwaZulu‐Natal 3 University of Limpopo 4 North‐West University 16 University of Pretoria 13 University of Stellenbosch 5 University of the Witwatersrand 11 Comprehensive Universities Nelson Mandela Metropolitan University 12 Unisa 37 University of Johannesburg 5 Universities of Technology Central University of Technology 13 Western University of Technology 33 Tshwane University of Technology 7 Nursing Colleges

Anne Latsky Nursing College 17 Excelsius Nursing College 12 Vrystaat Nursing College 11 TOTAL 203

% 0,9 2,5 0,5 1,5 1,5 2,0 7,9 6,4 2,5 2,0 5,9 18,2 2,5 6,4 16,3 3,4

8,4 5,8 5,4 100

Five hundred and one students participated in the student survey (table 2). The largest group of students was from Unisa (16.2%) and the University of Pretoria (14.7%). Most students were first year (26.5%) or fourth year (23.8%), females (66.5%) and were studying full time (76.8%). Overall, students from 12 institutions participated in the survey.

Although not all academic institutions in South Africa participated in the two surveys, the results represent most of the universities, comprehensive institutions, universities of technology and nursing colleges.

3. Research methodology The research approach was a quantitative study comprising two surveys conducted with students and academics.

3.1 Compiling the questionnaires The researcher had in‐depth discussions with the publishing company to determine the content and scope of the two questionnaires. It was decided that the questionnaires should not only focus on the publisher’s

255

Nico Martins marketing needs, but also give academics insight into the value of resources they use in teaching and the value of these to students. A project team, comprising the publishing company, academics and the researcher, then compiled the two questionnaires. Both questionnaires were piloted to determine whether students and academics would understand the terminology used and whether the length of the questionnaires was appropriate. After the pilot study, both questionnaires were finalised and distributed as discussed.

Table 2: Biographical information of students Universities Frequency University of the Free State 17 North‐West University 11 University of Pretoria 74 University of Stellenbosch 43 Comprehensive Universities Nelson Mandela Metropolitan University 23 Unisa 81 University of Johannesburg 42 University of the Western Cape 34 Universities of Technology Vaal University of Technology 49 Nursing Colleges Ann Latsky Nursing College 33 Excelsius Nursing College 48 Vrystaat Nursing College 40 No response 6 TOTAL 501 Academic Year of Study 1st year 133 2nd year 98 3rd year 75 4th year 119 5th year 13 Master’s and doctorate 15 No responses 48 TOTAL 501 Gender Male 148 Female 333 No response 20 TOTAL 501 Part‐time or full time Part‐time 102 Full time 385 No response 14 TOTAL 501

% 3,4 2,2 14,7 8,6 4,6 16,2 8,4 6,8 9,8 6,6 9,6 7,9 1,2 100 26,5 19,6 15,0 23,8 2,6 3,0 9,5 100 29,5 66,5 4,0 100 20,4 76,8 2,4 100

Both questionnaires consisted of biographical questions, “yes/no” questions and scale‐based questions using a four‐point scale. The decision was taken to use a four‐point scale to eliminate the midpoint tendency. Twenty‐ nine questions had two scales to determine the importance of an agreement with textbook characteristics.

Both scales used a four‐point scale. Scale 1, which measured agreement, referred to strongly disagree, disagree, agree and strongly agree. Scale 2, which measured importance, referred to insignificant, unimportant, important and very important. See table 3 for an example of typical questions asked and the layout of section B.

Owing to the length, scope and depth of the two questionnaires (the student questionnaire consisted of 96 questions and the lecturer questionnaire, 112 questions), it was decided that this paper would focus only on the analysis and interpretation of the two scale items, using quadrant analysis to determine the agreement (satisfaction) of students and lecturers, the importance of textbook characteristics and whether they enhance learning. This paper also only focuses on overall results and not on any other biographical/geographical analysis.

256

Nico Martins Table 3: Example of typical questions asked Section B Instructions for Completion Left scale Rate each of the statements below using the following scale: = Strongly disagree = Disagree = Agree = Strongly agree

Right scale (importance) Then indicate how important these elements are to you:

= Insignificant = Unimportant = Important = Very important

Please think of textbooks you are currently using and answer the following statements.

Which of the following textbook characteristics make it easier for you to study?

1

2

1 Strongly disagree Very important 4 2 Disagree Important 3 3 Agree Unimportant 2

4 Strongly agree Insignificant 1

3 4 Statements 1 2 3

When I can relate to the examples being used

When the language is easy to understand

When it is easy to understand the content

When it is easy to find information in the textbook

When the layout of the textbook is attractive

When difficult words are explained in the glossary

When the textbook makes use of icons throughout – these are a

useful “roadmap”

When the information is presented in “bite‐sized chunks” When the content is written in a conversational style When the textbook has clear headings and subheadings

4

For the answers, the participants were asked to rate the importance and satisfaction of a number of different textbook characteristics, for instance:

The relevance and understanding of the content of textbooks

The ease of understanding the language

The use of practical, real‐life examples

The inclusion of case studies

The inclusion of self‐evaluation questions or exercises, and the inclusion of model answers for them

Whether the activities promote analytical and critical thinking

Whether it is easy to find information in the textbooks

Whether the textbook has clear headings and subheadings

Whether the textbook helps students to perform better in tests and examinations

Whether main ideas are summarised at the beginning of a chapter

Whether difficult words are explained in the glossary

Whether the textbooks include tables and flow charts

Whether illustrations are up to date (eg not dated in terms of dress or hairstyles unless a certain period is portrayed)

Whether a summary is included at the end of a chapter

257

Nico Martins

4. Quadrant analysis

IMPORTANCE

Importance‐performance analysis was first proposed by Martilla and James (1977) to give management insight into identifying the strengths and weaknesses of a company. A number of studies have subsequently been conducted in various fields applying the technique, quadrant analysis, in fields such as marketing, quality of service, employee satisfaction surveys, measuring student satisfaction and by consultants (Angell, Heffernan and Megicks 2008; Borg and Mastrangelo 2008; Douglas, Douglas and Barnes 2006; Douglas, McClelland, Davies and Sudbury 2009; Ennew, Reed and Binks 1993, http://www.macorr.com/quadrant‐analysis‐ survey.htm). Quadrant analysis is used in conjunction with market research surveys, customer surveys and employee surveys to analyse gaps between expectations and actual or current perceptions.

According to Douglas, Douglas and Barnes (2006), so‐called “quadrant analysis” is a graphic technique used to analyse importance and attribute ratings. It produces a grid that shows which attributes are important among those that a service delivers. The analysis allows for the determination of whether aspects of a particular service provision are the aspects that respondents value as important. In this study, quadrant analysis was conducted on the top two questionnaire box scores for both importance (ie “very important” and “important”) and agreement ratings (ie “strongly agree” and “agree”). Thus for every characteristic aspect, the percentage scores of those respondents who either agreed or strongly agreed were identified, as were the percentages of respondents who considered that aspect important or very important. Once all questions had been analysed in a similar way, the median score was determined for both the agreement and satisfaction scores. For each percentage score, its distance from the median was then calculated. For analysing population measures, the difference in precision between the average based upon a two‐category scale with equal frequencies in each category and a continuous scale is the difference between the standard error of the median and the standard error of the mean. This could be positive or negative. The importance scores were the Y‐axis scores and the agreement scores the X‐axis scores measuring student satisfaction. For both surveys, the satisfaction and importance ratings were entered into SPSS by each participant. The data were then analysed using quadrant analysis to illustrate in graphic form the relationship between the importance ratings and satisfaction ratings for each question (see figure 1). The purpose of plotting the results on the grid has typically been to analyse student feedback data in UK universities for a number of years (Douglas, Douglas and Barnes 2006).

High

A

Concentrate Here Maintain Quality

C

C

C

D Low Priority

Possible Overkill

C Low C

Low SATISFACTION High

Figure 1: Quadrant analysis These four quadrants are composed of “keep up the good work” (quadrant B), “possible overkill” (quadrant D) “low priority” (quadrant C), and “concentrate here” (quadrant A). Quadrant B has both high satisfaction and high importance and can be considered as the opportunities to achieve or maintain competitiveness for organisations. Quadrant D has high satisfaction but low importance, which indicates that the resources committed to these attributes are excessive and should be deployed elsewhere. Quadrant C has the characteristics of both low satisfaction and low importance, and these attributes do not require additional efforts. Finally, the attributes located in quadrant A, with low satisfaction but high importance, are viewed as major weaknesses for an organisation. Immediate attention for improvement would thus be required.

Moreover, the inability to identify the attributes in quadrant A might result in low customer satisfaction, in this study, low student satisfaction with specific textbook characteristics. In fact, immediate improvement efforts

258

Nico Martins should be rated as the highest priority when major weaknesses are identified, while attributes in quadrant B, which are regarded as major strengths, should be maintained, leveraged and heavily promoted (Douglas, Douglas and Barnes 2006; Douglas, McClelland, Davies and Sudbury, 2009).

5. Results In order to put the current usage of the textbook by students and academics in perspective, a question was posed in both questionnaires to determine the extent to which lecturers prescribe textbooks and whether students buy textbooks. The results indicate that the majority of academics (85.6%) prescribe textbooks, and according to students (82%) of all institutions, lecturers use prescribed textbooks and thus most of them (students) (88%) purchase the prescribed textbooks.

The results indicate that most characteristics are clustered in quadrants B and C, indicating that students and lecturers perceive a number of characteristics as important and are also satisfied with the situation.The combined results of all participants, lecturers and students in all the institutions are displayed in tables 4 and 5. The results show that both students and lecturers perceive 11 characteristics (see the list under discussion and conclusions) of textbooks as being important and that they are also satisfied with the situation (tables 4 and 5 ‐ quadrant B).In addition, students also perceive the following characteristics as being of high importance and high satisfaction:

when they can relate to the examples used

when difficult words are explained

when the textbook includes self‐evaluation questions or exercises and provides model answers to them

Lecturers, however, perceive two other characteristics to be of high importance and high satisfaction:

Only one concept is introduced at a time, so that students do not feel overwhelmed by the volume of information

The text is interesting if students enjoy reading it.

The above indicates that although there is mostly correspondence regarding the perceptions of lecturers and students, there are also important differences.

The results, however, also indicate a number of characteristics that are perceived to be of low importance and low satisfaction (quadrant C). Interestingly, the students experienced 14 characteristics as being of low importance and low satisfaction, with only 11 for the lecturers. Comparing the results of quadrant C for lecturers and students, it is interesting to again note a number of similarities. Nine characteristics are perceived to be of low importance and low satisfaction by both groups. The four characteristics in the students’ list which are not included in quadrant C for lecturers are listed in quadrants A and B for the lecturers. This indicates either high importance and low satisfaction or low importance and high satisfaction for the lecturers. This again emphasises that the two groups have different views.

The results of interest and value to textbook writers and publishers are the results in quadrants D and A. Only the lecturers indicated that the following characteristics are currently overkill:

Difficult words are explained in the glossary (important to students).

Textbooks include tables and flow charts (students also perceive this to be of low importance and low satisfaction).

Illustrations are up to date (eg not dated in terms of dress or hairstyles unless a certain period is portrayed); (students also perceive this to be of low importance and low satisfaction).

The lecturers also indicated that two characteristics are of high importance, but they experience low satisfaction:

The textbook includes case studies (students experienced it to be of high importance and high satisfaction).

The textbook includes self‐evaluation questions or exercises and provides model answers for them (students experienced this to be of high importance and high satisfaction).

259

Nico Martins Table 4: Importance–satisfaction grid for students Quadrant C: Low Importance and Low Satisfaction When the layout of the textbook is attractive

Quadrant B: High Importance and High Satisfaction When I can relate to the examples being used

When the textbook makes use of icons throughout – these are a useful “road map” When the information is presented in “bite‐sized” chunks When the content is written in a conversational style When the textbook includes tables and flow charts When the illustrations or graphics are of high quality When the illustrations are up to date (eg not dated in terms of dress or hairstyles, unless a certain period is portrayed) When various ethnic groups are represented as part of the examples, case studies and illustrations in the book When the activities are interactive

When the language is easy to understand When it is easy to understand the content When it is easy to find information in the textbook When difficult words are explained in the glossary When the textbook has clear headings and subheadings When the content is relevant to the course

When the activities promote analytical and critical thinking When using the textbook helps me to perform better in tests and exams When the main ideas are summarised at the beginning of a chapter

When practical, real‐life examples are used

When the activities require me to collaborate with other students When one concept at a time is introduced – I don’t feel overwhelmed by the volume of information When the cover of the book is appealing If I like the visual images If the text is interesting, I enjoy reading it

Quadrant A: High Importance and Low Satisfaction None

When the textbook includes case studies When a summary is included at the end of the chapter When the textbook includes self‐evaluation questions or exercises, and provides model answers for them.

When sufficient examples are included to explain theoretical concepts Quadrant D: Low Importance and High Satisfaction None

Table 5: Importance – satisfaction grid for lecturers Quadrant C: Low Importance and Low Satisfaction The layout of the textbook is attractive.

The textbook makes use of icons throughout – these are a useful “road map”.

The information is presented in “bite‐sized” chunks.

The content is written in a conversational style.

The textbook includes bulleted lists.

The illustrations or graphics are of high quality.

Various ethnic groups are represented as part of the examples, case studies and illustrations in the book.

The activities are interactive.

Sufficient examples are included to explain theoretical concepts.

The cover of the book is appealing.

Students like the visual images.

Quadrant B: High Importance and High Satisfaction The language is easy to understand.

It is easy to understand the content.

It is easy to find information in the textbook.

The textbook has clear headings and subheadings.

The content is relevant to the course.

The activities promote analytical and critical thinking.

Using the textbook helps students to perform better in tests and exams.

The main ideas are summarised at the beginning of a chapter.

Practical, real‐life examples are used.

A summary is included at the end of a chapter.

Sufficient examples are included to explain theoretical concepts.

One concept at a time is introduced – students do not feel overwhelmed by the volume of information.

The text is interesting – students enjoy reading it.

Quadrant D: Low Importance and High Satisfaction Difficult words are explained in the glossary.

The textbook includes tables and flow charts.

Quadrant A: High Importance and Low Satisfaction The textbook includes case studies.

The textbook includes self‐evaluation questions or exercises, and provides model answers for them.

The illustrations are up to date (eg not dated in terms of dress or hairstyles unless a certain period is portrayed).

260

Nico Martins

6. Discussion and conclusion The focus of this study was to determine to what extent textbook characteristics enhance learning – as perceived by academics and students

The main characteristics of textbooks, which both academics and students perceive as important in making studying easier, are (quadrant B) as follows:

The content is relevant and easy to understand.

The language is easy to understand.

Practical, real‐life examples are used.

Activities promoting analytical and critical thinking.

It is easy to find information in the textbooks.

The textbook has clear headings and subheadings

Using the textbook helps students to perform better in tests and exams.

The main ideas are summarised at the beginning of a chapter.

A summary is included at the end of a chapter.

The content is relevant to the course.

Sufficient examples are included to explain theoretical concepts

It is important to note that academics and students are not that concerned about factors such as an appealing cover, an attractive textbook layout, the use of icons throughout, information presented in ‘’bite‐sized’’ chunks. They agree that illustrations are of high quality, the textbook is written in a conversational style, the activities are interactive and various other ethnic groups are represented in examples, case studies and illustrations in the textbook. In a nutshell, students and lecturers do not appear to be concerned about the visual presentation of the textbook.

In terms of the results of this research, a number of recommendations are made:

An important point is that academics who embark on writing a textbook should take note of the textbook characteristics students and academics feel do enhance learning and those characteristics that do not, those characteristics with low importance and low satisfaction, and those with low importance and high satisfaction.

Publishers should also note the research and cooperate closely with academics when planning textbooks for a specific module. Authors, academics and university administrators should take cognisance of these research results and develop textbooks and other supportive resources accordingly. It would further enhance the results if it can be confirmed by means of a qualitative study with interviews and focus groups.

The focus of this paper was on the analysis and interpretation of the two scale items, using quadrant analysis in order to determine the agreement (satisfaction) of students and lecturers and the importance of textbook characteristics and whether or not they enhance learning. It is clear from the results that lecturers as well as students agree that the textbook does enhance learning. However, there are a number of factors that should be taken into consideration when writing textbooks.

One of the limitations of the study was that the results did not include lecturers and students from all South African universities, universities of technology and nursing colleges. A second limitation is that all the survey results could not be reported in one article. Another important limitation is that not only one specific text book was evaluated, but textbooks in general were evaluated. However, these limitations create opportunities for future research and publications of results.

References Angell, R.J. Heffernan, T.W. and Megicks, P. (2008) “Service Quality in Postgraduate Education”, Quality Assurance in Education, Vol. 16, No. 3, pp 236–254.

Borg, I. and Mastrangelo, P.M. (2008). Employee Surveys in Management, Hogrefe & Huber, Toronto.

Douglas, J.A. McClelland, R. Davies, J. and Sudbury, L. (2009) “Using Critical Incident

261

Nico Martins Technique (CIT) to capture the voice of the student”, The TQM Journal, Vol. 21, No. 4, pp 305–318.

Douglas, J., Douglas, A.A. and Barnes, B. (2006) “Measuring Student Satisfaction at a UK University”, Quality Assurance in Education, Vol. 14, No. 3, pp 251–267 Ennew, C.T. Reed G.V. and Binks, M.R. (1993). Importance‐performance Analysis and the Measurement of Service Quality.

European Journal of Marketing, Vol. 27, No. 2, pp 59–70.

Higgs, P., Van Niekerk, L.J. and Van Wyk, B. (2010) “The Role of the Community in Curriculum Development in an African Context”, Progressio, Vol. 32, No. 2, pp 134–141.

Kim‐Prieto, C. and D’Oriano, C. (2011) “Integrating Research Training and the Teaching of Positive Psychology”, The Journal of Positive Psychology, Vol. 6, No. 6, November, pp 457–462.

Lianli, C.G. (2007) The Changing Context of Tertiary English Teaching in China and Teachers’ Responses to the Challenges.

Doctor of Education, University of Wollonyong.

Maccorr Research Solutions, [on line], http://www.macorr.com/quadrant‐analysis‐survey.htm [2 December 2013].

Martilla, J.A. and James, J.C. (1977) “Importance‐performance Analysis”, Journal of Marketing, Vol. 41, No. 1, pp 77–79 .

Watt, M.G. (2004) The Role of Curriculum Resources in Three Countries: the Impact of National Curriculum Reforms in the United Kingdom, the United States of America and Australia. Doctor of Philosophy, University of Canberra.

262

Using Focus Groups as a Diagnostic Tool With a Positivistic Approach Ellen Martins1 and Nico Martins2 1 Organisational Diagnostics, Johannesburg, South Africa 2 Department of Industrial and Organisational Psychology, College of Economic Sciences, Unisa, Pretoria, South Africa orgdia@iafrica.com martin@unisa.ac.za

Abstract: The downside of focus groups is that they can contribute to “groupthink”, encouraging the dominant voice in such a group to influence others to express themselves in the same way, regardless of whether this really reflects how they think. This research attempts to move beyond the traditional approach to focus groups by applying a more structured and positivistic approach. The purpose of the paper is to describe the research methodology that was followed to minimise the main weaknesses of focus groups, especially in a multicultural environment. Focus groups as a qualitative research methodology are described, paying attention to the characteristics, the influencing factors of groupthink and the conducting of focus groups in a multicultural environment, following a positive psychology approach. Moreover, the focus group phases, as an academic and applied research methodology, are described, namely, problem identification, planning, implementation and assessment. The weaknesses of focus groups are also discussed. The specific focus group process was applied in two different organisational settings where focus groups were deemed to be the best methodology for further analysis and interpretation of the results of two quantitative diagnostic surveys. For the purpose of the diagnostic effort, four questions were posed to participants during the focus groups to ensure validity. The focus was on positive aspects (a positive psychology approach) to encourage participation. The value of the current study lies in the richness of the data obtained in the cross‐cultural research setting with the diagnostic methods, techniques and processes employed. The systematic research procedure followed in order to gather data for further organisational diagnosis contributed to an increase in the validity and reliability of the organisational diagnosis process and the outcomes concerned. The approach used could be duplicated in other organisational settings.

Keywords: focus groups, groupthink, multicultural environment, validity, reliability, positive psychology research, focus groups weaknesses

1. Introduction The use of focus groups has grown considerably over the last two decades and is now one of the most widely used research methods in the social sciences (O’Raghallaigh, Sammon and Murhy 2012). Accounts of group interviews date back to the 1920s (Morgan in Rezabek 2000) but relatively little was published about focus groups in social science settings. Business interests were largely responsible for the continued interest in focus groups during the 1990s and 2000s (Morgan 1998). Organisational leaders are increasingly viewing organisational diagnosis as a key element in developing and maintaining competitive advantage (Lee and Brower cited in Martins and Coetzee 2009). Organisational diagnosis investigations that draw on concepts, models and methods from behavioural sciences are generally aimed at examining an organisation’s current state or at enhancing organisational effectiveness. The collaborative process embarked on between organisation members and organisational development consultants involves collecting pertinent information, analysing the data and drawing conclusions regarding potential change and improvement (Slocum and Hellriegel cited in Martins and Coetzee 2009). The focus group methodology is a qualitative method that can be used to collect information in an attempt to solve problems, find solutions and draw conclusions.

One of the disadvantages of focus groups is that they can lead to groupthink. The objective of this paper is to move beyond the traditional approach of focus groups by applying a more structured and positivistic approach to the information‐gathering process.

2. Focus groups within a qualitative research methodology approach As a form of qualitative research focus groups are basically a form of group interview (Morgan 1997). Roodt (2008) regards the use of focus groups as a diagnostic tool that is applied to establish a shared sense of a problem and its impact, as well as the solutions required. The reliance is on interaction within the group of participants, based on topics that are supplied by the researcher who typically takes the role of a moderator.

The task of the moderator is to promote interaction among a collection of participants who were brought

263

Ellen Martins and Nico Martins together to discuss and shed light on a particular issue, concern or topic (Rezabek 2000). The main characteristic of focus groups is their explicit use of group interaction to produce data and insights that would be less accessible without the interaction found in a group (Morgan 1997).Their use in applied research outside academic settings has grown rapidly due to social scientists’ ability to borrow from an established set of practices in marketing research (e.g. by Goldman and MacDonald 1987; Greenbaum (1993) and Hayes and Tatham (1989) all cited in Morgan 1997). Another major driving factor of the use of focus groups in the social sciences has been the ability of social scientists to adapt this technique for their own purposes (e.g. Krueger, 1994, Morgan, 1993a, Steward and Shamdasani, 1990; Vaughn, Schumm and Sinagub, 1996 all cited in Morgan 1997). It thus appears that considerable borrowing and innovation was characteristic of focus groups in the past.

Some factors that might influence the validity and reliability of data collection in focus groups are groupthink and conducting focus groups in a multicultural setting/environment. Groupthink can be defined as “the practice of thinking or making decision as a group in a way that discourages creativity or individual responsibility” (http://oxforddictionaries.com/definition/american_english/groupthink 1/8/2013). In focus groups this means that the dominant voice in the group might influence others to express themselves in the same way, regardless of whether such expression really reflects how they think (Rothewell and Sullivan 2005 cited in Martins and Coetzee 2009).

A multicultural workforce consists of a wide range of backgrounds with culturally distinguishing traits, such as age, race, gender, marital status and national origin, which bring a broader range of perspectives on issues and challenges. It might, however, also emphasise language, cultural and communication barriers. A diverse workforce represents increased potential for discrimination. It naturally occurs that if people with obvious distinguishing traits are put together, there is an increased risk that employees with biases and prejudices could use them against others (Kokemuller 2013a). Focus groups might thus be affected by this type of intolerance and non‐acceptance of multicultures. Another challenge presented by a multicultural workforce is that employees from different cultures sometimes speak different languages as their first language making it difficult to encourage communication. This can affect both formal and informal communication and cohesion (Kokemuller 2013b) by causing some people not to speak their minds fearing that they might be misunderstood, or it could lead to possible conflicts.

From a research paradigm point of view, focus groups lend themselves to approaching a research topic from a positive psychology perspective. Robbins (2013) reiterates that positive psychology has the goal of improving the widespread overemphasis within contemporary psychology on negative phenomena. Positive psychology essentially presents a limited version of human psychology by “emphasizing the positive and ignoring the shadow of human reality”. The questions posed in the focus groups can thus be formulated to emphasise the positive when gathering data. This aspect will be explained in more detail in the focus group design and methodology discussion.

3. Research methodology of focus groups The focus group research method is valuable to gain shared understanding while allowing for individuals to voice their opinions (Tremblay in O’Raghallaigh et al 2012). The level of interaction between participants and the synergy in the group are the main characteristics that distinguish focus groups from other approaches (Gibbs; Kritzinger in O’Raghallaigh et al 2012). According to Krueger and Casey (in O’Raghallaigh et al 2012), the unit of analysis is “at the group level rather than the individual participant level”. There are certain obstacles in group sessions that might affect the free sharing of individual opinions, which will be addressed in the section on the weaknesses of focus groups. Focus groups as an academic research method consist of four different steps, which can be displayed as follows: Table 1: Focus group phases as an academic research method Phases Phase 1: Problem identification Phase 2: Planning Phase 3: Implementation Phase 4: Assessment

Academic and applied research Purpose and research questions Research design Data collection and moderating Data analysis and reporting

Source: Adapted from Morgan (1998)

264

Ellen Martins and Nico Martins Some of the decisions and methods that need to be made, as found in the literature are briefly discussed below.

3.1 Problem identification The use of focus groups is related to the purpose(s) of the research study. At the problem identification phase the main object is to define a purpose/goal. Being an adaptable method, focus groups can have many purposes. This early stage benefits from the process of exploration and discovery. Discussions will be relatively unstructured and open‐ended (Morgan 1998). The statement of the specific objective(s) of the research is a very important part of the research process, as it becomes the driver of how the research will be conducted.

Ultimately, the success or failure of the research process is judged against this stated objective. The statement is usually included in a written proposal and enables the facilitators to get buy‐in from the management team to conduct the focus groups (Greenbaum 2000).

In academic research the problem identification is about generating questions. Some examples of when focus groups can be used in the organisational development field (which is the main discipline to be reported in this article) are to test new concepts, to develop a questionnaire, to generate ideas or support brainstorming (Edmunds 1999) and to clarify survey results by gaining a better understanding of the meaning behind the survey results.

Clear and thoughtful questions are the foundation for high‐quality focus group research. Certain guiding principles form the basis of question formulation. The first principle is to ask questions in a conversational manner. The wording of the questions must be direct, comfortable (not awkward) and simple in the language that is appropriate for the intended audience. Clear, brief and reasonable questions are effective, just as short, one‐dimensional and jargon‐free questions are usually clear and understandable. Each question should be limited to one dimension (Krueger 1998). The questions should provide data that will be directly related to achieving the purpose of the focus groups.

3.2 Planning This stage is about finding the best way to achieve the goals or purposes of the focus groups (Morgan 1998).

The key to planning in focus groups is to think through the whole project from start to finish. The detailed elements that form part of the planning process are the types of person to be invited to participate, the group size, the number of groups, the setting up of the sessions in terms of location, venue, tools and equipment to be used, time lines for the project and timing per focus group session (Morgan 1998; Greenbaum 2000).

In terms of the types of person, focus groups rely mostly on purposive sampling, whereby participants are chosen according to the project’s specific goals. The general goal of focus groups is to gain insight and understanding by gathering in‐depth data from people and this requires selecting a purposive sample that will generate the most productive discussions (Morgan 1998).

One aspect related to the types of person in a focus group in a multicultural or international setting is the possible language barrier (Edmunds 1999). The focus group can be conducted in the official business language of the organisation and if some people are not comfortable in the language, an interpreter might be used, especially in groups consisting of lower levels of staff and also depending on their literacy levels.

Homogeneity (the need for compatible participants) and segmentation play a role in the composition of groups. Mixed groups may spend more time getting to know each other and building trust before they feel safe to share information. Segmentation refers to dividing up the composition of a set of focus groups by carefully selecting categories of participants that fit the research purpose (Morgan 1998).

Timing guidelines relate to the duration of each focus group session and the time allocated for the discussion of each topic. Greenbaum (2000) suggests that, as a general rule, a focus group should not be longer that two hours and time should be allowed between sessions to clean up the room. The moderator should ensure that all topics are covered during the session.

Morgan (1998) suggests that for most purposes six participants would be considered a relatively small group and ten would be a relatively large group. However, selecting a group size of six to ten participants is too

265

Ellen Martins and Nico Martins vague for most purposes. Smaller groups give each person a chance to talk, but place a greater burden on each person to carry the conversation. On the other hand, large groups might lead to group think. The reality is that some participants might not feel comfortable speaking in front of other participants while others will be very outspoken (Morgan 1998).

Typically, the number of groups is three to five if people are only moderately diverse and the topic is only moderately complex. When groups are more diverse, the responses might be more diverse and it will take considerably more groups to hear what people have to say before reaching a point of “theoretical saturation” (Morgan 1998).

Moderators usually know what types of tool to use at the right time and whether to use a highly structured or less structured approach. Effective moderators should be proficient and flexible with tools and techniques by judging accurately what the group needs. Effective moderators/facilitators should also be ‘light on their feet’, that is, able to make spontaneous adjustments to the process based on signals from the group (Weaver and Farrell 1997). Equipment such as flipcharts, pens and post‐it notes are useful to capture what people say, think or feel in focus groups.

3.3 Implementation The role played by the moderator in focus groups entails guiding the discussion and gathering data by listening to the participants’ views, but they should not participate in the discussion. One of the most influential factors affecting the quality of focus group results is moderator respect for participants. The moderators must believe that participants have valuable wisdom, regardless of their level of education, experience or background. Each moderator should bring their own unique skills and abilities to the moderating experience. If moderators are comfortable and natural, participants will feel more relaxed and willing to share (Krueger 1998).

Nevertheless there is a degree of moderator involvement, such as subtle body language, friendliness and a sense of humour. Although the moderator should not share personal opinions in an attempt to foster greater sharing among participants, it is desirable for moderators to share factual or demographic information about themselves. This occurs if the group needs reassurance that all participants and the moderators have critical factors in common. Conscious consideration should be given to factors such as gender, language, race, age and technical knowledge.

When conducting research in an organisation there are two choices of moderators, namely professional moderators or internal staff member moderators. The advantages of making use of professional moderators are that they bring experience in working with people and they know how to get information without leading participants, how to make people feel comfortable, how to refocus questions and keep the discussion on track.

They create an unbiased and neutral atmosphere, and they often specialise in a certain area, such as the service sector, organisational issues and non‐profit and public sector aspects, hence serving as a cross between moderator and consultant. In this cross role they can provide advice regarding the research methodology (e.g. setting clear goals and objectives and determining the specifications of the research) (Greenbaum 2000). The disadvantage is that the organisation has to pay for their services (Krueger 1998).

In order to capture as much data as possible, it is advisable that each focus group has two moderators. While both will be listening and asking questions, the key moderator will be facilitating group behaviour and the assistant moderator will be capturing the ideas on a flip chart or by taking notes (Krueger 1998).

3.4 Assessment The main purpose of the analysis and reporting phase is to retype, categorise, consolidate and analyse the qualitative data that was collected during the focus groups. The final stage is to compile a report and provide feedback to the organisation.

The problem/concern being investigated in the study drives the analysis. The researcher needs to be aware of the trap of under‐ or overanalysing the data, and of elaborate analysis related to a trivial problem or an inadequate analysis of a complex problem or major concern. The focus should be on the trends and patterns that reappear among various focus groups (Morgan 1998; Krueger 1998).

266

Ellen Martins and Nico Martins Reporting on and sharing the results relates to being clear on the purpose and getting to the point quickly, regardless of the type of reporting (written or spoken, formal or informal, one‐on‐one or to a large group). The primary point of reports should be to enlighten (bring knowledge, provide understanding and convey information on what is important, valuable and new) the audience (Krueger 1998). Analysis and reporting (assessment) is the last step in focus group research methodology. The following section discusses the weaknesses of focus groups.

4. Weaknesses of focus groups Although the use of focus groups in research is a powerful method, there are certain weaknesses that the researcher needs to be aware of and to try to eliminate as far as possible in order to conduct valid and reliable research. The main weaknesses of focus groups are summarised in table 2. To these weaknesses may be added groupthink and difficulties in dealing with multicultural groups

In the current research conducted in two multicultural organisations, the researchers tried to minimise the weaknesses inherent in the focus groups. In the next section the focus group design and methodology applied as well as the way in which the weaknesses were addressed, will be discussed.

5. Focus group design and methodology using a positive psychology approach The focus group design and methodology that were used in the two multicultural organisations are described below in relation to the theoretical discussion on focus groups as a research method (section 3).

5.1 Purpose and research questions In both organisations quantitative organisational assessments were conducted and, in both instances, the survey results created the need for further investigation and understanding of the results. It was proposed that the focus group methodology be used for the main purpose of elaborating and clarifying the organisations’ current state of functioning with a focus on the future. The outcome should indicate to the management teams what they are doing well and how to improve their organisational effectiveness.

Taking the guidelines in section 3.1 into consideration, as well as a positive psychology perspective, the following four questions were formulated and used in the focus groups in both organisations:

1. What are the characteristics of an employer of choice? (Question 1) 2. What excites you about your work (makes you happy) at the organisation? (Question 2) 3. What does the organisation do particularly well? (Question 3) 4. What would you change at the organisation if you had the opportunity? (Question 4)

From the above it is apparent that the first three questions focus on current positive aspects and the last question has a focus on future improvements instead of focusing on what is wrong or negative in the organisation.

5.2 Research design In both organisations with English as the business language, guidelines on the composition of the focus groups, such as selecting employees from all levels, business units, gender, race and national origins were discussed with the human resources practitioners. In both instances purposive sampling was used to select and invite participants. In the Middle Eastern organisation 16 focus groups consisting of 147 participants were conducted and in the Namibian organisation eight focus groups consisting of 58 participants were conducted. Most of the focus groups consisted of eight to ten staff members. The duration of each focus group was two hours.

5.3 Data collection and moderating The focus groups were conducted by two professional moderators with consulting and moderating experience.

The common factor between the moderators and the participants was that they were from several different national origins (cultures), hence creating an understanding of a multicultural environment. After a brief introduction, the purpose of the session, the ground rules for the session and the roles of the moderators were explained. Groupthink and multicultural differences were controlled as followed:

267

Ellen Martins and Nico Martins

Each of the four questions was separately listed on flipchart paper.

The participants were requested to write down their answers individually on oval post‐it notepaper. In order to maintain confidentiality they were asked not to discuss the answers with their colleagues (figure 1).

Figure 1: A multicultural focus group writing down their answers

The oval post‐it notes were collected by the researchers (moderators) and quickly reviewed. The responses were then provisionally grouped by the managerial dimension/theme identified in the Burke‐ Litwin (1992) model (figure 2):

Figure 2: Example of data collection

All misunderstandings or unclear answer were then discussed. Notes were made on the oval post‐it notes, or on separate sheets of flipchart paper. During this phase the contributions made by the dominant participants were facilitated and the participation of those participants who appeared to be shy was encouraged.

The above process was repeated for each of the four questions.

The methodology followed ensured that no one person came to dominate the group and allowed for all participants to obtain clarification on any of the questions. The sequence of questions also started with more general questions to encourage participation (Waclawski and Church 2002).

5.4 Data analysis and reporting A thematic content analysis method was used to analyse the results, which were linked to a diagnostic organisational model. Thematic content analysis is defined by Braun and Clarke (2006) as a qualitative diagnostic method for identifying, analysing and reporting on patterns or themes detected in data. In the context of organisational diagnostic approaches, the factual bases of data are established by separating the factual from the non‐factual data (which comprise perceptions, skewed facts or interpretations/inferences).

Confirmed facts that bear some similarity to one another are then clustered into separate categories, or groups of facts. The categories are then labelled so as to depict accurately the core character of the group of

268

Ellen Martins and Nico Martins facts and to clearly differentiate one category from another (Van Tonder and Dietrichsen 2008). The relationship between different categories is established by identifying a sufficiently detailed and valid diagnostic model or framework of organisational dynamics that addresses and incorporates the majority of the categories identified during the clustering process. The chosen diagnostic model or framework is then conceptually overlaid on the categorical data and the cause–effect relationships in the diagnostic model are transferred to the data (Van Tonder and Dietrichsen 2008). The provisional grouping of data per managerial dimension/theme simplified the data‐recording process. In line with the research objective, the Burke‐Litwin (1992) model of organisational performance and change was applied to categorise and analyse those themes regarded as confirmed facts that emerged from the focus group participants. In both organisations the subsequent report together with recommendations led to a better understanding of the development areas that were highlighted in the quantitative surveys.

6. How weaknesses were addressed The main weaknesses that could have an impact on the focus group research methodology in this research, and as summarised by O’Raghallaigh et al (2012), were addressed or minimised as outlined below (table 2): Table 2: Main weaknesses of focus groups as a research method Weaknesses Groupthink

Multicultural groups

Might become “moan and groan” sessions Difficult to assemble and encourage participation Loss of control over the data produced

Description The dominant voice in the group might influence others to express themselves in the same way, regardless of whether such expression really reflects their opinion.

Weaknesses related to overcoming cultural and communication barriers, as well as the potential for discrimination

When discussions focus on what is wrong/negative in the organisation Focus groups can be difficult to assemble and to obtain a representative sample that is willing to participate.

Analysis of the data can be difficult

Other than keeping participants focused on the topic, the moderator has less control over the data produced as participants must be allowed to talk to each other, ask questions, and express doubts and opinions It cannot be assumed that participants are expressing their personal views as they may be expressing those that they perceive to be most acceptable to the group.

The discussions in focus groups are never fully confidential or anonymous because the content is being shared with others in the group, resulting in some participants deciding to withhold relevant information.

The open‐ended nature of the questions and the wide‐ranging responses can make the analysis of the data difficult.

Time constraints and complexity of discussions

The time available for discussions in a focus group is limited and this means that some complex issues may not

Responses from group members are not independent of one another Results are not fully confidential or anonymous

269

How weaknesses where addressed / minimised The researchers controlled the possibility of groupthink by using the approach described in section 5.3.

Moderators and group members introduced themselves by referring to their cultural background.

Possible discrimination and biases were to a large extent eliminated because responses remained anonymous.

Writing down ideas alleviated the fear in participants who do not speak English as their first language.

A positive psychology approach focuses on a positive way of viewing the organisation.

Human resources assisted in selecting the sample.

By allowing each individual to write down their ideas, a lack of participation resulting from shyness was eliminated.

After collecting ideas on post‐it notes group discussion was encouraged to further clarify and add more information. This information was captured on a flip chart.

Independence of ideas is obtained by having individual participants write down their ideas on post‐it notes without influencing each other.

Collecting the post‐it notes and roughly categorising them on the wall ensures confidentiality because participants do not know who wrote down what.

Initial categorising under “cloud” themes made analysis of the data easier. Secondly, an organisational model was used to categorise the data.

The limited number of questions asked (4) and with a specific focus, collecting written ideas and allowing open discussion ensured that topics were

Ellen Martins and Nico Martins Weaknesses

Description receive sufficient attention and may not be covered adequately.

How weaknesses where addressed / minimised appropriately covered.

Requirement for a skilled and experienced moderator

A skilled and experienced moderator is needed for an effective research study.

Two professionally qualified and experienced moderators were used to moderate the focus groups.

Source: Adapted from O’Raghallaigh et al. (2012)

7. Conclusions The methodology applied ensured that a single person could never dominate the group and allowed for all participants to obtain clarification on any of the questions. By carefully managing the possibility of groupthink, the researchers were able to explore any answers that they felt required more detailed clarification by obtaining inputs from the group as needed. The difficulties involved in facilitating focus groups were to a large extent minimised by the process. The methodology followed a positive psychological approach which minimised many of the typical problems associated with focus groups. As with most organisational focused research, the results cannot be generalised to all kinds of organisations. What came out of this study that could be applied, however, is the methodology.

References Braun, V. and Clarke, V. (2006) Doing Thematic Analysis in Psychology. Qualitative Research in Psychology, Vol. 1, No. 3, pp 77‐101.

Burke, W.W. and Litwin, G.H. (1992) A Causal Model of Organisational Performance and Change. Journal of Management, Vol. 8, No. 3, pp 523‐546.

Edmunds, H. (1999) The Focus Group Handbook, NTC Business Books, Lincolnwood (Chicago), Illinois.

Greenbaum, T.L. (2000) Moderating Focus Groups: A Practical Guide for Group Facilitation, Sage, Thousand Oaks, CA. Johnson, R.B. and Onwuegbuzie, A.J. (2004) Mixed Methods Research: A Research Paradigm Whose Time Has Come.

Educational Researcher, Vol. 33, No. 7 (Oct., 2004), pp. 14‐26, Published by: American Educational Research Association Stable URL: http://www.jstor.org/stable/3700093 (Accessed: 10 January 2013).

Kokemuller, N. (2013a) Advantages and Disadvantages of a Multicultural Workforce. Small Business by Demand Media.

Available from: http://smallbusiness.chron.com/advantages‐disadvantages‐multicultural‐workforce‐18903.html (Accessed: 10 January 2013).

Kokemuller, N. (2013b) Advantages and Disadvantages of Having a More Diverse Workforce. Small Business by Demand Media. http://smallbusiness.chron.com/advantages‐disadvantages‐having‐diverse‐workforce‐22935.html Krueger, R.A. (1998) Developing Questions for Focus Groups, (Focus Group Kit 3), Sage, Thousand Oaks, CA. Martins, N. and Coetzee, M. (2009) Applying the Burke‐Litwin Model as a Diagnostic Framework for Assesing Organisational Effectiveness, SA Journal of Human Resource Management/SA Tydskrif vir Menslikehulpbronbestuur, Vol. 7, No. 1, pp 1‐13. Art. #177, 13 pages. DOI: 10.4102/sajhrm.v7i1.177. (Also available at: http://www.sajhrm.co.za) Morgan, D.L. (1997) Focus Groups as Qualitative Research, 2nd edition, (Qualitative Research Methods: Series 16), a Sage University Paper, Sage, Thousand Oaks, CA. Morgan, D.L. (1998) The Focus Group Guidebook, (Focus Group Kit 1), Sage, Thousand Oaks, CA. O’Raghallaigh, P., Sammon, D. and Murhy, C. (2012) “Using Focus Groups to Evaluate Artefacts in Design Research”, Proceedings of the 6th European Conference on Information Management and Evaluation, edited by T. Nagle, University of Cork, Ireland, 13‐14 September 2012, pp 251‐257. http://oxforddictionaries.com/definition/american english/groupthink (Accessed: 8 January 2013) Rezabek, R.J. (2000) Online Focus Groups: Electronic Discussions for Research. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research. Available from http://www.qualitative‐research.net/fqs‐texte/1‐00/1‐00rezabek‐e.htm 1 Robbins, B. (2013?) The Negative Shadow Cast by Positive Psychology: Contrasting Views and Implications of Humanistic and Positive Psychology on Resiliency. Available from: http://www.academia.edu/696747/The_Negative_Shadow_Cast_by_Positive_Psychology_Contrasting_Views_and_I mplications_of_Humanistic_and_Positive_Psychology_on_Resiliency (Accessed: 11 January 2013).

Roodt, G. (2008) “Survey research and data feedback”, in Organisation Development: Theory and Practice, edited by C. L. van Tonder and G. Roodt, Van Schaik, Pretoria, pp 251‐268.

Van Tonder, C. and Dietrichsen, P. (2008) The Art of Diagnosis. In Organisation Development: Theory and Practice edited by C.L. van Tonder and G. Roodt, Van Schaick, Pretoria, pp 133‐166.

Waclawski, J. and Chruch, A.H. (2002) Organizational Development: A Data‐driven Approach to Organizational Change.

Jossey‐Bass, Boston, MA. Weaver, R.G. and Farrell, J.D. (1997) Managers as Facilitators, Berrett‐Koehler, San Francisco, CA.

270

Characterizing Intuition in Information Systems Research Dan McAran and Sharm Manwani Henley Business School, Henley‐on‐Thames, UK danmca@hotmail.com sharm.manwani@henley.ac.uk

Abstract: The study of intuition is an emerging area of research in psychology, social sciences, and business studies. It is increasingly of interest to the study of management, for example in decision‐making as a counterpoint to structured approaches. Recently work has been undertaken to conceptualize a construct for the intuitive nature of technology.

However to‐date there is no common understanding of the term intuition in information systems (IS) research. This paper extends the study of intuition in IS research by using exploratory research to categorize the use of the word “intuition” and related terms in papers published in two prominent IS journals over a ten year period. The entire text of MIS Quarterly and Information Systems Research was reviewed for the years 1999 through 2008 using searchable PDF versions of these publications. As far as could be determined, this is the first application of this approach in the analysis of the text of IS academic journals. The use of the word “intuition” and related terms was categorized using coding consistent with Grounded Theory. The focus of this research was on the first two stages of Grounded Theory analysis ‐ the development of codes and constructs. Saturation of coding was not reached: an extended review of these publications would be required to enable theory development. Over 400 incidents of the use of “intuition”, and related terms were found in the articles reviewed. The most prominent use of the term of “intuition” was coded as “Intuition as Authority” in which intuition was used to validate a research objective or finding; representing approximately 37 per cent of codes assigned. The second most common coding occurred in research articles with mathematical analysis, representing about 19 per cent of the codes assigned, for example where a mathematical formulation or result was “intuitive”. The possibly most impactful use of the term “intuition” was “Intuition as Outcome”, representing approximately 7 per cent of all coding, which characterized research results as adding to the intuitive understanding of a research topic or phenomena. This research contributes to a greater theoretical understanding of intuition enabling insight into the use of intuition, and the eventual development of a theory on the use of intuition in academic IS research publications. It also provides potential benefits to practitioners by providing insight into and validation of the use of intuition in IS management. Research directions include the creation of reflective and/or formative constructs for intuition in information systems research.

Keywords: intuition, grounded theory, information systems research

1. Introduction This research seeks to characterize the use of intuition in Information Systems (IS) research with the ultimate target of developing a theory concerning such use. The entire text of MIS Quarterly (MISQ) and Information Systems Research (ISR) issues was reviewed for the years 1999 through 2008 using searchable PDF text versions of these publications for the term “intuition” and related terms. As far as could be determined, this is the first use of this procedure in the analysis of text from academic journals.

The lack of prior theory and the behavioural use of intuition in research support a Grounded Theory approach, which originated in sociology. Grounded Theory has subsequently been applied in many disciplines including business studies (Goulding 2002).

The focus of this paper is on developing concepts and categories rather than the development of theories, which requires saturation to be reached in the coding process – which has not occurred in this research. It is hoped that by obtaining an understanding of the use of “intuition” in the IS literature, insight will be obtained to facilitate the creation of a construct to measure the “intuitive” nature of an IS technology product.

This research, based on a new method and extensive review of targeted literature, provides the foundation for a greater understanding of the use of intuition in information systems. It offers insights both for academics and practitioners, most notably for those concerned with user experience design.

2. Literature The Literature Review is divided into two sections: a review of general literature on intuition and emergent research on intuition in IS.

271

Dan McAran and Sharm Manwani

2.1 General literature on intuition Shirley and Langan‐Fox (1996) reviewed the psychological literature and found limited research into intuition and noted the increasing importance of intuition in modern society “and particularly in business settings” (p.

563). They define intuition “as a feeling of knowing with certitude on the basis on inadequate information and without conscious awareness of rational thinking” (p. 564).

Kahneman (2002), in his Noble Prize lecture, also recognized the importance of intuition in human cognition: From its earliest days, the research that Tversky and I conducted was guided by the idea that intuitive judgments occupy a position – perhaps corresponding to evolutionary history – between the automatic operations of perception and the deliberate operations of reasoning (p. 450) Insight into the nature of the human cognition, intuition, design, and computer technology can also be found in Simon’s (1996) seminal work The Sciences of the Artificial. Simon sees computer technology potentially designed in the ‘image of man’ (p. 21). Simon (1960) sees no restrictions on the capability of computer technology to parallel human problem solving: In principle, the potentialities of a computer for flexible and adaptive cognitive response to a task environment are no narrower or no wider than the potentialities of a human (p. 24) Use of intuition is a method of human problem solving: this allows us to speak of the ‘intuitive’ capabilities of computer technology.

2.2 Emergent intuition research in IS Simon (1997) clearly envisaged ‘intuitive’ computer technology: In medical diagnosis where there has been much study of both human intuition and expert systems… recognition capabilities, the if‐then pairs, account for their intuitive or judgement ability; their inferencing processes account for their analytical ability (p. 135) O’Brien, et al. (2010) provides the following definition of ‘Intuitive Human‐Computer Interaction (HCI)’: interactions between humans and high technology in lenient learning environments that allow the human to use a combination of prior experience and feedforward methods to achieve their functional and abstract goals. (p. 107) Prior to the work of O’Brien, et al. (2010), the issue of the intuitiveness in computer technology has been discussed primarily in reference to the design of the computer technology interface, but there has been recognition that the concept of what is intuitive is related the specific technology in question.

An intuitive interface … is not one that can be used without any prior knowledge or learning, but one that builds as much as possible on existing general and technology‐specific user knowledge, requires minimum learning for a given task complexity…(Bullinger, Ziegler, and Bauer 2002 p. 4) Shaw & Manwani (2011) found a link between the intuitiveness of a specific technology function and its adoption. Shaw (2011) provides evidence to support the concept of intuitive in relation to the adoption of Electronic Medical Records by physicians: It hasn‘t changed the way I worked. …it‘s very intuitive as far as the way the family physicians practice. Everything on the screen is – it just automatically fits in.

The system works exactly the way I work. That‘s the beauty of that particular system. (p. 159) The concept of intuitive technology is an emergent area of research (O’Brien, et al. 2010; Blackler, Hurtienne 2007). Understanding the use of intuition in IS academic research would potentially provide insight into the nature of intuitive IS technology.

272

Dan McAran and Sharm Manwani

3. Methodology 3.1 Grounded theory Grounded Theory was developed by Glaser and Straus to ground theory in data and in reaction to the established theories then dominating sociology (Goulding 2002). The four stages of the Constant Comparative Method, central to the practice to the practice of Grounded Theory, identified by Glaser and Straus (2009), are: “(1) comparing incidents applicable to each category, (2) integrating categories and their properties, (3) delimiting the theory, and (4) writing the theory” (p. 105). The process by refining categories should continue until theoretical saturation has occurred (Glaser and Straus 2009).

There is another way of approaching the analysis of the text: the references to “intuition” and related terms can be viewed as an attempt at a form of Hermeneutic Understanding. McAuley (2004) notes, “Lying at the heart of hermeneutics are issues of intuition, interpretation, understanding the relationship between the researcher and the subject of the research and the reader” (p. 192).

Gummerson (2000) notes that the number of cases needed for Grounded Theory is determined by “saturation – that is, the diminishing marginal contribution of each additional case” (p. 96). In this research saturation has not been reached, the process of assigning codes and refinement has begun.

3.2 Research method Online versions of MISQ and ISR, provided by EBSCOhost Business Complete, were reviewed for the period 1999 to 2008 (inclusive). The upper limit of 2008 was chosen because of the ISR online full text access embargo of 35 months. PDF versions of articles were searched three times for the term “intu” which enabled detection of the word “intuition” and related terms; the relevant text was extracted and coded using Excel. For MISQ, 7 PDF files searched were not found or unusable and were excluded from the research.

Coding of the text which included the words “intuition” and related terms was based on the context in which the term was found and revised several times. In almost all cases the text before and following the term was reviewed, usually the text reviewed was a paragraph or less. The coding was done based on concepts that emerged from the text, consistent with Grounded Theory. Where two or more references to “intuition” or a related term where found close to each other, and were of similar meaning, they were coded with one code.

References to “intuition” and related terms that appeared in Table headings and labels, and which provided no new additional information, were excluded from coding.

Text specifically excluded from the target journals appears in Table 1 below: Table 1: Excluded JOURNAL ITEms Excluded Journal Items In Memoriam Notice About Our Authors notes Acknowledgment of Reviewers.

The Gordon B. Davis Symposium.

Working Conferences (Listing of)

4. Results The process of code generation and refinement resulted in 14 codes; the summary of codes appears in Table 2 and the eventual definitions created appear in Table 3.

One potentially confounding factor in the analysis of articles appearing in MISQ was the use of Executive Summaries, which potentially distorted comparative analysis of the number of articles in the two publications under study. The use of Executive Summaries was discontinued in 2002.

There were 62 Executive Summaries in the issues of MISQ reviewed; the Executive Summaries were excluded from coding; 320 MISQ articles (excluding Executive Summaries) and 231 ISR articles were reviewed.

Table 6 identifies the frequency of the double coding by publication.

273

Dan McAran and Sharm Manwani Table 2: Summary of codes assigned Code Intuition as Authority Intuition as Authority – Postulate Intuition as Authority – Result Intuition Reference Intuition in Math

Intuitive Technology Intuitive Decisions Counter –Intuitive Management Intuition Intuition as Strategy Intuition as Outcome Intuitive Culture Cognitive Intuition Meyers Briggs Intuition Totals

ISR Total Codes 11 38

Percent of Total ISR Codes 5.4% 18.5%

MISQ Total Codes 25 34

Percent of Total MISQ Codes 12.7% 17.3%

Total Codes 36 72

Percent of Total Codes 8.9% 17.9%

27

13.2%

15

7.6%

42

10.5%

7 62

3.4% 30.2%

12 14

6.1% 7.1%

19 76

4.7% 18.9%

3 4 20 2

1.5% 2.0% 9.7% 1.0%

9 0 25 1

4.6% 0% 12.7% 0.5%

12 4 45 3

3% 1% 11.2% .7%

1 15 0 8 7

0.5% 7.3% 0% 3.9% 3.4%

33 13 2 13 1

16.7% 6.6% 1% 6.6% 0.5%

34 28 2 21 8

8.5% 7.0% 0.5% 5.2% 2%

205

100%

197

100%

402

100%

Table 3: Definitions of codes ID 1

Code Intuition as Authority

2

Intuition as Authority – Postulate

3

Intuition as Authority – Result

4

Intuition Reference

5

Intuition in Math

6

Intuitive Technology

7

Intuitive Decisions

8

Counter–Intuitive

9

Management Intuition

10

Intuition as Strategy

Definitions This coding was used where “intuition” or related terms was used to validate a statement and where the statement validated could not be related to the postulates or research results.

This coding was used where “intuition” or related terms were used to validate a statement and where the statement validated could be related to the postulates or model of the research.

This coding was used where “intuition” or related terms was used to validate a statement and where the statement validated could be related to the results or consequences of the research.

This coding was used where “intuition” or related terms were found in the title of references appearing in the article.

This coding was used when “intuition” or related terms was used to validate a mathematical formulation or result.

This coding was used when there was reference to technology or a computer/web interface as being intuitive.

This coding was used when there was a reference to intuition as used to make decisions or where the context was such that it could be inferred that intuition was being used to make decisions.

This coding was used when the term “counter‐intuitive” was used or where terms such as “not intuitive” or similar phrasing was used.

This coding was used where text was found which indicated intuition was used by management or there was reference to management intuition.

This coding was used where text was found which indicated intuition had been used as the basis for performance of a specific task.

274

Dan McAran and Sharm Manwani ID 11

Code Intuition as Outcome

12

Intuitive Culture

13

Cognitive Intuition

14

Myers Briggs Intuition

Definitions This coding was used where text was found which indicated an event, occurrence or research result added to the intuitive understanding of a topic or phenomena.

This coding is used when there is direct reference in the text to a culture which is intuitive.

This coding is used when there was reference to the psychological process, or to the biological process, of human intuition.

This coding is used when there is a reference to intuition as used in the Myers Briggs instrument.

Table 4: Summary of double coding Summary of Double Coding ISR MISQ Total Intuition as Authority Intuition as Strategy ‐ 11 11 Intuition as Outcome Intuitive Technology

Cognitive Intuition Intuition as Strategy ‐ 1 1 Intuition as Authority – Postulate Intuition in Math

Intuition as Authority – Results Intuition in Math ‐ 1 1 Intuition as Authority –Result Intuition as Strategy 6 5 11 Intuition as Authority ‐ Postulate Intuition as Strategy

Intuition as Authority – Postulate Intuition Reference 7 ‐ 7 Cognitive Intuition Intuition as Outcome

Intuition as Authority ‐ Results Intuition as Outcome ‐ 8 8 Intuition as Authority ‐ 6 6 Cognitive Intuition

1 ‐ 1

‐ 1 1 1 ‐ 1

‐ 1 1 Totals

15

34

49

Miles and Huberman (1994) state multiple coding is useful in exploratory research, particularly when they reflect both the “descriptive and inferential” (p. 65). In this research inferential can, in the case of the combined “Authority/Math” and “Authority/Strategy” combinations, be interpreted as the “Authority” component and “Math”/“Strategy” as the descriptive component. These codes represent over 80 per cent of the double codes assigned.

The combination of Intuition as Authority – Postulate/ Intuition in Math was found in theoretical articles where there was a considerable amount of mathematical formulation presented as part of the theory development.

In addition, there was a dual presentation of results in a number of articles with textual explanation and interpretation of mathematical expressions, which resulted in the assignment of the “Intuition as Authority – Results”/“Intuitive in Math” coding.

In the coding there is an apparent strong link between “Intuition as Authority”/“Intuition as Authority – Postulate”/“Intuition as Authority – Result” and “Intuition as Strategy”. There were twenty‐five instances where these code combinations were assigned to one textual extraction.

As will be outlined in the Discussion section of this paper the double coding also reflects the articles in ISR on econometrics/pricing and the articles in MISQ with a management/decision focus. Table 5 below summaries the frequency of coding by year.

Figure 2 shows a regression line with postive slope to the number of total codes assigned to the ISR and MISQ articles in the period 1999 to 2008 – the inference could be made that there is increasing use of “intuition`” in IS academic research.

275

Dan McAran and Sharm Manwani Table 5: Frequency of total codes by year Year

MISQ Percentage of Total

ISR

Percentage of Total Total Percentage of Total

1999 2000

16 27

8.1% 13.7%

5 0

2.4% 0.0%

21 27

5.2% 6.7%

2001 2002 2003 2004 2005 2006 2007 2008

3 6 16 13 20 42 16 38

1.5% 3.1% 8.1% 6.6% 10.2% 21.3% 8.10% 19.3%

25 8 31 27 46 18 27 18

12.2% 3.9% 15.1% 13.2% 22.4% 8.8% 13.2% 8.8%

28 14 47 40 66 60 43 56

7.0% 3.5% 11.7% 10.0% 16.4% 14.9% 10.7% 13.9%

Total

197

100%

205

100%

402

100.0%

Num ber of C odes A s s ig ned by Y ear

Number of Codes Assigned

70 60 50 40

MIS Q IS R

30

C om bine d

20 10 0 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

Figure 1: Number of codes assigned by year (created using Microsoft Excel)

Figure 2: Regression line of combined codes for MISQ and ISR by Year (created using Microsoft Excel)

276

Dan McAran and Sharm Manwani

5. Discussion Based on a review of the codes there is significant similarities in total numbers and the distribution of the codes between MISQ and ISR. Total coding for the three combined classes of “Intuition as Authority” (Postulate, Results and Undifferentiated) represented 37.1 per cent of all codes assigned for ISR and 37.6 per cent for MISQ. “Intuition as Outcome” represented approximately 7 per cent of the coding for both publications under study.

There were two prominent differences: in ISR there were 62 codes (30.2 per cent) assigned as “Intuition in Math” as compared to 14 (7.1 per cent) for MISQ. Similarly in MISQ, 33 codes (16.7 per cent) of all codes, were assigned to “Intuition as Strategy”; in ISR there was only one such code assigned (0.5 per cent).

The number of “Math” codes assigned to ISR and the corresponding number of “Strategy” codes assigned to MISQ can be traced to the orientation of the articles found in these publications in the period under review.

ISR had a number of articles which had a strong mathematical focus on pricing. In 2005, there were 22 “Math” codes assigned to 7 articles with a computational focus. More than a total of 5 “Math” codes were assigned to ISR articles in each of 2003, 2004, 2006 and 2007. In one case the reference to “counter‐intuitive” was made in reference to a mathematical formulation, this was coded not as “Counter‐Intuitive” but as “Intuition in Math”.

In the review of “Intuition in Math” codes assigned, there is a sense that the author of the article is seeking to obtain a hermeneutic like understanding of the mathematical formulation or result. An example of this type of coding would be: Several studies have shown that the nonparametric method is quite robust, provided that the empirical rules are intuitively reasonable (Smith 1989, Tawn 1988). In our analysis we will follow the nonparametric (Jingguo, Chaudhury and Rao 2008 p. 113).

Similarly MISQ had articles which attracted a significant number of “Strategy” codes. In 2000 (Issue 2), two articles on Real Options together attracted 6 such codes. A similar article in 2006 (Issue 4), was assigned 9 “Strategy” codes. Again, a 2008 (Issue 2) article on managing offshore outsourcing had 4 such codes. The MISQ articles that attracted “Strategy” codes had a management and decision focus.

In contrast there was only one coding for ISR in regards to “Strategy” that referred to “simple stationary strategies of intuitive appeal” (Dellarcros 2005 p. 217).

Overall the significant coding results for “Intuition in Math” and “Intuition in Strategy” appear to be related to the difference in content of MISQ and ISR, with ISR focusing more on pricing and econometric issues and MISQ having more focus on real options, management and decision related issues.

Other patterns emerged in the review of the coding. The coding “Intuition as Strategy”, “Management Intuition”, and “Intuitive Decisions” appear closely related. The coding “Intuitive Decisions” was used for one article where intuition was used as part of a specific decision making process; “Management Intuition” codes were only assigned in two articles. Both “Management Intuition” and “Intuitive Decisions” can be interpreted as contained in “Intuition as Strategy”, and seen as specific applications of intuition to strategy by management and in decision making. Similarly the coding “Myers Briggs Intuition” is likely included in the coding “Cognitive Intuition” as the Myers Briggs instrument measures, on a comparative basis, the degree an individual uses cognitive intuition. The coding related to “Myers Briggs Intuition” occurred in two articles with a focus on the Myers Briggs instrument.

The coding “Counter‐Intuitive”, which indicates “counter‐intuitive” was used in the text, may also indicate an attempt at a hermeneutic‐like understanding. Alternatively, the use of “counter‐intuitive” may simply be a reflection of the style of the author. Text was also coded as “counter‐intuitive” where it was clear that the meaning of was “not‐intuitive”.

There may, however, be other interpretations. One interpretation is that the “counter‐intuitive” coding refers to “intuition” that was sought but not found or validated. In the case below perhaps the best interpretation of the meaning of the term “counter‐intuitive” would be to interpret is being used a synonym for “unexpected”:

277

Dan McAran and Sharm Manwani In my own research work on conceptual modeling, the theories I have used have sometimes generated counterintuitive predictions about the utility of certain conceptual modeling practices that are used widely (Weber 2003 p. viii) In future studies it would be useful explore these issues and to differentiate in coding between the “counter‐ intuitive” in postulates and in results in the much the same way as this has been done for “Intuition as Authority”.

Of particular interest were extracts of articles coded “Intuition as Outcome”, representing 7.0 per cent of all coding, and characterized as adding to the intuitive understanding of a research topic, event or phenomena; or where the text refers to an “intuition” as being created or having resulted.

In the following, the work experience of doctoral students is credited with enabling them to develop intuition: “Moreover, their experience in these CoP (Communities of Practice) helps them develop effective intuition and implicit learning processes” (Klein, Rowe 2008 p. 677).

In a second example the paper describes the outcome of the eBay’s feedback mechanism as providing intuitive rules.

As an application of the general results, the paper offers a simplified model of eBay’s feedback mechanism… it allows even inexperienced traders to perform well by relying on simple, intuitive behavior rules. (Dellarocas 2005 p. 224) Of some interest in the use of term “intuition” and related terms in editorials; of the total of 71 editorials appearing in the ISR and MISQ issues reviewed only 2, in MISQ, used “intuition” or related terms.

Review of the text corresponding to the “Intuitive Technology” coding provides insights useful in the creation of a reflective and/or formative constructs for intuition in information systems research. A selection of the relevant text appears in Table 6 below: Table 6: Selected texts coded as “intuitive technology” (Burke, Chidambaram 1999) (Agarwal, Karahanna 2000)

(Chen, Hitt 2002)

Richer media are generally characterized by intuitive interfaces for instance, for most people, talking would be considered more natural and easier than typing. p. 562 Moreover, the technology should provide users with a sense of being in command of the interaction through judicious use of help menus and hot keys that support easy and intuitive navigation (Shneiderman 1987). p. 688

Ease of Use. The Web site of a top firm in this category boasts a consistent and intuitive layout with tightly integrated content and functionality, useful demos, and extensive online help. p. 272

6. Limitations, future research, management implications 6.1 Limitations This research only reviewed ten years of articles in MISQ and ISR; there could be wide variation in the use of the terms related to intuition across time segments. As an example, the use of sophisticated mathematical techniques has generally increased, consequently it can be inferred that references the “intuition” in relation to mathematical formulations has likely increased over time.

On one occasion, in an issue of MISQ, it was noted that the word “intuition” appeared in an abstract and was not detected by the search feature included in the PDF technology. Use of a different online source of MISQ articles detected incidents of “intuition” in this article. This problem may be related to quality of the particular PDF file in question. In addition, a number of the PDF files available were of poor quality which may have affected the ability to detect references to “intuition” and related terms. Further reliability testing in warranted in regards to this new research procedure, although it is believed that the PDF search method used is suitable for this exploratory research.

278

Dan McAran and Sharm Manwani

6.2 Further research It is hoped to extend this research to additional articles of MISQ and ISR. It is likely that saturation will be reached during this process, allowing the development of theory concerning the utilization of intuition in the IS research process. In addition the procedure introduced in this paper could be used to scan the literature of other research areas.

The analysis of the coding for “Intuition as Outcome”, “Intuition in Math”, “Intuition in Strategy” and “Counter‐ Intuitive” could be investigated further.

Ultimately this research could lead to insight which could be utilized in the development of a construct to measure the intuitive nature of IS technology. Research directions include the creation of reflective and/or formative constructs for intuition in information systems research. In addition, the research results on the use of intuition in IS research could be contrasted to the measurement items of such an intuition construct.

6.3 Management implications The main implication from this research is made through analogy. Intuition has been demonstrated as having significant use in academic IS research. The current environment of IS practice is one of constant and, in many cases, overwhelming change, similar to the environment faced by IS academics. As such, if intuition is a useful tool in the academic research process, it is likely to be of value to the practitioner. Consequently, understanding the use and limitations of intuition in academic research can lead to recommendations for the best practice in using intuition in IS management.

7. Conclusion In the research only stages one and two of Grounded Theory has been addressed. Only characterizations can be made at this juncture concerning the use of “intuition” and related terms in the academic IS literature. The road ahead for this research lies in an extension to additional issues of MISQ and ISR. After this process is complete, if the saturation stage of coding is reached, a theory of the use of intuition in academic IS research may emerge. The results of this research could potentially be used to validate measurement items created in the process of development of a Perceived Intuitiveness construct for IS technology.

References Agarwal, R. and Karahanna, E. (2000) “Time Flies When You’re Having Fun: Cognitive Absorption and Beliefs About Information Technology Usage”, MIS Quarterly, Vol 24, No. 4, pp. 665‐694.

Blackler, A. and Hurtienne, J. (2007) “Towards a unified view of intuitive interaction: Definitions, models and tools across the world”, MMI‐Interaktiv, Vol 13, pp. 37‐55.

Bullinger, H.J., Ziegler, J. and Bauer, W. (2002) “Intuitive Human‐Computer Interaction‐ Toward a User‐Friendly Information Society”, International Journal of Human Computer Interaction, Vol 14, No. 1, pp. 1‐23.

Burke, K. and Chidambaram, L. (1999) “How much bandwidth is enough? A longitudinal examination of media characteristics and group outcomes”, MIS Quarterly, Vol 23, No. 4, pp. 557‐580.

Chen, P. and Hitt, L.M. (2002) “Measuring Switching Costs and Their Determinants in Internet Enabled Businesses: A Study of the Online Brokerage Industry”, Information Systems Research, Vol 13, No. 3, pp. 255‐276.

Dellarcros, C. (2005) “Reputation Mechanism Design on Online Trading Environments with Pure Moral Hazard”, Information Systems Research, Vol 16, No. 21, pp. 209‐230.

Glaser, B.G. and Straus, A.L. (2009) The Discovery of Grounded Theory: strategies for qualitative research, (Fourth Paperback Printing), AldineTransaction, London.

Goulding, C. (2002) Grounded Theory: A Practical Guide for Management, Business and Market Researchers, London: Sage.

Gummerson. E. (2000) Qualitative Methods in Management Research (Second Edition), Sage, Thousand Oaks.

Jingguo, W., Chaudhury, A. and Rao, H.R. (2008) “A Value‐at‐Risk Approach to Information Security Investment”, Information Systems Research, 19(1), pp. 106‐120.

Kahneman, D. (2002) “Maps of Bounded Rationality: A Perspective on Intuitive Judgement and Choice, Prize Lecture”, [Online], IDEAS website. Available from: http://ideas.repec.org/p/ris/nobelp/2002_004.html.

Klein H.K. and Rowe, F. (2008) “Marshalling the Professional Experience of Doctoral Students: A Contribution to the Practical Relevance Debate”, MIS Quarterly, Vol 32, No. 4, pp. 675‐686.

McAuley, J. (2004) Hermeneutic Understanding, in: Cassell, C. and Symon, G. (Eds.) Essential Guide to Qualitative Methods in Organizational Research, London, Sage.

Miles, M.B. and Hubermam, M.A. (1994) Qualitative Data Analysis, Sage, Thousand Oaks.

O’Brien, M.A., Rogers, W.A. and Fisk, A.D. (2010) “Developing an Organizational Model for Intuitive Design. Technical Report HFA‐TR‐100” [Online], Georgia Institute of Technology School of Psychology – Human Factors and Aging

279

Dan McAran and Sharm Manwani Laboratory Atlanta, GA, http://smartech.gatech.edu/bitstream/handle/1853/40563/HFA‐TR‐1001 IntuitiveDesignConceptualOverview.pdf?sequence=1 Shaw, N. (2011) “Exploring systems usage at the feature level by reconceptualizing the dependent variable as a formative construct”, DBA thesis, Henley Business School, Henley‐on‐Thames, United Kingdom.

Shaw, N. and Manwani, S. (2011) Extending feature usage: a study of the post‐adoption of electronic medical records, Paper presented at the 19th European Conference on Information Systems, Helsinki.

Shirley, D.A. and Langan‐Fox, J. (1996) Intuition: A Review of the Literature. Psychological Reports, Vol 79, pp. 563‐584.

Simon, H.A. (1960) The New Science of Management Decision, Harper & Row, New York. th Simon, H.A. (1997) Administrative Behavior, 4 Edition, Macmillan: Free Press, New York.

Simon, H.A. (1996) The Sciences of the Artificial, Third Edition, The MIT Press, Cambridge, Mass.

Weber, R. (2003) “Editor’s Comments”, MIS Quarterly, Vol 27, No. 4, pp. v‐xiv.

280

Sharing the Research Journey – Fostering a Love of Research in Coursework Masters Students Grace McCarthy Sydney Business School, University of Wollongong, New South Wales, Australia gracemc@uow.edu.au

Abstract: Based on addressing the motivational needs identified in Self‐Determination Theory (Ryan & Deci, 2000), the paper outlines an engaging approach to developing research skills in a coursework masters program. Self‐Determination theory is a needs‐based theory of motivation, focusing on basic human needs to relate to others in a group, to strive for competence, and to enjoy autonomy. According to Deci and Ryan (2002), social environments which fulfil these needs will result in motivated, engaged and successful individuals. Our students enjoy autonomy in their choice of topic and develop competence in research skills. Although each student conducts an individual project, they support each other throughout the program. The paper reports the findings of a survey of the participants in the 2012 cohort, identifying whether students perceived a difference before and after the research subject in their competence and confidence in conducting research. It reports the actions students considered had contributed to any differences, whether they were actions by the students themselves, by their fellow students or members of staff. The paper also discusses a staged approach to assessment in this research subject, and shares an example of a rubric for critical analysis. The paper concludes that creating a positive environment with support from staff and fellow students enhances motivation for current and future research. Limitations of the study include the small sample size. The paper offers some suggestions for further research, including replicating the study with larger samples.

Keywords: research skills development, self determination theory, motivation, self‐efficacy, staged assessments,rubrics

1. Introduction Students undertaking a coursework masters are understandably nervous about undertaking research projects.

However there is a growing demand for students to have the skills to undertake an independent research project. For example, in Australia, the revised Australian Qualifications Framework (AQF, 2013) requires coursework masters students to be able ‘to plan and execute a substantial research‐based project, capstone experience and /or piece of scholarship’. Even if students take a capstone subject rather than a research project, all coursework masters students are expected to have ‘knowledge of research principles and methods applicable to a filed of work and/or learning’. If done well, research projects represent a way to encourage the quantity and quality of practitioner research, which is important in all management disciplines, not only the emerging discipline of coaching. In coaching however, it is particularly important as there have been many criticisms of early coaching research which was often conducted by commercial organisations keen to sell their services (Grant, Passmore, Cavanagh, & Parker, 2010). Practitioner research often addresses problems of which academics are unaware. Practitioners also may obtain different responses to questions than academics, because of their familiarity with their organisation or industry. They do of course have to take care with ethical issues, particularly if they are conducting research within their own organisations, such as ensuring that participants are genuinely free to participate or not (McCarthy, 2011).

The students on this course are mature professionals, average age 40 – 45. They include full‐time internal and external coaches, coaching managers, human resource managers, learning and development practitioners, and a range of other professions. The course takes place in two‐day workshops. In their first year, the students develop their understanding of coaching theory and their coaching skills, and in parallel develop their academic skills in finding relevant reliable sources for information, how to use information ethically and appropriately, and how to write in an appropriate style. They complete the research project in their second year. At their age, they often have carer responsibilities for children or elderly relatives, demanding jobs, and other social commitments. Juggling these competing demands with postgraduate study is demanding.

This paper offers an alternative to the traditional individual master‐apprentice model to which many research students are still exposed. Based on addressing the motivational needs identified in Self‐Determination Theory (Ryan & Deci, 2000) of autonomy, competence and relatedness, the paper outlines a way to make the process of learning how to do research more engaging than sitting listening to lectures about research methodology.

Our students enjoy autonomy in their choice of research topic, they develop competence in research skills, and although each student conducts an individual project, they support each other throughout the program. This

281

Grace McCarthy begins on the first day when students share their topics with each other. Each student is asked to say something positive about each topic they hear, and then to share any questions or suggestions. Students are encouraged to share resources which may help other students and to help each other refine research and survey or interview questions. Workshops on topics such as literature reviews and research methods help students develop their self‐efficacy (Bandura, 1977), i.e. their belief that they can complete the task successfully. Without such a belief, students may consider the task too hard and not put in great effort. Staged assessments and non‐assessed presentations ensure students receive formative feedback throughout their projects.

2. Literature review The underlying theory for the approach outlined in this paper is Self‐Determination Theory, a needs‐based theory of motivation, focusing on basic human needs to relate to others in a group, to strive for competence, and to enjoy autonomy. According to Deci and Ryan (2002), social environments which fulfil these needs will result in motivated, engaged and successful individuals. Deci and Ryan (2000:234) argue that autonomy and competence are essential for intrinsic motivation which they define as: “Intrinsic motivation involves people freely engaging in activities that they find interesting, that provide novelty and optimal challenge”.

The third element of Self‐Determination Theory, relatedness, is also important for intrinsic motivation, according to Deci and Ryan (2000) although less so than autonomy and competence. It is important for the level of challenge or difficulty of each task to be at the right level for each individual to avoid anxiety and disengagement if too difficult or boredom and alienation if too easy (Csikszentmihalyi, 1990).

Gagne and Deci (2005 ) reported that support for autonomy in a work environment includes choice and meaningful positive feedback as well as the interpersonal context such as managers’ styles and organisational climate. Their meta analysis of studies in organisations found that promoting autonomy led to intrinsic motivation, and a range of positive outcomes including improved performance, job satisfaction, positive work attitudes, organizational commitment, and psychological well‐being.

Applications of Self‐Determination Theory in educational theory have found that autonomy has a key role to play in student motivation (Reeve, 2002). Reeve’s study of teachers in the classroom found that students showed higher levels of autonomy and perceived competence when teachers listened to students, allowed more time for individual work, avoided directives, responded to student‐generated questions, and resisted giving answers. Relevance has long been identified as a key factor in motivation, e.g. (Killen, 2007; Knowles, Holton III, & Swanson, 2005). Schreiner, Hulme, Hetzel and Lopez (2009) note that students who are genuinely motivated, are more likely to engage in learning, and that their engagement results not only in better performance in exams but also lead to personal growth and development.

Schreiner, Hulme, Hetzel and Lopez (2009) also stress the importance of timely, frequent and constructive feedback and this is consistent with the education literature more broadly. Encouraging students to self‐asses as well as to get feedback from peers and lecturers encourage students to identify their own benchmarks of good practice, to relate theory to practice and to develop action plans for improving their knowledge and skills. This promotes their capacity for lifelong learning (Boud, 2007).

3. Research method The research method was an online survey completed after the subject results were declared and time for appeals passed. The lecturer for the subject was not teaching these students in a later subject and so they had no reason to modify their answers to please or impress. Ethics approval was granted by the University’s Human Research Ethics Committee. An online survey was regarded as appropriate as all the students were computer literate with easy access to the Internet. It is a convenient way for students to give their responses in their own time. Unlike interviews or focus groups, online surveys ensure the researcher is unaware of the respondents’ identities as the researcher does not know who has submitted which responses. Online surveys offer a similar advantage over hand‐written surveys as there is no risk of recognising a participant’s handwriting.

282

Grace McCarthy There were ten students in 2012 cohort and all ten completed the survey, although not all questions were answered. The survey comprised a mix of closed question some with an option for a free text response and free text questions. The free text responses provided some rich insights, enhancing the answers to the numeric questions, thus overcoming one of the potential limitations of a survey, of providing only superficial ‘tick‐box’ answers.

4. Findings As can be seen from Table 1, over half the students did not feel confident about conducting a research project before they began the subject.

Table 1: Student confidence before starting subject Before starting TBS 960, how confident did you feel about conducting a research project?

Response % Number of participants Not at all confident 11.1% 1 Not very confident 44.4% 4 Confident 11.1% 1 Quite confident 22.2% 2 Very confident 11.1% 1 Skipped question 11.1% 1 Total 100% 10

Table 2 shows the improvement in confidence in their ability to conduct a research project after completing the subject, with all students stating they felt ‘confident’, ‘quite confident’ or ‘very confident’.

Table 2: Student confidence after completing subject After completing TBS 960, how confident do you feel about conducting a research project?

Response % Number of participants Not at all confident 0% 0 Not very confident 0% 0 Confident 11.1% 1 Quite confident 66.7% 6 Very confident 22.2% 2 Skipped question 11.1% 1 Total 100% 10

Table 3 shows the students’ perception of their competence in conducting a research project before they began the subject.

Table 3: Student competence before starting subject Before starting TBS 960, how competent did you feel about conducting a research project?

Response % Number of participants Not at all competent 11.1% 1 Not very competent 44.4% 4 Competent 0% 0 Quite competent 33.3% 3 Very competent 11.1% 1 Skipped question 11.1% 1 Total 100% 10

Table 4 shows the improvement in students’ perception of their competence to conduct a research project after completing the subject, with all students stating they felt ‘competent’, ‘quite competent’ or ‘very competent’. The degree of improvement was slightly less than with confidence, with more choosing ‘competent’ and fewer choosing ‘quite competent’ or ‘very competent’.

If there was a difference between their before and after responses, the students were asked about the actions the students considered had contributed to these differences, whether they were actions by the students themselves, by their fellow students or members of staff. Table 5 shows the highest ranked actions (4 respondents and above) which the students took themselves.

283

Grace McCarthy Table 4: Student competence after completing subject After completing TBS 960, how competent do you feel about conducting a research project?

Response % Number of participants Not at all competent 0% 0 Not very competent 0% 0 Competent 33.3% 3 Quite competent 55.6% 5 Very competent 11.1% 1 Skipped question 11.1% 1 Total 100% 10

Table 5: Students’ own actions If there was a change between your before and after answers, to what extent (if any) did your own actions contribute? Please tick all that apply, using a scale of 1‐5 where 1 is 'did not help at all' and 5 is 'helped to a great extent'.

Response Number of participants rating 5 ‘helped to a great extent’ Choosing a topic that mattered to me personally 8 Choosing a topic that mattered to me professionally 6 Being able to choose my own research topic 5 Learning by doing the research topic 5 Learning from feedback on each assignment 5 The effort I put in to carrying out my research 4

Additional free text responses to this question included “Selecting the research topic was an essential element that added value for myself and was a far greater motivator than workplace scenarios, group work and presentations that are provided as exemplars of industry practice. This was real.”

“The research project provided additional motivation to learn from the project rather than completion of another subject of study.”

“Personal commitment to give this task my best effort.”

Students were also asked about actions by their fellow students which they believed had contributed to these differences. Highest ranked responses (4 respondents and above) are shown in Table 6.

Table 6: Actions of fellow students If there was a change between your before and after answers, to what extent (if any) did your fellow students contribute? Please tick all that apply, using a scale of 1‐5 where 1 is 'did not help at all' and 5 is 'helped to a great extent'.

Response Number of participants rating 5 ‘helped to a great extent’ Students helping each other with motivation 6 Support in refining my survey/interview questions 6 Support in refining my research question 4 Support in piloting my survey/interview 4 Positive response of other students to my topic and presentations 4

Free text responses to this question highlighted the support students gave and gained from each other: “This group has an extremely high degree of respect for each other, valuable industry experience and willingness to help each other.”

“The facilitation of presentations and group work in developing and evaluating the research project greatly contributed to allowing everyone to provide their perspective and ideas.”

“Our group has been a constant source of inspiration and support for me.” ”I have sought to make a strong contribution to the program, our group and my own learning and have found a great level of support from my colleagues.”

Students were next asked about actions by staff which they believed had contributed to these differences.

Highest ranked responses (5 respondents and above) are shown in Table 7.

284

Grace McCarthy Table 7: Actions of staff If there was a change between your before and after answers, to what extent (if any) did your fellow students contribute? Please tick all that apply, using a scale of 1‐5 where 1 is 'did not help at all' and 5 is 'helped to a great extent'.

Response Number of participants rating 5 ‘helped to a great extent’ Lecturer's belief in my ability to succeed 8 Lecturer's belief in the value of my topic 8 Positive response of lecturer to my topic and presentations 8 Support of lecturer in refining my survey/interview questions 8 Support of lecturer in refining my research question 7 Support in obtaining ethics approval 7 Constructive feedback on drafts or queries 7 Constructive feedback on Assignments 1 and 2 in time to incorporate in 7 Assignment 3 Constructive feedback on Assignment 3 Research Report 7 Constructive questions / comments on my presentations 6 Clear grading guidelines 6 Clear marking criteria 6 Explanations of research methods in class 5 Offers of support in publishing findings 5

Free text responses to this question expressed their appreciation of the support from staff: “I have commented previously that the support of my lecturer and suggestions throughout this process have been invaluable.” ”Strong support from lecturers including suggestions for improvement, articles of relevance, and willingness to go the extra mile to help through a challenging session.”

“I have made good use of the SBS library through this research project and had excellent support from the librarian.”

Finally students were asked to identify their key learnings from the subject in free text responses. Only one student referred to their topic. All others referred to what they had learned about the process of doing research, e.g.

“Keep the topic very narrow and try not to get sidetracked by tangential interesting information.”

“Even though I thought my topic was precise, the results provide unexpected information that takes you in other directions ‐ you have to let some things go or take them up as a separate research topic.”

“This was an interesting process where I was challenged by my own assumptions at the commencement of the research.”

“Sorting, collating, analysing and drawing conclusions from the research is challenging and demands consistent time and effort to do well.“ “Share knowledge and skills ‐ each person has something to offer whether using Survey Monkey, producing charts or being clearer about the questions to be used in the survey.”

“An appreciation and respect for evidenced based research” “Ethical issues in research such as perceived power in an employing organisation and potential bias in questions were new learnings”. “There is more research I can do on my chosen topic.”

These comments illustrate that students have reflected on their learning and can identify what they found of value. While not all will become future producers of research, all will be able to assess the quality of the research they use as the basis for evidence‐based practice.

Student surveys run by the university were very positive for this subject, with an overall mean of 5.75/6.0.

Responses to individual survey questions are shown in Table 8.along with the university mean for the same question. University means are usually lowest for Q5 ‘This teacher stimulates me to think about the subject’ and Q8 ‘Because of this teacher I have felt enthusiastic about studying this subject’.

285

Grace McCarthy Table 8: Student survey responses Question Q1. This teacher is well prepared for the subject.

Q2. This teacher presents the subject matter clearly.

Q3 This teacher organises and sequences the subject matter well.

Q4 This teacher presents an appropriate amount of material for the time available.

Q5. This teacher stimulates me to think about the subject.

Q6 This teacher appears to be interested in assisting me to learn.

Q7 This teacher is helpful in response to my questions or problems.

Q8. Because of this teacher I have felt enthusiastic about studying this subject.

Subject Mean TBS 960 June 2012 5.88 5.75 5.62

University Mean Dec. 2011 5.49 5.24 5.30

5.75

.22

5.75 6.00

5.12 5.34

5.88

5.31

5.88

4.98

5. Discussion The findings show that the students taking this subject increased in their perceptions of confidence and competence. The top three responses relating to the student’s own actions related to the choice of topic, which demonstrates autonomy in Deci and Ryan’s Self‐Determination Theory. The top response referring to actions by fellow students was about how other students helped their motivation as well as helping them refine their questions. The free text responses confirm the importance of the support of the group which fits the ‘relatedness’ aspect of Self‐Determination Theory. The lecturer’s support was also vital both in terms of motivation and support. The key learnings indicate that students gained many insights about the research process, in addition to anything they learned about their topic. There was also a strongly positive response to the constructive timely feedback, the marking criteria and rubrics.

As noted above, feedback on assessments was noted as one of the actions by staff which helped students develop their competence and competence. Rather than have all the marks for the final research report, the students submit a research proposal weighted at 30%, a presentation where they present their draft findings weighted at 15%, and the final report is weighted at 55%. In addition, the students receive formative feedback on their initial presentations of their research topics and on their presentation of their interim findings and progress reports. They also receive individual feedback on drafts and on their ethics applications. Formative feedback is provided by peers as well as staff, while summative feedback is provided only by academic staff.

Feedback has led to improvements in areas that are new for these students, for example mean marks for critical analysis increased from 70% in Proposal to 85% in the final report while the mean mark for linking findings with literature increased from 64% in draft findings to 75% in the final report.

The marking criteria and rubrics are shared with the students from the start of the subject and discussed in class to ensure a shared understanding of what is expected. The feedback on each assignment relates to the rubrics and this helps students understand the rubrics better for the next assignment.

Table 9 shows an example of a rubric for critical analysis, for reasons of space only three grades are shown, pass (50‐64%), credit (65‐75%) and high distinction (85‐100%). Similar rubrics are available for research depth, written presentation, and other marking criteria. Although academics may argue about the wording of the rubrics, the important thing is for lecturer and students to have a shared understanding of expectations.

While not the focus of this paper, a 2012 anonymous survey of alumni who had graduated from the Master of Business Coaching found that they valued clear task instructions, meaningful assessment tasks and marking criteria communicated in advance, with a shared understanding of what represents a pass, fail, credit, distinction and high distinction. Graduates regarded it as important for feedback to be constructive, specific and timely. In addition graduates praised feedback they had received as honest, respectful, insightful and highlighting how they could improve in future (unpublished survey). Feedback contributes hugely to the learning process, hence feedback during the session and not only on a final report, helps students understand how to improve.

286

Grace McCarthy Table 9: Rubric for critical analysis High Distinction (85–100%)

Demonstrates deep understanding of topic Carefully and thoroughly evaluates previous research from all relevant perspectives, taking care not to let researcher’s own assumptions or bias affect the review Summarises key themes Identifies gaps in the literature If direct quotes are used, they are used sparingly and to great effect Insightful conclusion is clearly linked to concepts developed in the paper

Critical Analysis Credit (65‐74%) Demonstrates understanding of topic Includes some evaluation of previous research Uses few, short direct quotes Conclusion present but not fully based on arguments made in the paper

Pass (50‐64%) Demonstrates understanding of topic Descriptive summary of previous research Heavy reliance on direct quotes Weak conclusion

6. Conclusion, limitations, and suggestions for further research Developing research skills is about more than technical skills

Creating a positive environment with support from staff and fellow students enhances motivation for current and future research. Two of these ten students are already enrolled in doctoral programs at this university while others plan to apply for next year. Another has chosen not to enrol in a research degree himself but to collaborate with university researchers on projects in his organisation. The students are considering publishing an eBook of their combined papers and two have submitted papers to conferences. They are proud of their accomplishments. They no longer see research as a mysterious activity conducted by academics, but as a community in which they are competent to take part. The support they gave each other during their research projects continued during their later subjects (they have completed two more subjects since their research paper) and no student withdrew from the course after the research subject, suggesting that peer support may form a useful defence against attrition. While this study did not explore student well‐being, Deci and Ryan (2000) found that people who successfully meet their needs for autonomy, competence and relatedness experience positive psychological outcomes, a further benefit of this approach if confirmed by future studies.

The staging of assignments combined with constructive feedback related to marking criteria and rubrics shared in advance helps the students understand what is expected. It also helps academics to articulate what it is they are looking for in advance, rather than wait until they see it. Four of the top ranked answers by students about how staff helped them develop their competence and confidence related to feedback and a further two to marking criteria and rubrics (grading guidelines).

The limitations of the study include the small sample size (there were only ten students in the 2012 cohort).

Replicating the study with larger samples, with students on research degrees, and with other academic staff, would be useful. It would also be useful to conduct before and after surveys, as this study was carried out only after the subject was completed, and people’s memory may be faulty. It would also be interesting to monitor the progress of students who have completed this research subject and who continue to research degrees. As the number of students undertaking research degrees is increasing each year, it would be helpful to understand if this approach can be used to guide clusters of students on related topics, supplementing some individual supervision sessions with group sessions. Finally, it would be useful to investigate the impact of this approach on attrition of research students.

References AQF. 2013. Australian Qualifications Framework, 2nd ed. South Australia: Australian Qualifications Framework Council.

Bandura, A. 1977. Self‐efficacy: towards a unifying theory of behavioral change. Psychological Review, Vol. 84 No. 2, pp 191‐215.

Boud, D. 2007. Reframing assessment as if learning were important. In D. Boud, & N. Falchikov (Eds.), Rethinking assessment in higher education, learning for the long term pp. Routledge, Abingdon.

Csikszentmihalyi, M. 1990. Flow. Harper & Row: New York.

287

Grace McCarthy Deci, E. L., & Ryan, R. M. 2000. The “What” and “Why” of Goal Pursuits: Human Needs and the Self‐Determination of Behavior. Psychological Inquiry, Vol. 11 No. 4, pp 227–268.

Gagne, M., & Deci, E. L. 2005 Self‐determination theory and work motivation. Journal of Organizational Behavior, Vol. 26 No., pp 331‐362.

Grant, A. M., Passmore, J., Cavanagh, M., & Parker, H. 2010. The state of play in coaching today: a comprehensive review of the field. International Review of Industrial and Organizational Psychology, Vol. 25 No., pp 125‐167.

Killen, R. 2007. Effective teaching strategies, lessons from research and practice (4th ed.). Thomson: Melbourne.

Knowles, M. S., Holton III, E. F., & Swanson, R. A. 2005. The adult learner (6th ed.). Elsevier: San Diego, CA. McCarthy, G. 2011. The 'how' to of research. Coachlink, No. Aug‐Sept, pp 4‐5.

Reeve, J. 2002. Self‐Determination Theory applied to educational settings. In E. L. Deci, & R. M. Ryan (Eds.), Handbook of Self‐Determination Research pp 183‐203. University of Rochester Press, Rochester.

Ryan, R. M., & Deci, E. L. 2000. Self‐determination theory and the facilitation of intrinsic motivation, social development and well‐being. American Psychologist, Vol. 55 No. 1, pp 68‐78.

Ryan, R. M., & Deci, E. L. 2002. Overview of Self‐Determination Theory: an organismic dialectical perspective In R. M. Ryan, & E. L. Deci (Eds.), Handbook of Self‐Determination Research pp 3‐33. University of Rochester Press, Rochester, NY. Schreiner, L. A., Hulme, E., Hetzel, R., & Lopez, S. J. 2009. Positive psychology on campus. In C. R. Snyder, & S. J. Lopez (Eds.), Oxford handbook of Positive Psychology, 2nd ed. pp 569‐578. Oxford University Press, Oxford.

288

Narrating an Emerging Cross‐Sector Partnership: From Brand Orange to Taste Orange Cathi McMullen and Ian Braithwaite Charles Sturt University, Bathurst, Australia cmcmullen@csu.edu.au Ibraithwaite@csu.edu.au

Abstract: This paper uses a narrative approach to explore the emergence of a cross‐sector partnership established to engage in collaborative marketing activities to promote a regional Australian city as a food and wine tourist destination, as well as promoting the food and wine of the region to large urban markets. Cross‐sectors partnerships involving diverse stakeholders from public and private sectors experience significant challenges in mobilising these diverse partners to act collaboratively to achieve collective outcomes. One way to examine cross‐sector partnerships is from a communication perspective where organizations are seen as being constituted primarily through communication patterns (Koschmann, Kuhn & Pfarrer 2012). The narrative approach that we take in this study helps us to understand the dynamic process at play in the emergence of a cross‐sector partnership and explore the role of communication in its constitution. The approach taken locates our study in an emerging body of work that conceptualises organisation as a social process and generatively studies organisations in ways other than the dominant rational scientific traditions, traditions that abstract and generalise and lose touch with practice in the process. Our particular interest is the insights that can be gained from applying this communicative perspective to an emerging cross‐sector partnership. Specifically, we look at three newsletters written across the three‐year initial funding period for a collaborative marketing project for a regional area and the food and wine from this region. By conceptualising these newsletters as a narrative that gives direction, coherence and clarity during the period of emergence, we can explore the role of communication as this marketing alliance emerges into a successful partnership. Use of narrative inquiry enables us to (1) work with the temporal aspects of an emerging organisation; (2) provide an alternative perspective on practices for mobilising a variety of stakeholders undertaking collaborative activity; (3) illustrate the contribution narrative inquiry can make to the study of marketing implementation in collaborative marketing activities and; (4) more broadly contribute to strategies to achieve cross‐sector partnership outcomes.

Keywords: narrative, place branding, cross‐sector partnerships, collaboration, marketing implementation

1. Introduction Our work on this research study has its origins in practice. We first came across the story of Brand Orange in 2008 when undertaking research on local farmers’ markets. Orange is a regional Australian city with a population of around 40,000, located three hours west of Sydney. Looking at the Orange Farmer’s Market website we found links to a wealth of documents on a local collaborative food and wine marketing partnership with the provisional title, Brand Orange. Cross‐sector partnerships involve diverse stakeholders from public and private sectors and experience significant challenges in mobilising these diverse partners to act collaboratively to achieve collective outcomes. The stated aim of this place branding project was to continue building an integrated wine/food tourism industry by increasing awareness of the ‘Orange’ brand, increasing visitor numbers and average visitor spending. There was a focus on achieving greater recognition and opportunities for local wines, delivering more of these wines into urban markets and consolidating Orange’s reputation as a place of great wine and food.

Launched in January 2006, the Brand Orange project was funded for a three‐year period by state and local government and the Orange Region Vignerons Association. The New South Wales government envisaged Brand Orange as a pilot project that could potentially be replicated in other regions. An Executive Officer was appointed and in consultation with stakeholders, work began on establishing an umbrella brand for the marketing, promotion and development of the Orange region, based primarily on wine and food tourism.

What grabbed our attention were the sheer volume, diversity and richness of documents accessible from the Brand Orange website and local government websites. These documents include Brand Orange newsletters to stakeholders, local government documents, presentations outlining the branding approach, prospectus documents for members, media releases, news and lifestyle articles. Most interesting at first glance were three newsletters (ranging from 12‐16 pages each) written by the Executive Officer at various points across the initial three years of the project when an emerging organisation was coming in to being. We read these three newsletters as a narrative producing coherence and continuity to this developing partnership.

289

Cathi McMullen and Ian Braithwaite Reading the newsletters we felt that we were stepping back in time and experiencing part of how this not‐for‐ profit organisation had emerged. Brand Orange – which later became Taste Orange – was lauded as a success story that the state government felt worthy of replication in other regional areas. At the end of the first two years of the project recognition of the district’s reputation as a wine tourism destination moved from 42nd to the top 8 in Australia (Taste Orange 2008). This was a significant outcome given that Orange is the newest wine district in Australia with first plantings in 1980. According to the Executive Officer of Brand Orange monthly visits to the region showed a 500% increase across the 3rd and subsequent year of the project, that is, December 2007 – December 2009 (Currie 2009). This was during period when domestic tourism was down nationally.

Brand Orange successfully brought together a range of diverse stakeholders, created an identity beyond the individual member organisations, crossed the public‐private sector divide and at the end of the three year funding period had conducted a program of successful collaborative marketing activities. In addition, it has extended its scope beyond food and wine to include accommodation and a variety of other local businesses.

These collaborative marketing activities, both within and outside the region, were conducted under a regional umbrella brand ‐ Taste Orange. Our focus in this paper is on the establishment phase of the branding project and our analysis will be concentrated on three newsletters published over the first two years of Brand Orange project. The branding aspects of this project from a consumer perspective are outside the scope of this particular paper.

What is relevant about place branding in this particular study is that the process of place branding is usually carried out by a partnership between public and private sector stakeholders (Hankinson 2010). A place has to be marketed through partnerships, both formal and informal (Warnaby et al. 2002). Partnerships implementing place or destination brands require a framework that embraces clearly defined organisational arrangements and good communications (Hankinson 2009).

One way to examine these cross‐sector partnerships is from a communication perspective where organizations are seen as being constituted primarily through communication patterns (Koschmann, Kuhn, Pfarrer 2012). As we became more familiar with this case we observed that this cross‐sector partnership narrative exhibited certain characteristics associated with entrepreneurial narratives (both individual and industry). These included the need to inspire confidence, establish a common ground for communication with a variety of audiences and validate the unique contribution of the organizations (O’Connor 2002). However, they have the additional challenge of creating a collective identity beyond the separate organisational identities of the individual members. They also need to develop collective agency to achieve their goals (Koschmann et al 2012). This collective agency reflects the interdependence of the members and when successful results in outcomes that exceed those that could be achieved by members working independently.

A narrative approach is well suited to “illuminating the social world” (Ospina & Dodge 2005: 151) and its use opens up new possibilities for examining collaborative activity in emergent cross sector partnerships in ways we argue promote a more complex understanding of the dynamic and relational aspects. It provides a number of key advantages. Firstly, it well suited to the complexity and richness of everyday practice. With Brand Orange it provides a way to examine a branding collaboration in a very concrete and detailed way. Second it enables a process focus as process can be studied through the unfolding story. Third, narrative work can focus on construction of identity – individual or collective.

Our research focus is on the key practices involved in mobilising diverse stakeholders in the development of an emerging cross‐sector partnership and how can these practices be better understood through the use of narrative inquiry. Questions of interest include: How might these newsletters represent/construct the work that was involved in bringing people together and keeping them together through the development of and implementation of collaborative marketing activities? What work are these newsletters doing in providing continuity and coherence whilst mobilising a diverse group of stakeholders in the development of a brand?

How does this change across the life of the project?

This paper proceeds in a series of stages. First, we locate our study providing a background on narrative inquiry and examining specifically the role played by narratives in emerging organisations. This is followed by a narrative analysis of an exemplar case study of a cross‐sector collaborative marketing initiative. Finally we discuss the contribution of a narrative approach to study emergent organisations in not‐for‐profit sectors,

290

Cathi McMullen and Ian Braithwaite marketing implementation in collaborative marketing activities and more broadly contribute to strategies to achieve cross‐sector partnership outcomes.

2. The role of narratives in emerging organisations Narrative research is interdisciplinary in its origins, including elements of literary, historical, anthropological, sociological, psychological and cultural studies (Casey 1994). Renewed interest in narrative from the 1980s can be seen in a range of fields including psychology (Bruner 1986; Mishler 1992; Gergen 1992; Polkinghorne 1995), education (Connelly & Clandinin 1999; Casey 1994) and sociology (Denzin 1997; Gubrium & Holstein 1998). Narrative approaches have additionally made a considerable contribution to management and organisation theory. Rhodes and Brown (2005) in a comprehensive review of narrative approaches in management and organization theory, identify five principal areas where a narrative inquiry has been directed: (1) sense–making; (2) communication; (3) learning/change; (4) politics and power; and (5) identity and identification.

Narrative inquiry is based in the premise that, as human beings, we come to understand and give meaning to our lives through story. Narratives compose and order life experiences. By being structured and recounted through story form, experiences are accounted for and given meaning and significance (Usher 1997). The narrative turn has opened up new pathways for research that focus on interpreting social events and understanding the intentions and meanings of social actors, rather than just explaining and predicting their behavior (Dodge, Ospina & Foldy 2005). In narrating, a narrator communicates and captures nuances of event, relationship, and purpose that are dropped in the abstraction process that permits categorisation and correlation in the logico‐scientific model where the focus is on propositions or rules that connect categories of behaviour to categories of actors and situations. In contrast, narrative thinking places these elements into a sequenced, contextualized statement with a plot (Tsoukas & Hatch 2001).

To demonstrate the potential value of a narrative approach in emergent organisations we discuss two studies that promote the use of a narrative lens to study organisational innovation and acquisition of capital from external sources. In a recent study on the role of narratives in sustaining organizational innovation Bartel and Garud (2009) propose that while designs and processes are necessary for innovation to occur they may not be sufficient. They argue that innovation narratives are key cultural mechanisms that facilitate the productive social interactions needed in the implementation of organizational designs and processes. One of the key aspects in this coordination through narrative is the translation of ideas across the organisation so that they are comprehensible and appear legitimate to others. They also highlight the need for both coherence and flexibility in narratives. They point out that narratives are especially instrumental in socialising newcomers and creating a common ground of social action within organizations. Also that narrative allows for flexibility in social interactions by presenting information, ideas or practices in a manner that is evocative.

In a similar vein, in a study of the role of entrepreneurial narratives, Martens, Jennings & Jennings (2007) argue for the effects of story telling on a firm’s ability to secure finance from external sources. Obtaining adequate capital is vital for survival and growth but it is widely recognised as a complex and challenging task. A particular challenge for entrepreneurial firms is that in their early stages of development they lack a track record and are requiring potential funding parties to make a leap of faith with limited information and uncertainty about the value of the firm’s entrepreneurial opportunities and their capacity to exploit them. In this context, a narrative provides: (1) a means to communicate a coherent story about the firm and the opportunities it is developing; (2) construct an identity for the firm and; (3) work towards reducing the ambiguity and complexity around the situation of the organisation.

Martens et al. (2007) identify the three ways in which narratives can work towards enhancing resource acquisition for entrepreneurial organisations. Firstly by communicating identity that is comprehensive and memorable. Second, by communicating current and intended actions, making sense in an ambiguous situation and communicating insights. It is argued that strategic plans presented in a narrative form create a richer picture for the audience, including illustrating the developments in the writer’s insights (Shaw, Brown & Bromiley 1998), contributing to making those actions appear more justified or plausible in the given context (Barry & Elmes 1997). Finally, they argue that interest in the activities of the company and commitment to financially supporting those activities are enhanced by narrative work.

291

Cathi McMullen and Ian Braithwaite “Narrative scholars argue that successful stories don’t just provide inform readers; they generate interest and commitment, thereby motivating audience members to act in a manner consistent with the author’s intended outcomes” Martens et al. (2007:117).

In summary, these studies can inform in a range of ways our discussion of the role of narratives in developing and coordinating collaborative marketing activities in emerging cross sector partnerships. First, through translating and communicating ideas and plans in rich and evocative ways that justify and enhance legitimacy of past actions and future plans. Second, by assisting in socialising newcomers and creating a common ground to facilitate collective action. Third, facilitating the ongoing construction of a coherent conception of identity and intention (Kuhn 2008).

Our focus in this paper on the collaborative marketing activities in a cross sector partnership that help develop a place brand, draw on and is illustrated by findings from a narrative analysis of a case study of an exemplar. In the coming section we demonstrate the use of a narrative approach to help understand and enhance the practice of cross‐sector partnership development more broadly and specifically place branding.

3. Narrating an emerging cross–sector partnership: From Brand Orange to Taste Orange Narrating organizations consists of telling about organizations using a narrative structure, that is, a sequence of events or plot (Czarniawska 1998). Examining this emerging cross sector partnership through a narrative lens includes a focus on: (1) how work is enacted through narrative to retrospectively account for what has been done; (2) how identity work is done; and (3) how complexity and ambiguity are addressed by presenting a cohesive narrative around the project and imagining a future for the project. The process of documentation renders particular aspects of marketing work more visible.

This study can be described as a theoretical reflection based on an illustrative case study. An abductive research strategy has been adopted with theory and research intimately intertwined (Blaikie 2010). As researchers we have spent alternating periods of immersion in the relevant documents and periods of withdrawal for reflection and analysis of data. Categories have been developed through an iterative process of going back and forth between theoretical concepts and the data (Alvesson & Sköldberg 2009). Our research has been shaped by the specific theoretical positions we have adopted.

The particular case selected for examination – Brand Orange was chosen for two key reasons (1) the organisation has been recognized for its success in meeting the objectives of its branding initiative and a model for other regional areas and could thus be considered an exemplar; and (2) a large volume of documents (approximately 50) relating to the branding initiative are publically available online. This provides an extensive data source on the project. This data covers a time period of several years so gives a longitudinal perspective to the analysis. This temporal aspect to the data is critical when the considering the process of the development of the initiative. For this particular case study were have selected a group of 3 documents for intensive analysis – documents that are suited to addressing the research question. A timeline for the Brand Orange project from 2006‐2008 is presented in Figure 1. Dates of each newsletter and milestones for the project are highlighted.

The selection of an exemplar reflects purposeful sampling designed to provide an ‘information rich’ case from which a great deal can be learnt about issues of central importance to the project (Patton 2002). Working with a single case study allows for the development of a ‘rich story’, an aspect often missing when multiple case studies are used (Eisenhardt & Graebner 2007). It is the capacity for illustration rather than any claims to generalisabilty that are the strength of the single case study approach. As textual analysis depends on very detailed data analysis it recommended that within a data set there should be a focus on only a few texts or parts of texts (Silverman 2003). Thus we have limited our analysis to selected extracts of the three newsletters.

The use of documents in a case provides an alternative data source to the more commonly used interviews in case study research. Documents published during the time illustrate the phenomena in its real‐time context (Yin 2003) rather than the retrospective accounts that generally characterize interview data where accounts can be compromised by memory loss and rationalisation (Dahlin, Fors & Öberg 2006). In analysing the three newsletters we are not looking at narrative as a transparent representation of what went on and why. Rather, our attention is directed to the way in which a narrative strategy has been used (consciously or unconsciously) to mobilise a diverse group of members to act in ways to their collective benefit during the emergence of this

292

Cathi McMullen and Ian Braithwaite organisation. No attempt is being made to project findings from this case to a general population. Yet we would argue that the findings of this study are transferable more broadly to cross sector partnerships and place branding initiatives.

Figure 1: Brand Orange timeline 2006 to 2008 Narrative analysis can take many forms reflecting the research question and the researcher’s theoretical orientation. We could examine the plot constructed across the three newsletters, for instance, as a progressive narrative showing advancement towards to a goal (Gergen & Gergen 1997). Or we could look at the discourses that are drawn on in the telling of stories within the newsletters. However given, the limitations of space we will restrict our analysis to the work that is being done in the each of the three newsletters. This approach aligns with the studies discussed earlier. The sections dealing with each newsletter have been named to reflect the aspects of work that appear most critical in that document. Within each of these sections we undertake textual analysis to draw attention to the way language has been used to invoke and mobilise stakeholders. What will be demonstrated is that while the newsletters give an account of the project to the various stakeholders, much more complex work can be observed through careful textual analysis.

3.1 Phase 1: Foundational work The first of the newsletters was written at the end of the first year of the project informing members of progress. The document was a 12 page colour documents made available electronically and in hard copy. Its stated aim was to ensure that members were kept well informed about the progress of the Brand Orange project, and to provide a forum for discussion and feedback. A key theme that emerged in our analysis was establishing legitimacy both for the project and the project leader. Table 1 displays excerpts to illustrate this work and how it had been enacted. The narrator’s intentions have been interpreted by the researchers through close engagement with these texts and relevant theory. As with any interpretive work other researchers and the newsletter’s author may ascribe alternative intentions.

Emerging organisations need legitimacy in a way that is assumed in existing organisations (Golant & Sillence 2007). For emerging organisations the establishment of legitimacy is important to secure resources and other forms of support. The placement of reporting of progress in the very early stages of the first newsletter suggests a choice has been made that this accounting needs to be done before further discussion of the project can be introduced. Lists are provided to enhance the sense of completeness and passive voice used to enhance the factual status of the account by removing the presence of the author in this section. Following the account there is justification and contextualising of the critical role of this groundwork as a basis for activities in the coming years. As well as informing members there is an attempt to involve and include members. This is performed through empathy and reassurance that progress is being made. There is also a very clear

293

Cathi McMullen and Ian Braithwaite articulation of the imperative for the organisation to be self‐funding by the end of the three‐year funding period.

Table 1: Brand Orange phase 1 – Foundational work* Intentions

Establishing legitimacy

Practices

Illustrative excerpts

Accounting

It is a year when a lot has been achieved…

There was a very high level of support expressed for the project progress This foundation building work during the year places us well for the roll out of our major events in Sydney next year.

It’s a hard year to ask anyone to be patient. But we are now in an excellent position to get down to the really exciting part…

2006 involved you ... showing vision, commitment and more than a decent dash of trust mixed with optimism and nothing left to lose. 2007 is where you justifiably get to see results 2008 we need to have been not only successful in delivering to members ‐ we need to be looking down the nose of independence from State funding.

Justifying

Building connection & community

Reassuring and empathising

Recognising contribution to date

Articulating a vision

Advocating an entrepreneurial orientation

* All excerpts sourced from Brand Orange Newsletter No 1.

3.2 Phase 2: Collaborative work and collective agency The second newsletter was distributed at the midway point in the project. Given the limited time frame to achieve results from the three year funding commitment there is pressure for decisions to be made and for the ‘collective inertia” often observed in multi‐partner projects to be avoided. An important decision to be negotiated with members is the venue for a major event in a large metropolitan area. Detailed analysis of this stage highlights the challenges in deciding at what stage to invite member feedback on the specifics of planned activities. The capacity to direct without appearing to dominate and to marshal consent requires skilful narration. One technique used by the narrator is a repeated series of “if” questions to present the key points of her position in a structured and systematic way building the weight of her argument in a cumulative way.

Selected excerpts in Table 2 illustrate this.

Table 2: Brand Orange phase 2 – collaborative work and collective agency* Intentions Achieving collective action

Practices Marshalling consent

Managing conflict

Facilitating purposeful social interaction

Maintaining connection & community

Illustrative excerpts If this is our target do we really want …

If a target audience has to make a conscious effort….

If we showcase wine in true partnership..

The most acceptable membership is likely to be…

The usual format is an informal supper followed by a tightly structured hour of reviewing the project to date and prioritising future planning.

* All excerpts sourced from Brand Orange Newsletter No. 2

At various points conflict is addressed in an indirect way through vagueness. Vagueness, strategically used, can help consensus with strategic ambiguity providing the space for equivalent rather than shared meanings (Weick & Browning 1986). Attention is given to the social interaction but it is combined with purposeful activity.

294

Cathi McMullen and Ian Braithwaite

3.3 Phase 3 – Identity work: Building a heritage and imagining a sustainable future The final newsletter has a celebratory tone acknowledging the success of the endeavour and reflecting on both collective identity and future directions. Table 3 illustrates how identity is partly defined through differentiation from other regions. There is also a very clear trajectory for the organisation.

Table 3: Brand Orange phase 3 – Identity work and imaging the future* Intentions Identity work

Practices Defining and differentiating

Imagining the future

Advocating an entrepreneurial trajectory

Illustrative excerpts We are not like any other wine region ‐ we’re young, energetic, spirited, small, boutique, quality, family, community based with the best food industry of any wine region in Australia.

We have an opportunity in the New Year to prove that we can manage our own destiny, publish credibly, build a solid financial base and expand the services to industry and community in partnership with local government.

* All excerpts sourced from Brand Orange Newsletter No. 3

4. Concluding comments What is evident from the narrative analysis in this study is that collaborative activity in a cross‐sector partnership involves much more than getting stakeholders together and working through steps in a plan. This analysis has presented a picture of one such project in a vivid way highlights the complex relational and communicative work that may be undertaken in successful cross sector partnerships and other multi ‐partner collaborative ventures.

Acknowledgements The authors gratefully acknowledge support in the completion of this project from a Faculty of Business Small Research Grant, Charles Sturt University.

References Alvesson, M, & Sköldberg, K. (2009). Reflexive methodology: New vistas for qualitative research (2nd ed.). London: Sage Publications.

Barry, D, & Elmes, M. (1997). Strategy retold: Toward a narrative view of strategic discourse. Academy of Management Review, 22(2), 429‐452.

Bartel, C.A, & Garud, R. (2009). The role of narratives in sustaining organizational innovation. Organization Science, 20(1), 107‐117.

Blaikie, N. (2010). Designing social research. Cambridge: Polity Press.

Brand Orange. (2006). Brand Orange newsletter: Issue 1. Orange: Brand Orange.

Brand Orange. (2007). Brand Orange newsletter: Issue 2. Orange: Brand Orange.

Brand Orange. (2007). Brand Orange newsletter: Issue 3. Orange: Brand Orange.

Bruner, J. (1986). Actual minds, possible worlds. Cambridge, MA.: Harvard University Press.

Casey, K. (1994). The new narrative research in education. Review of Research in Education, 21, 211‐253.

Connelly, F, & Clandinin, D. (1999). Narrative inquiry. In J. Keeves & G. Lakomski (Eds.), Issues in educational research (pp.

132‐140). Oxford: Pergamon.

Currie, K. (2009). Brand Orange: Integrated Event Marketing Strategy Powerpoint Presentation. Paper presented at the Forum: Invigorating town centres through alternate local fresh food marketing strategies ‐ 4 May, Penrith, NSW. Czarniawska, B. (1998). A narrative approach to organizational studies. Thousand Oaks: Sage Publications.

Dahlin, P, Fors, J, & Öberg, C. (2006, September 7‐9,). Press releases, annual reports and newspaper articles ‐ Using alternative data sources for studies on business network dynamics. Paper presented at the 22nd Annual IMP Conference, Milan.

Denzin, N. (1997). Interpretive ethnography: Ethnographic practices for the 21st century. Thousand Oaks: Sage Publications.

Dodge, J, Ospina, S.M, & Foldy, E.G. (2005). Integrating rigor and relevance in public administration scholarship: The contribution of narrative inquiry. Public Administration Review, 65(3), 286‐300.

Eisenhardt, K.M, & Graebner, M.E. (2007). Theory building from cases: Opportunities and challenges. Academy of Management Journal, 50(1), 25‐32.

Gergen, M.M. (1992). Life stories: Pieces of a dream. In G. Rosenwald & R. Ochberg (Eds.), Storied Lives: The cultural politics of self understanding (pp. 127‐144). New Haven: Yale University Press.

Gergen, K. & Gergen, M. 1997. Narratives of the self. In L. Hinchman & S. Hinchman (eds.), Memory, identity, community: The idea of narrative in the human sciences. New York: State University of New York.

295

Cathi McMullen and Ian Braithwaite Gubrium, J.F., & Holstein, J.A. (1997). The new language of qualitative method. New York: Oxford University Press.

Gubrium, J.F, & Holstein, J.A. (1998). Narrative practice and the coherence of personal stories. The Sociological Quarterly, 39(1), 163‐187.

Hankinson, G. 2009. Managing destination brands: Establishing a theoretical foundation. Journal of Marketing Management, 25, 97‐115.

Hankinson, G. (2010). Place branding research: A cross‐disciplinary agenda and the views of practitioners. Place branding, 6(4), 300‐315.

Koschmann, M.A, Kuhn, T.R, & Pfarrer, M.D. (2012). A communicative framework of value in cross‐sector partnerships.

Academy of Management Review, 37(3), 332‐354.

Kuhn, T. (2008). A communicative theory of the firm: Developing an alternative perspective on intra‐organizational power and stakeholder relationships. Organization Science, 29(8/9), 1227‐1254.

Martens, M.L, Jennings, J.E, & Jennings, P.D. (2007). Do the stories they tell get them the money they need?: The role of entrepreneurial narratives in resource acquisition. Academy of Management Journal, 50(5), 1107‐1132.

Mishler, E.G. (1992). Work, identity and narrative: An artist‐craftsman's story. In G. Rosenwald & R. Ochberg (Eds.), Storied Lives: The cultural politics of self understanding (pp. 21‐40). New Haven: Yale University Press.

O'Connor, E. (2002). Storied business: Typology, intertextuality, and traffic in entrepreneurial narrative. The Journal of Business Communications, 39(1), 36‐54.

Ospina, S.M, & Dodge, J. (2005). It's about time: Catching method up to meaning ‐ The usefulness of narrative inquiry in public administration research. Public Administration Review, 65(2), 143‐157.

Patton, M. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks: Sage Publications.

Polkinghorne, DE. (1995). Narrative configuration in qualitative analysis. In J. Hatch & R. Wisniewski (Eds.), Life history and narrative (pp. 5‐23). London: The Falmer Press.

Rhodes, C, & Brown, A.D. (2005). Narrative, organizations and research. International Journal of Management Reviews, 7(3), 167‐188.

Rosenwald, G, & Ochberg, R. (1992). Storied lives: The cultural politics of self understanding. New Haven: Yale University Press.

Shaw, G., Brown, R. & Bromiley, P. 1998. Strategic stories: How 3M is rewriting business planning. Harvard Business Review, May/June, 41‐50.

Silverman, D. (2003). Analyzing talk and text. In N. K. Denzin & Y. S. Lincoln (Eds.), Collecting and Interpreting Qualitative Materials (pp. 340‐362). Thousand Oaks: Sage Publications.

Taste Orange. (2008). Orange NSW: Official 2008 regional visitors guide. Orange: Orange City Council.

Tsoukas, H, & Hatch, M.J. (2001). Complex thinking, complex practice: The case for a narrative approach to organizational complexity. Human Relations, 54(8), 979‐1013.

Usher, R. (1997). Experiential learning and the shaping of subjectivity in the workplace. Studies in the Education of Adults, 31(2), 155‐163.

Warnaby, G., Bennison, D., Davies, B. & Hughes, H. 2002. Marketing UK towns and cities as shopping destinations. Journal of Marketing Management, 18, 877‐904.

Weick, K. E. & Browning, L. D. 1986. Argument and narration in organizational communication. Journal of Management, 12, 243‐259.

Yin, R.K. (2003). Case studies research: Design and methods. Thousand Oaks: Sage Publications.

296

The Three World Model of Knowledge as a Solution to Failing Research Centres Graham Trevor Myers Durban University of Technology, Durban, South Africa myersg52@gmail.com

Abstract: Three methods are used to determine what the main objective of a research department or centre should be.

Badat emphasises it should be research. Wielicki says it should have knowledge creation, understanding and wisdom as the objectives while Mouton places it squarely in his World 2 of knowledge where scientific relationships are examined, pushing back the boundaries of knowledge, with regular sojourns into the World 3 of knowledge where meta‐analysis forms a meta‐science (above science) knowledge. In this World 3, knowledge is created by drawing all the conclusions together from World 2, and also approaching research unconventionally by making paradigm shifts during periods of change. Research departments or centres show a myriad of characteristics which indicate failure but the one principle that stands at the core is the knowledge creation. When this begins to wane other signs should be examined to see in which of the 5 stages of decline they are in according to Collin’s Model. The difficulty in restoring such a research department or centre is dependent on this analysis. But in every case the only question to be examined is how far they have strayed from their central reason for existing, their ‘flywheel’ – creating knowledge.

Keywords: failing research centres, meta‐science, meta‐analysis, research administration, three world model of knowledge, Collin’s 5 Stage Model of decline, core value of research centres

1. Introduction Tolstoy’s novel entitled Anna Karenina, begins with the words “All happy families are alike; each unhappy family is unhappy in its own way”. So it is with research departments or centres. Those that are effective or successful have common characteristics while those doomed to dysfunction show some signs which are informative as to their wellbeing.

Ralph Waldo Emerson puts this idea nicely, saying, “As to methods there may be a million and then some, but principles are few. The man who grasps principles can successfully select his own methods. The man who tries methods, ignoring principles, is sure to have trouble” (Ferriss 2011:150) It is these principles that this paper will examine, by looking at the work of Collins, Myers and Yearwood, Badat, Wielichi, Baird and Mouton amongst others.

Education is examined by looking at its purpose, its outcomes and how knowledge is structured to determine the driver behind a research department or centre.

2. The purpose of higher education In his address in the Wolpe Dialogue lecture at the University of the Witwatersrand, Badat (2012) examines the purpose of Higher Education in some detail. Quoting the Yale Report of 1828 he concludes that one should not ask of education whether it has had an effect, but what is the purpose of Higher Education? This forms the basis of this paper because one needs to identify the “flywheel” of education to determine what keeps the system going.

The purpose of higher education is not in the teaching or research content, how education takes place, or the admission policies used, but in three distinct spheres (Badat 2012). These are:‐

Research ‐ in which there is a production of knowledge. Knowledge is tested and invigorated, solutions to contemporary problems are sought for immediate and long term benefit, and the ‘right questions’ to guide the investigation is formulated.

Teaching ‐ in which knowledge is disseminated and cognitive knowledge is formed and cultivated through mastery of a field of knowledge, the ability to examine it critically, and to be able to apply it broadly to ethical and moral situations.

Community Engagement ‐ through being responsive to political, economic and social environmental factors. This will then inform the content, how education takes place and who is admitted.

297

Graham Trevor Myers

3. The outcomes of the educational process The three purposes outlined above are not hierarchical. Wielicki (2008) introduces hierarchical outcomes that are expected to occur in the mind of the learner, each at a higher level of complexity and sophistication. These are:‐

Data retention – where symbols and facts which are the starting points of information creation are easily stored, disseminated and memorised. This being the lowest level objective of the learning process;

Information learning – where the product of categorising and organising data so as to answer basic questions is constructed by asking ‘what? when? who? and where?’;

Knowledge creation – which is the use of data and information to solve the question of ‘how to?’ It allows for the development of skills;

Understanding – seldom achieved by average educational institutions. It revolves around the question ‘why’? It measures the relevance of information or skills in problem solving;

Wisdom – the pinnacle of the educational processes, where value judgements are incorporated with respect to actions and decisions based on ethical and cultural values.

4. The three world model of knowledge Mouton’s (1996) 3‐world model of knowledge (Myers and Yearwood 2012) purports that academics operate in multiple worlds of knowledge with each one requiring different kinds of knowledge. Often academics operate in these worlds independently, in which different forms of analysis are used and different types of ‘knowledge’ is transferred in different ways; the relationship between these worlds often not being considered as conjoint.

From the understanding of Mouton’s writing, it would seem that these ‘worlds’ are not hierarchical but rather differ in complexity and focus, even though the diagrams presented by Mouton (1996) indicate a flow of information.

4.1 World 1 – the world of lay knowledge This world of knowledge is considered to be the realm of ‘everyday life’ knowledge. It consists of wise sayings; common sense; savvy; ‘how to do’; or what may be termed pragmatic knowledge enabling citizens to manage their daily operations. Its territory is the ‘social’, ‘physical’ and symbolic worlds in which ‘entities’, individuals, collectives, social practices, organisations and institutions operate (Babbie and Mouton 1998:14). Rich in meaning and ‘personal symbolism’ these ‘pieces’ of knowledge from various sources are uncritically applied in the individual’s ‘real’ world.

This could be similar to the teaching sphere mentioned above as well as the data retention and information learning outcomes above. These tend to find residence in the primary degrees in Universities.

4.2 World 2 – the world of science This is the world of ‘two dimensional science’ knowledge where links are created between constructs. Taking the world 1 knowledge and subjecting it to rigorous and systematic examination within a different paradigm, and using these observations and measurements to espouse linkages between the disparate political, economic, social, physical ….etc. facts in an attempt to define external truth, and extend the boundaries of knowledge.

The world of science is not homogeneous, each discipline or sub‐discipline develops in its own evolutionary way by falsification (Popper 1972). During periods of normal development there is progress within one paradigm in which ‘World 1’ knowledge produces additions to ‘World 2’ knowledge. During periods of scientific ‘revolution’, says Thomas Kuhn (1970), there is a shift to another paradigm. It is during these periods that World 3 knowledge flourishes, implies Mouton.

The knowledge created in this world corresponds with the research output of educational institutions where the outcomes of ‘knowledge creation’ and ‘understanding’ are focused on interlinking existing knowledge.

298

Graham Trevor Myers

4.3 World 3 – the world of meta‐science It is in the sphere of World 3 knowledge that critical reflection occurs, according to Mouton (1998) and where ‘new’ knowledge is created which may not be the result of previous knowledge but is a product of ‘revolutionary’ development. Collectively, the scientific community ‘examines’ the presentations within the scientific world and places these in congruent, comprehensible wholes. It examines the Epistemology (Structure of the knowledge through collating all the disparate scientific links of World 2) and through an Ontological process finds the gaps and tries to fill these. It is here that the ‘scientists’ reflect on what they are doing and the methodology they will use to advance the various ‘truths’ like Scientific Truth, Religious Truth, Moral Truth, Historical Truth etc.” (Myers and Yearwood 2012). This process is referred to as a meta‐analysis in the work by Myers and Yearwood.

It is thus in the world of meta‐science that discussions occur on the advancement of knowledge, the parameters of the paradigm in which it is examined and the meta‐analysis used to construct the greater whole. It is the advancement of knowledge for the sake of pushing back the boundaries to allow the inclusion of future question to be examined in World 2. Often the ‘fringe researchers’ are ignored even though they glimpse possible avenues of research that at present make no sense as they undergo a paradigm shift.

McNiff and Whitehead (2005:56) refer to this as a transformational evolutionary process. One often feels lost in this place which Schon (1995: 28) calls a ‘swampy lowland’ where one tries to make observations from the lowest point about topography that you know must be there. The concepts one grapples with are not easy to communicate as one actually has no cohesive understanding of them as yet (Myers 2012). This is called a ‘cloud of unknowing’ by Johnson (1995:69).

5. The ‘flywheel’ of a research department or centre Outcomes 1 (Data retention) and 2 (Information learning) above would be placed in World 1 (lay knowledge) resulting in qualifications which emphasises ‘knowing existing knowledge and mastering existing skills’, which Badat (2012) calls teaching;

Outcome 3 (Knowledge creation) would be achieved in World 2 (Science) resulting in the achievement of examining new relationships between existing knowledge. This would occur in post graduate qualifications at most universities, while outcomes 4 (Understanding) and 5 (Wisdom) would be places in World 3 (Meta‐ science) and in the research category mentioned by Badat (2012). It is here that ‘truth, reasoning, problem solving, and values’ are intertwined in the development of a ‘grand theory’.

The ‘flywheel’ which keeps a research department of centre ‘going’ is a mixture of World 2 knowledge creation with the aim of constantly sojourning into the World 3 space. As the students learn the skills it should be up to the leaders to be guiding their knowledge creation to assist in providing the missing sections of the grand theory, keeping variables and constructs constant to allow for meta‐analysis (Glass 1976).

Let us examine what tends to be some of the problems in Research Departments and Centres at the present time.

6. Research department specifics In his work on “What Characterises a Productive Research Department?”, Baird (1986) put forward five characteristics that were highly correlated with the average annual number of articles published per faculty member in 3 different departments in a national survey in The USA, published articles being a measure of productivity. These were:‐

Well trained Faculty;

Able Students;

Financial Support;

Resource support; and

The emphasis was on research (rather than financial output and administration).

299

Graham Trevor Myers The responses from his data collection that did not correlate with the average annual number of articles published per faculty member in the 3 different departments were:‐

Collegiality;

Person‐role fit;

Group Morale;

Level of Human and Mutual concern

Sitnikov, (undated) when examining total quality management models in education points out three developments that have constrained education. These are:‐

The use of a team approach in a bureaucratic structure which denies effective leadership and hamstrings the independence of academics;

The difficulty of quantifying the output of academic departments; and

Assessment floors in the evaluation of the teaching and learning process.

7. Supply chain management Another angle which can be used to examine the central core of what a research department or centre does is that of Supply Chain Management (SCM). SCM is often equated with Public Management. Marie de Beer (2012) in conversation about this paper indicated that this is the confusion that exists and damages the concept. Public management deals with administration whereas SCM is based on strategic thinking. SCM is also not monitoring the movement of goods through a process or management of costs. It is far more than that says de Beer. It is the strategic selection of the most appropriate inputs, through the correct process, to supply the most relevant output.

In research departments the output often is ‘money for the unit’, but by paying researchers a bare minimum or making it part of their conditions of service. There is no emphasis on a value chain, which in this SCM analysis is the development of the researcher if the reason for a research department is educational as outlined above. Some research departments just ‘clone’ researchers in method and focus, stifling original thought through laziness and over‐supervision. The researcher is becoming “lost in the quagmire of self‐ importance” and administration (de Beer 2012).

The Educational Supply Chain examined in research centres begs the question whether they can function in the dynamics of customer requirements and are enabled to fulfil orders quickly and precisely.

Wielichi (2008) tries to apply SCM theory to an IT educational world based on the customer requirements of “quick and precise order fulfilment” (2008:65). He indicates that future educational provision should:‐

Have increased flexibility;

Incorporate IT without jeopardising educational values but adding market value;

Overcoming technical and financial barriers;

Controlling the educational value of the offering;

Dealing with increased completion; and

Enhancing global context.

The emphasis in most research centres tends to be on research administration rather than the core business of teaching, research and community engagement. This is due to the emphasis on some of the factors that Collins warns against in his five stage model. A keen awareness of these factors identifies the risk to research in various entities. One attitude, worth listing, appears to be ‘who should control research?’ In Sweden (Butler 2000) a single national council, legislated by government is replacing the “myriad of research councils” that controlled scientific development. Whether this is appropriate or not is left for future discussion. There are some authors who are against stifling research departments with restrictive process and procedures. One of these is Hitt (2000). He emphasises the use of talents of the team members and their strengths and weaknesses to sustain a competitive advantage. The management in the new Millennium must be characterised by Flexibility, Agility, and Human Capital Development, Non‐linear thinking, using IT to harness

300

Graham Trevor Myers technology, multiple skill development, skills portfolios and multiple career paths, knowledge dissemination and application, global markets and the development of core competencies. In his 1997 Presidential address to the academy of Management Hitt picked up four of these broad categories, Globalisation, Quality, Continuous Evaluation and Thinking Strategically. Most importantly for this paper is a comment he makes….”notice that I do not suggest ‘control’, ‘govern’, or ‘oversee’, but rather we should work with our colleagues as partners” (Hitt 1997).

There are numerous articles espousing the control of research (or academic) departments. They range from financial to structural, but all suggesting ‘managerial’ type governance. These are possibly a topic of discussion for the future but are counterproductive to research development.

8. The flywheel of education departments What seems to be the driving force that keeps research departments functioning is the emphasis on research for its own sake and the training of researchers in World 2. This principle would tend to keep things going.

What then are the dangers that must be examined if this is going to be compromised?

9. Colin’s five stages of decline model When referring to companies failing, Collins (2009) identifies five distinguishable stages that can be observed which will indicate the possibility of failure. He writes “I’ve come to see institutional decline like a staged disease: harder to detect but easier to cure in the early stages, easier to detect but harder to cure in the later stages” (2009:5).

Collins starts describing his model by indicating that there are more ways to fail than to become successful, but if one can identify the signs that are evident at various stages in the development of an organisation one can begin to correct the tendency towards failure.

9.1 Stage 1: Hubris born of success This stage is evident when members overemphasise their own merit and in their arrogance regarding success and honour as an entitlement.

In academic institutions this would be seen in research centres that insist that others need to agree to their rules. There is no voluntary move into new technologies and methods. Excessive pride is expressed by leaders.

There are undisciplined leaps into areas where the centre can’t be the best. Management decisions are taken that fly in the face of reason. The core business, outlined above, is disregarded at the expense of putting creative energy into other fields and the primary ‘flywheel’ is neglected because of boredom or the assumption that it will continue automatically with the core business.

9.2 Stage 2: Undisciplined pursuit of more It is at this point that the motto ‘more is better’ becomes the chant. It is a straying from the disciplined creativity that was responsible for their success in the first place. It can be more of the same, if this is not possible then just more of what is considered as being achievements of success. The growth is often faster than staff can be recruited or funding sourced. This overreaching is what is evident in this stage.

Academic institutions are inclined to overreach without stimulating innovation, to ignite change and generally become lazy. Too much risk is taken, there is far too much leverage, financially or operationally, too much aggressive opportunism and far too much growth (in publications or higher degrees usually measured in income terms). Primarily there is a move away from research for its pure purposes and the investment in people, to research as a profit motive.

There is an addiction to scale and emphasis is on personal success. Administration is bureaucratic rather than driven by a culture of self‐management and self‐motivation and centred in committees. This drives away the right people that are required and is indicated by the number of people with the correct qualifications filling the positions within the centre even if the passion and the skills are not there.

301

Graham Trevor Myers The leadership indicator of trouble is the presence of an autocratic leader (or management) and the lack of a succession plan. Often this happens when the incubator leader retires or departs leaving the position to be filled by an outsider who does not understand the culture.

9.3 Stage 3: Denial of risk and peril The warning signs that ‘all is not right’ begin to become evident internally but the external achievements are still being lauded. This is often evident in the shifting of the locus of control for problems to external sources rather than acknowledging the possibility that they may be internal.

Academic centres start to take big bets in the face of mounting evidence that there are problems. There is a denial of any risk and any possible peril. There is a move away from experimenting with smaller interests of which some will be successful, towards one large ‘launch decision’ even though there may be holes below the waterline. There is a decline in customer engagement especially students, erosion of pipeline projects, a subtle decline in income and a loss of competitive advantage to other centres.

Customer loyalty and stakeholder engagement start to slow. Employees start to shield managers from negative facts, strong opinions are voiced without supporting data, the leaders have a low question to statement ratio when running meetings, decisions are not unanimously accepted, leaders try to get the credit and try to argue every point to look ‘smart’ and promote themselves through symbols and perks. Autopsies are carried out with blame as the outcome, especially when leaders do not deliver, resulting in unhealthy team dynamics and a marked decrease in the quantity and quality of dialogue and debate.

The final confirmation of this stage is an obsession with reorganising, by moving boxes around on organisation charts and drastically revised mission statements and strategic plans.

9.4 Stage 4: Grasping for salvation In this stage the enterprise enters a phase of visible decline. Response by management is to identify a quick salvation in the form of a visionary leader, a re‐engineering of the entity, the culture or the product. This approach is very short lived and temporary. The first sign of final failure is the search by the research centre for a ‘silver bullet’ in the form of adopting unfamiliar technology, trying to penetrate unfamiliar markets, gambling on a ‘game change’, hiring consultants, espousing the ‘rhetoric’ of change. All of these destroy momentum. What is really needed is a return to core values and espousal of the core business that gave the ‘flywheel’ momentum in the first place. Financial help and new ideas are temporary to say the least. The road to recovery is now long and hard but not impossible at this point. Should a research centre proceed to stage 5 there is usually no salvation.

9.5 Stage 5: Capitulation to irrelevance or death During this stage financial strength is eroded, the spirit of the individual is lost and the leaders abandon the entity by closing it or selling out. When this point is reached by a research centre all resources are eroded. This means cash, hope, enthusiasm, research skills and management backing. This ‘bleeding’, especially cash, has to stop first before any recovery can possibly be contemplated. Then one has to start to build again the culture of a winning organisation. As failure is usually in the mind, tactics have to be changed, the core purpose must be revisited and emphasised. But most of all the management must be purged bringing back the right people in the right seats.

What then is the ‘flywheel’ that keeps a research centre or department going?

10. Conclusion To answer the question then as to what is the ‘flywheel’ of a research department the following are put forward as suggestions:‐

It is the development of knowledge in the form of grand theory;

It takes students into the realms of ‘Understanding’ and ‘Wisdom’;

It advances knowledge in general;

302

Graham Trevor Myers

Is guided by principles rather than methodology;

In doing this a research unit should show the following characteristics:

Have staff who are community (student) centred rather than self‐centred;

Be accepting of divergent thinking;

Have staff who work on research principles rather than focus on methods;

Stick to what they know;

Show development rather than growth;

They are supported by finance and administration rather than driven by these;

There is a very strong connection to academic departments and students as their customers;

Leadership is not autocratic;

There is a spirit of empowerment within the centre;

Strategic decisions are made in consultation with all;

Funding and resources should be adequate rather than something which always hinders;

There should be a transformational evolutionary process; but most importantly

Values and Principles should dominate.

In conclusion, taking all the suggestions made. A Research Centre; be it a department or stand‐alone units in universities should at the centre of their objectives have the will to position themselves in the centre of World 3 of Mouton’s Model by using and directing World 2 knowledge. The main focus in any research department should be their commitment to retaining knowledge development as the core of their objectives and mission.

If this is not the case then the criteria that Collins puts forward in his model should be used to examine where the centre is at and this will determine the interventions that are necessary at that point in time.

References Babbie, E. and Mouton, J. (1998) The Practice of Social Research: South Africa Edition Oxford University Press. Cape Town.

Badat, S. (2012) Higher Education Change in post‐1994 South Africa. Wolpe Dialogue. Wits University.

Baird, LL. (1986) What Characterises a Productive Research Department? Research in Higher Education Vol 25, No3.

Butler, D. (2000) Sweden streamlines research administration. Nature Vol406 23 July 2000. www.nature.com.

Collins, J. (2009) How the Mighty Fall and some companies never give in. Business Books. London.

De Beer, M. (2012) Conversation documented concerning Supply Chain Management. Durban University of Technology (November) Ferriss, T. (2011) The 4‐Hour work week. Vermilion. London.

Glass, GV. 1976. Educational Researcher Nov. 1976 Vol. 5 no 10 3‐8 Primary, Secondary, and Meta‐Analysis of Research Hitt, M. A. (1997) Twenty‐First‐Century Organisations: Business Firms, Business Schools, and The Academy Presidential Address. Academkiy of Management Review 1998 Vol 23 No2 218 – 224.

Hitt, MA. (2000) The New Frontier: Transformation of Management for the New Millenium. Organisational Dynamics Winter edition Johnson, W. (1995) Mystical Theology: The Science of Love. Harper Collins. London. nd Kuhn, TS. 1970. The structure of scientific revolutions 2 ed. University of Chicargo Press. Chircargo.

McNiff, J. and Whitehead, J. (2005) Action Research for Teachers: A Practical Guide. David Foulton. London.

Mouton, J. (1996) Understanding Social Research. Van Schaik. Pretoria.

Myers, GT. (2012) Solving Problems: A Living Theory Approach to Research Development. European Conference of Research Methods Conference Proceedings.

Myers,GT and Yearwood, VM. (2012) A Journey into Analysis in the Development of a World of Meta‐Science. European Conference of Research Methods Conference Proceedings.

Popper, KR. (1972) Objective knowledge: an evolutionary approach. Clarenden. Oxford.

Schon, D. (1995) ‘Knowing‐in‐Action: The New Scholarship required a New Epistemology.’ Change November – December 27‐34.

Sitnikov, C. (undated) Education – Research process from Quality Management Models’ Perspective. Faculty of Economics and Business Administration, University of Craiova, Romania.

Tolstoy, L. Anna Karenina. Sparknotes Literature Guide Series at www.barnesand noble.com/sample/read/2940000054949 accessed on 7/12/2012 Wielicki, T. (2008) A concept of the Educational Supplu Chain (ESC): ICT Driven Paradigm Shift in Education Economics and Organisation of Enterprise Vol 2 (2) p 62 ‐67.

303

Thematic Content Analysis: Is There a Difference Between the Support Provided by the MAXQDA® and NVivo® Software Packages?

Mírian Oliveira1, Claudia Bitencourt2, Eduardo Teixeira1, Ana Clarissa Santos2 1 Pontifícia Universidade Católica do Rio Grande do Sul, Porto Alegre‐RS, Brazil 2 Universidade do Vale do Rio dos Sinos, São Leopoldo‐RS, Brazil miriano@pucrs.br claucbitencourt@gmail.com edkteixeira@yahoo.com.br aclarissa@cpovo.net

Abstract: Content analysis is a widely used technique in qualitative research. It can be organized into three approaches: lexical analysis (nature and richness of the vocabulary), syntactic (verb tenses and modes) and theme (themes and frequency). These approaches may be complementary or used separately, depending on the research objective. This study will focus on the analysis of thematic content. Whatever the approach or approaches chosen, the volume of work is usually large and qualitative data analysis software could help simplify the operational activities. The aim of this study is to comparatively analyze the contribution of the NVivo® and MAXQDA® software packages in thematic content analysis.

Achieving this objective will involve, firstly, a review of the main characteristics of thematic content analysis, then, an examination of the features of the NVivo® and MAXQDA® software packages, and finally, a comparison between them. The main results are: the similarity in terms of the features available for qualitative data analysis in the two software packages and the differences in terms of their interfaces. It should be noted that the potential gain provided by using these software packages for thematic content analysis depends on the researcher’s knowledge of content analysis, since they do not take the place of the researcher in the coding process. On the other hand, the support provided by such software packages means there is a considerable reduction in execution time as well as in the presentation based on different filters, which indirectly assists the investigator reflect on the process and on the results obtained from the content analysis. Moreover, the use of software to streamline operational tasks facilitates the researcher’s capacity to organize codes and obtain inferences. Hopefully, the results of this study will be of use to researchers interested in conducting thematic content ® ® analysis with the support of the MAXQDA or NVivo software packages.

® ® Keywords: content analysis, MAXQDA , NVIVO , coding, benefits, comparative analysis

1. Introduction Content analysis is one of the most widely used techniques for analyzing qualitative data. Content analysis can be organized into three approaches: lexical analysis (nature and richness of the vocabulary), syntactic (verb tenses and modes) and theme (themes and frequency). These approaches may be used together or separately, depending on the research objective. Whatever the approach or approaches chosen, the volume of work is usually large, depending on the amount of data involved as well as the maturity of the recursive process, that is, whether it is at the open, axial or selective coding stage. This paper addresses the analysis of thematic content and the use of software as a means to facilitate the process of defining and analyzing the categories (themes).

During open coding, the concepts and their properties are identified, as at this stage the goal is to generate codes. Axial coding is employed to develop the categories systematically and organize them according to the relationship between the codes. Finally, selective coding is used to integrate and refine the categories.

The encoding process demands time and attention on the part of the researcher, and can lead to errors in the application of codes to the text under analysis. One way to avoid such errors is to have the same researcher, or other researchers, repeat the encoding process. In addition, the use of this type of software for data analysis can help avoid coding errors. Pelle (2004) describes some features of word processors that can aid in content analysis, although they were not designed for the purpose. Lewis (2004) examines the major features of NVivo® and Atlas.ti®, two qualitative data analysis software packages.

The use of qualitative data analysis software to aid thematic content analysis: a) speeds up the process; b) enhances the rigor; c) provides more flexible data analysis from different perspectives; d) facilitates the exchange and reproduction of data; and e) allows the researcher to reflect in greater depth by reducing the operational activities (Bardin, 2009). It should be noted that the software does not replace the researcher in

304

Mírian Oliveira et al. the tasks of creating the codes, deciding the association of a given code to a specific piece of data, and in deciding which codes should be grouped in a certain category. According to Gibbs (2009), encoding and accessing are the main functions of most qualitative data analysis software, although some also offer a text and code search function, which can be very useful in data analysis. For example, MAXQDA® and NVivo® can be used for coding, accessing text, displaying the completed codes, writing memos and presenting the results in the form of tables and graphs.

® ® The aim of this study is to comparatively analyze the contribution provided by the MAXQDA and NVivo packages in thematic content analysis. In terms of structure, the remainder of this paper is organized as follows: section 2 contains a review of the literature on thematic content analysis; in sections 3 and 4, the features of the MAXQDA® and NVivo® software packages are described; in section 5 the similarities and differences of the two software packages are discussed; and in section 6, the conclusion is presented together with some ideas for further research.

2. Thematic content analysis Content analysis is defined as a set of analytical techniques (syntactic, lexical and thematic), in which systematic and objective procedures are employed to describe the content of messages, using qualitative or quantitative indicators that allow knowledge to be inferred (Bardin 2009). Thematic content analysis, as its name states, involves describing content based on themes.

Thematic content analysis consists of three stages: 1) pre‐analysis, 2) exploration, and 3) treatment and interpretation (Bardin 2009). Pre‐analysis involves: defining the objectives of the content analysis, selecting the material according to its relevance in relation to the goal, reading the material to be analyzed, and organizing the material for analysis. The material to be analyzed can be in written or oral form, a monologue or dialogue. At this stage, it is necessary to know the file formats that can be used with the qualitative data analysis software. For example, files can be formatted in txt, doc, rtf and pdf, among others.

The exploration stage involves defining the unit of analysis, i.e. the smallest part of the content in which an element can be identified, that is, the portion of text to which the code is associated. The unit of analysis can be a word, paragraph or theme. Besides the unit of analysis, the codes must also be defined. The codes may arise from reading the text, which are called in‐vivo codes, but may also be defined based on the literature, known as constructed code, or a combination of these two types. “Coding is a way of indexing or categorizing the text in order to establish a structure of thematic ideas in relation to it” (Gibbs 2009 p.60).

Codes can be: a) descriptive – which describe what is in the text, are close to what is said in the text, for example, the code could be “dancing” when the respondent mentions this activity in the text of an interview; b) analytical – which require reflection on what is expressed in the text, for example, when the respondent mentions in the text that after the accident that he cannot dance with his wife, the code could be “interrupted joint activity” (Gibbs 2009). The choice of the type of code to be used depends on the research objectives. The codes should be: comprehensive ‐ permit all the material to be classified; homogeneous ‐ attend a principle; exclusionary ‐ belong to only one code; relevant ‐ be related to the topic under study; productive ‐ allow inferences; and objective ‐ ensure the same result can be obtained by different people (Bardin 2009). The codebook should include: all the codes with their definitions, organized hierarchically, if there is more than one level; the rules for their application in the text; and any notes the investigator deems appropriate (Gibbs 2009).

Thematic content analysis may use quantitative and qualitative procedures. This choice will influence the definition of the coding rules, which may be: the presence or absence, frequency, intensity (use of adjectives, adverbs, etc.), direction (positive, negative neutral); order, or co‐occurrence. In quantitative terms, the coding aims to identify the frequency of the topics in the content to be analyzed. While in the qualitative perspective, the focus is on identifying the presence or absence of a particular topic in the content to be analyzed, so as, then, be able to better understand how this is presented in the text and its peculiarities, giving the analysis greater depth.

Coding involves three stages: open, axial and selective coding. “Open coding is intended to express data and phenomena in the form of concepts” (Flick 2004 p. 188). In open coding, reading and reflection on the text

305

Mírian Oliveira et al. facilitate the definition of the codes (Gibbs 2009). As a result of this phase there may be a large number of codes, which then must be clustered around phenomena that are relevant to the research question. The main result of open coding is the production of a list of codes, grouped into categories, and the definitions and observations generated at the time of creating the codes and categories (Flick 2004). The focus of axial coding is to identify relationships between the codes and categories defined during the open coding, adjusting the categories resulting from the open coding (Gibbs 2009). During axial coding, the researcher must be alternately inductive, developing the categories from the text, and deductive testing categories in different texts (Flick 2004). During selective coding, the core category is identified and its relationships defined with the other categories are defined (Gibbs 2009). This stage is a continuance of axial coding, but with a higher level of abstraction (Flick 2004). The objective of selective coding is to integrate all the categories (Strauss & Corbin 2008).

At this stage of thematic content analysis (exploration), in relation to the qualitative data analysis software, it is important to check how the codes are placed in software, the memos associated with the codes (definition of the codes, examples, reminders), and the relationship between the codes and the categories. Another important concern is how changes are made in the codes and their relationships in the qualitative data analysis software during the coding process, given that coding is a dynamic process during which new codes and categories may arise.

In the last step, known as treatment and interpretation, the content is coded, considering the rules defined in the exploration stage, and inferences can be drawn from it. The main problem at this stage of content analysis is the bias of the researcher. In order to mitigate this problem the reliability of the coding is checked. This can be done in three ways: stability ‐ the process is repeated by the same person; reproducibility ‐ the process is repeated for another person; accuracy ‐ the result is compared to a standard (if one exists) (Krippendorff 1980; Weber 1990). Testing stability is the weakest form of checking reliability, while the accuracy is considered the strongest (Krippendorff 1980). Reproducibility, also referred to as intercoder reliability by Kurasaki (2010), is a measure of agreement between two or more researchers of how the codes were applied to the text.

In the data treatment and interpretation stage it is important to check the way in which the qualitative data analysis software displays the text associated with each of the codes and categories, as well as the tables and figures generated by the software that summarize the coding. Moreover, when testing the reliability of the coding it is useful to be able to compare the coding carried out by other researchers or by the same investigator at different times.

3. MAXQDA® software The MAXQDA® is a software package for analyzing qualitative data that can be used for content analysis. Its interface offers an experience very similar to that provided by WINDOWS® based software. Its basic structure consists of four windows: one that provides the data from each project, the texts and groups of texts (document system); one that provides the structure of the codes and categories (system code); one for editing and consulting texts (document browser), and another for conducting searches and checks regarding coded material (retrieved segments).

The creation of codes and categories in MAXQDA® is controlled by the user, who can create codes before, during or after the material is analyzed. The codes are presented in a hierarchical, tree‐like, structure that allows several levels of codes to be indexed. The whole system of codes can be deleted, edited, and reclassified, without the loss of any content previously examined by the user.

In MAXQDA®, the colors of the codes and categories can be customized. This means that when a color is assigned to a certain category, its codes do not automatically inherit that color, but can be user‐defined. The color system is a tool that facilitates visual identification of the coded parts. Reminders, which may vary in form from descriptions of the meaning of the code to specific notes, can also be associated to the codes.

Moreover, reminders free can be created within a project, and associate a set of codes chosen by the user.

Another possibility is to associate variables to the texts. With this function, texts representing different respondents can be separated according to, for example, gender, profession, nationality, etc. Together with the categorization by variables MAXQDA®, has mixed functions that allow these variables to be combined with

306

Mírian Oliveira et al. qualitative data such as: quote matrix, which creates a file that separates segments associated with a code determined by selected variables, allowing codes to be associated with the different classes; crosstabs, in which the coded segments associated with variables are created and evaluated, based on the variables; and typology tables, which show an overall composition of the variables associated with the typologies generated by the qualitative data.

MAXQDA® can use different units of meaning in the coding process. The user can assign a code to a unit of meaning that may range from a letter or syllable to entire documents. It should be noted that the program can code not only texts, but also image files. Furthermore, it is possible to associate the same passage of text to more than one code. These two features together allow the user to superimpose codes without them necessarily being totally equal.

Codes can be associated with the selected text in several ways: some part of the text is indexed to an existing code; a new code is created and associated with a part of the text, a theme is sought in a text and the parts pertaining to that theme are automatically coded.

As with the codebook, the coded excerpts may be modified in relation to their content or assigned code. The user can edit and delete the coded excerpts by modifying the code to which the passage was indexed, editing the content of the coding or deleting the coded excerpt. In addition, weights can be assigned to the coded passages.

During or after the work of coding, the user can check the correspondence between a set of codes and texts (segments of texts that have been marked with certain codes) using the window “retrieved segments.” The ® ® content of this window can be viewed directly, or presented in spreadsheets in the EXCEL or HTML formats.

There are many ways to present the content of the coded segments. The “text retrieval” function in MAXQDA® allows different types of searches, such as: combining all the retained segments; retained segments that represent the intersection of codes; sum of the coded segments that include an intersection; delete overlapping coded segments; segments in sequence, or that are close to each other.

In addition to the options mentioned above in this section, MAXQDA® has a set of visual tools to facilitate the analysis of coded segments: code matrix browser, which provides a preview of the number of segments coded for each code and each document; code relations browser, which provides a view of the intersection of codes in a document, a group of documents or all documents, thus facilitating recognition of the interrelationship of the codes; document portrait is a representation of all coded segments in a document; codeline, which generates a matrix that lists the paragraphs of a text with codes that were used in it and the document comparison chart, which compares the codes used in each document.

® MAXQDA has other useful tools for the user, such as basic tools for transcribing documents; map building tools that show the data connections, tools for building dictionaries, and tools to create links between the material that is internal and the that is material external to the project.

4. NVivo® software NVivo® is a software package for digitally coding texts or images that allows the user to synchronize evidence and make analytically richer intersections (Teixeira 2009). By using this software, the researcher is able to manage the empirical material in a single location (text, spreadsheets, audio, video or images), and can add, modify, connect and cross reference data, or even record ideas in the form of memos, in order not to lose any insights generated while viewing the material.

Moreover, the sources can be coded, that is, the material can be gathered into categories or “nodes”, where portions of the collected material are indexed and can be classified according to persons, places or organizations. Each category contains a central meaning and is represented by a keyword. Importantly, the categories, created before the coding, are defined by the researcher, in order to identify commonalities in the texts and sort them for later analysis. Collected material can be transcribed directly in the software by inserting the recording directly into system. If the researcher prefers not to transcribe the interview or video, the audio clips can also be coded.

307

Mírian Oliveira et al.

The items available for navigating in the program are (NVivo® 9 Manual):

Sources ‐ Space for the importation of collected material;

Nodes ‐ containers that allow excerpts from the material to be gathered according to the research categories;

Classifications ‐ space for descriptive information about sources, nodes and relationships;

Clusters ‐ material grouped according to research requirements;

Consultations ‐ allows the researcher to perform searches in the coded material and save them during the analyses;

Reports ‐ An item that provides reports and extracts for tracking the progress of the research or providing data for use in other applications;

Models ‐ forms and connectors that allow the research data to be visually presented.

First, the nodes or categories that will serve to code the collected material in accordance with the theoretical and research question are created. The researcher then accesses the fonts that were entered into the program and starts the coding process. To insert an excerpt in a node, the user has to simply select it and drag it to the corresponding node which appears on the left side of the program interface.

In addition to being able to code audio clips, if there is no need to transcribe the collected material, a common ® method of coding with NVivo is to directly read the selected words, phrases or longer excerpts from the sources. The coding of an excerpt in one node does not prevent it from being coded in another, since the program does not work in a destructive way by removing the words from the original selected source (Teixeira 2009). The program allows the user to make automatic searches for words or lexical patterns during the coding of the texts, images, audios or videos.

When the excerpts are coded, the number of sources and references inserted in each node can be viewed, as well as in how many and in which nodes each excerpt is coded. Upon opening a node, by clicking on its name, coded sections are explored, so identifying the source and the percentage complete source, the text or length of time of video/audio that the excerpt represents.

An advantage of this software is the option to note and save comments and reminders which are often overlooked during the analysis. These texts are recorded in memos. When the coding of the collected material is completed, the process of classifying the sources (people, places, organizations) can begin.

In the item Classification, viewed in the program menu, the classification spreadsheets of the sources and nodes are structured, providing an overview of the specific items. Here, the attributes and their values are established and the nodes and sources are classified according to criteria defined in data collection or even after the preliminary analysis of the sources.

Once the attributes are defined and the sources and/or nodes classified, the coding matrix can be constructed, in which the relations between the coded excerpts and the attributes are established. In addition to these relations, content close to coded excerpts can be combined, thus facilitating the emergence of new insights as the analysis progresses. Further, the program is able to compare coding, i.e. comparisons can e made between the coding of different researchers/groups by accessing the “coding comparison” resource.

Once the comparison matrices are established, charts and graphs can be created to display or explore the research data, so diversifying the means by which the results can be understood. Besides the graphics, the program can be used to create models showing the connections between the data by linking the forms of the model to the research items.

While the data analysis is being performed, the reports provided by the program allow the user to analyze and review the work in progress; identify the most frequent themes; present the insights noted during the coding ® ® and work with the data in other applications (Excel or Word ).

308

Mírian Oliveira et al.

The NVivo® program helps the researcher’s work, enabling greater data organization based on the established categories, especially when the research comprises a large volume of data. In addition, it allows relationships among the data to be determined faster and at greater depth. It is important to emphasize that both during the coding process and the cross referencing between the nodes and cases, the result will depend on factors outside the program, such as: the quality of the material used; the research objectives; and the ability of the researcher to analyze and establish links with the literature.

5. Comparative analyses: the MAXQDA® and NVivo® software packages The two analyzed software programs have similar features. The main differences were found to lie in interface and the nomenclature, which may lead to a preference for one or the other software, although they do not constitute the bases for preference in relation to their use in thematic content analysis. It should be noted that both software programs include the key features relevant to thematic content analysis: definition of the codes and categories at different times during the analysis; and hierarchical levels (codes and categories).

Among the benefits found of using these qualitative data analysis software packages, were: the incorporation of new texts in the software, so that coding can begin before all the material has been collected; the organization and storage of the empirical material allowing cross‐referencing that would be difficult to perform manually; the creation of memos, permitting insights obtained the coding process to be recorded, which also serve to exchange ideas between different researchers; the use of hierarchical levels, allowing the association of codes into categories.

® ® The table below shows some of the features of the MAXQDA and NVivo qualitative data analysis software packages.

Table 1: Main features of MAXQDA® and NVivo® software packages Features Import and display texts in various formats Create codes (hierarchical levels) Delete and alter codes View lists of codes and categories Write memos Code text Associate variables to the text Search text Access coded texts View the coded text in various forms.

Graphs and tables

MAXQDA® X X X X X X X X X X X

NVivo® X X X X X X X X X X X

6. Conclusions This article has analyzed the use of software for thematic qualitative content analysis. Thus, two software packages that are widely used in this type of analysis: MAXQDA® and NVivo® are compared by highlighting the features and peculiarities, benefits and tools provided by each software package.

Thus, the benefits of using these software packages in thematic content analysis can be highlighted. It should be noted that the potential gain in using these software programs for thematic content analysis depends on the researcher’s knowledge of content analysis, since these software packages do not replace the role of the researcher in the coding process. On the other hand, the support provided by such software packages means there is a considerable reduction in execution time as well as in the presentation based on different filters, which indirectly helps the investigator reflect on the process and on the results obtained from the content analysis.

The analyzed software packages are similar in some ways, although they are different in terms of the nomenclature used and the way the content analysis process and results are presented. It is hoped that the results of this research will of use to researchers interested in conducting thematic content analysis with the ® ® support of the MAXQDA or NVivo software packages, allowing them to identify the one that best meets their needs by pointing out some differences and similarities between them.

309

Mírian Oliveira et al.

Following this study, a comparison of the MAXQDA® and NVivo® software packages in operational terms will be made, by using an example of thematic content analysis conducted with the aid of the two software packages.

Acknowledgements The authors are grateful for the support received from CNPq (Conselho Nacional de Desenvolvimento Científico e Tecnológico – Brazil) and CAPES (Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brazil).

References Bardin, L. (2009) Análise de conteúdo, PUF, Paris.

Flick, U. (2004) Uma introdução à pesquisa qualitativa, Bookman, Porto Alegre.

Gibbs, G. (2009) Análise de dados qualitativos, Bookman, Porto Alegre.

Krippendorff, K. (1980) Content analysis ‐ an introduction to its methodology, Sage Publications, Thousand Oaks.

Kurasaki, K.S. (2010) “Intercoder reliability for validating conclusions drawn from open‐ended interview data”, Field Methods, Vol 12, No. 3, pp 179‐194.

Lewis, R.B. (2004) “NVIVO 2.0 and ATLAS.ti 5.0: a comparative review of two popular qualitative data‐analysis programs”, Field Methods, Vol 16, No. 4, pp 439‐469.

Manual do Nvivo 9 (2013) www.qrsinternational.com.

Pelle, N.L. (2004) “Simplifying qualitative data analysis using general purpose software tools”, Field Methods, Vol 16, No. 1, pp 85‐108.

Strauss, A. and Corbin, J. (2008) Pesquisa qualitativa, Bookman, Porto Alegre.

Teixeira, A. N. (2009) “A produção televisiva do crime violento na modernidade tardia”, Thesis, UFRGS, Porto Alegre.

Weber, R. P. (1990) Basic content analysis, Sage Publications, Thousand Oaks.

310

A Comparison of Three Projective Techniques: Lessons Learned From a Business Research Class Phallapa Petison College of Management, Mahidol University, Thailand Phallapa.pet@mahidol.ac.th

Abstract: Although projective techniques are becoming more popular in the business field, the techniques are mainly adopted to explore consumer purchasing behavior and the consumer’s perception towards brand. It is interesting to note that a limited number of studies in the management field have been conducted in order to compare different results from the application of different projective techniques. Therefore, this research has been conducted using three different projective techniques: word association, sentence completion, and cartoon test aimed at exploring how to apply each technique effectively when a respondent has no experience with this projective research method. Sixty‐ six respondents participated in this research. They were Thai students, registered for a business research class in a Master of Management degree course. Some of them had experience in conducting research at bachelor level but were inexperienced with projective techniques. Each respondent was required to apply three projective techniques to examine their perception of a Japanese product. After the students had completed the assignment, focus groups were conducted. In terms of applying the techniques effectively, the research results revealed that the respondents had different perceptions on different techniques. Comparing the three techniques, respondents expressed that the order least difficult to most difficult technique was; word association, sentence completion, and cartoon test. This is because the word association technique in this study is slightly adapted from a general format. The adaption is similar to the one used during a popular television program where people are asked to share three words, expressing an idea. By familiarity with the three word expressions, respondents could generate an idea of a Japanese product quickly and easily. The characteristic of sentence completion helped stimulate ideas for inexperienced respondents. On the application of a cartoon test with inexperienced respondents, a step by step clarification of how to fill up a balloon dialogue was considered as a crucial initial step in order to obtain the expected research results. Moreover, clear explanations help prevent misunderstandings and allow researchers to obtain more useable resulting material. Furthermore, introduction of the famous Japanese cartoon, “Doraemon”, which was known to all respondents, produced a positive stimulus for expressing their perception of a Japanese product. Thus with inexperienced respondents, different techniques provide different results. Therefore, the decision to select an appropriate technique should be made according to the purpose of the research. Finally, adaption of the three techniques to render them more familiar to inexperienced respondents is suggested in order to obtain greater participation and therefore expose more insightful, truer and accurate feelings.

Keywords: projective techniques, word association, sentence completion, cartoon test, business research

1. Introduction An incremental interest in adopting qualitative research in the business management field has been noted during the last twenty five years. However, projective techniques have not been employed as much as much other qualitative research tools (Özlem, 2009) although projective techniques have long been recorded as successful tools in psychology since its origination in 1895 (Sayre, 2001). Thereafter, it was increasing adopted in social science and later on in business studies. This may be because in conducting research, we begin with the objectives of what we want to know. For most business research questions, we generally gather reliable and valid data from surveys in order to generalize business phenomena. Quantitative research then takes a principal role that best suits to explain the cause and effect of business scenarios (Sayre, 2001). Conversely, in exploring causal relationships through quantitative research, there is a need to understand the relationship between the consumer and product/service. Gathering information from consumers; somehow, may not disclose directly their preferences and feelings. Therefore, projective techniques were adopted.

From literature reviews, projective technique paradigms are to indirectly elicit a response from stimuli that can be in the structured or unstructured form in order to discover the inner thoughts of respondents (Donoghue, 2000). Projective techniques are applied for examining attitude (Stienman, 2008; Will, Eadie, and MacAskill, 1996), motive (Özlem, 2009), behavior (Choo, 1966, Özlem, 2009), perception (Choo, 1966; Ramsey et. al.

2004), feeling/emotion (Stienman, 2008; Will, Eadie, and MacAskill, 1996; Ramsey, Ibbotson, Bell and Gray, B. (2004), thought (Stienman, 2008; Ramsey et. al. 2004), and experience (Stienman, 2008). The first projective techniques study published in the business field was Haire’s shopping list in 1950 (Steinman, 2008). Later on, more complex business scenarios demanded understanding of the insight of consumer driven usage of projective techniques in the field.

311

Phallapa Petison Based on the generic principle of projective techniques, in business, they are considered appropriate to identify market opportunity, generate hypotheses and ideas, explore and develop new concepts of product/service (Roininen, Arvola, and Lähteenmäki, 2006), understand consumer perspective (Vidal, Ares, and Giménez, 2013), discover consumer decision making, and auditing brand perception (Hofstede, Joris, Natascha, and Menno,1997).

Although recently projective techniques are becoming more popular in the business field, the techniques are mainly adopted to explore consumer purchasing behavior and perceptions regarding brand. It is interesting to note that a limited number of studies in the management field have been conducted in order to compare different results from the application of different projective techniques.

Therefore, this research has been conducted using three different projective techniques: word association, sentence completion, and a cartoon test aimed at (1) exploring how to apply each technique effectively when a respondent has no experience with this projective research method; (2) studying what are the concerns when adopting each technique and (3) analyzing what are different results when apply different techniques

2. Projective techniques in the business field Business scholars pointed out an appropriation of using projective techniques in the field of management by giving reasons that with its principles of projective techniques allowed researchers to overcome barriers of both the researcher conducting the research and the respondents involved in the study. For the research side, a projective was appropriated for the non skilled researcher (Sayre, 2001). Furthermore, it helps eliminate communication barriers such as language. For respondents, it helps eliminate fear of being judged as negative manager (Stienman, 2008).

Unlike in others fields of study where types of projective techniques are classified according to process e.g.

Zubin, Evron and Schumer (1965), Dole, Levitt, Baggaley, and Stewart (1986), etc. classified projective techniques into 2 types: Normal (in which sub‐ classification was of two types: classic, and generalized) and abnormal. In the business field, projective techniques are recommended for application according to task, response type and research purpose. However, there is inconsistency in terms of classification among scholars.

In total, there are four types:

Association: This type is the most adopted projective technique in business studies (Donoghue, 2000).

Response is automatic to stimuli which can be word or image. The association is appropriated for extracting information about product and brand (Will, Eadie, and MacAskill, 1996; Burns and Lennon, 1993). Furthermore, particular research in branding development for what is called “Personification,” and is based on a generic concept of association aimed at exploring brand personality and characteristic.

Completion: For completion, respondents are asked to complete an incomplete sentence or story (Donoghue, 2000). Stienman (2008) indicated that completion is frequently applied in business research as it benefits in giving rich information within a short time span. Completion helps express relationships with brand/product by, descriptive user of the product (Sayre, 2001).

Construction: Bubble drawing and third person questioning are the most common use of construction (Stienman, 2008). Construction is regarded as suitable in the exploration of respondent attitude and preference toward brand and retailer (Sayre, 2001) as well as respondent behavior (Donoghue, 2000).

Expressive: Expressive allows the respondent to express feelings toward product/service/brand through role play, draw, or painting.

Practically, projective techniques are unique ways of asking customers about themselves and the products they use. Combining various projective technique results helps the researcher understand more about consumer reality (Sayre, 2001).

3. Research methodologies This research has been conducted using three different projective techniques: word association, sentence completion, and cartoon test; corresponding to three different techniques: association, completion, and construction.

312

Phallapa Petison Sixty‐ six respondents participated in this research. They were Thai students, registered for a business research class in a Master of Management degree course. The course was delivered in the third term of their five term length course.

Among those, there were 18 males and 48 females. They were from various bachelor degree backgrounds, from pure science to art (art, language, education, business, engineer, law, pharmaceutical, science, etc.).

Their range of age is from 24‐38. The average age is 28. Some had experience in conducting research at bachelor degree level but were inexperienced with projective techniques. Each respondent was required to apply three projective techniques to examine their perceptions of a Japanese product. The theory of projective ideology was introduced to them prior to testing of an application of the three techniques.

For the association type, word association was selected with a slight adaption for local context. Generally, the word association technique is implemented by either (1) asking respondents to choose a list of words which first come to mind or (2) presenting stimuli then asking for the first thought that comes to mind. In this study, neither of the two implementations mentioned in literature reviews was conducted. This study decided to adapt a technique; however, it was still based on the grounded ideology of completion type: first thought coming to mind. Currently, in Thailand, there is a TV program named VRZO that is popular among teenagers.

The program emcee will ask interviewees to explain their opinion about any topic, using only three words.

Thus, in this study, respondents were requested to provide three words that they thought of on the topic of “Japanese products”.

For the completion type, a sentence completion technique was selected instead of story completion technique because story completion is mainly claimed to help examine the emotions of respondents towards brand as the main propose (Choo, 1966), but the test in this study was to allow the respondents to experience how to answer and how to decide on a word completion test rather than to explore students’ feelings toward any particular Japanese brand. In total, the seventeen in completed sentences are asked to complete. The incomplete sentences were designed based on an adaptation of sample questions from Reddy and Acharyulu (2008); Gupta (2003). Samples of incomplete sentences are: Japanese products are....; Japanese products should.....; Japanese products should not...., my most favourite Japanese product is....; the most uniqueness characteristic of Japanese product is....; etc.

Additionally, the sentence completion test was designed with the intention of replication of the question. The incomplete sentence numbers 3 and 8; asking “Japanese products seem like .....”; are replications. This is aimed to examine whether novice student respondents would notice the replication or not and whether the replication question should be used to ensure the validity of the answer.

For the construction type, a cartoon test was selected. Generally, when applying this technique in business, an ambiguous consumer‐related scenario (e.g. customer stands in front of the shelf, customer queues up at cashier counter) is suggested. The main purpose of this study is to allow the student respondents to experience different techniques and find out what helps them understand more as a novice researcher, but not to examine their perception towards a Japanese brand. Creation of cartoon selection is brought in.

Thai students are very familiar with Japanese Manga (Japanese cartoon). Therefore, the test was created from the Japanese cartoon “Doraemon”. Doraemon is a story about the daily life of an elementary school boy named “Nobita” and his robot cat “Doraemon”. The reasons for the Doraemon selection are that this cartoon is very famous. Almost all Thai students have been familiar with this cartoon since their formative years, both in the TV program and comic book form. Furthermore, this cartoon is not gender specific as both boys and girls like it. The test was comprised of twelve cartoon blocks and sixteen balloon dialogues. Instead of randomly picking a page of cartoons from a comic book and removing its original dialogue, the cartoon page was newly created by selecting blocks from various comic issues. This enabled the selection of cartoon scenes that expressed less emotion between the two main characters, and involved two character dialogue and individual thought between Nobita and Doraemon.

From the three projective technique categories, the student respondents were instructed to begin with the association technique as sequentially it seems less ambiguous than the other two techniques.

313

Phallapa Petison After the student respondents had completed all three techniques, focus groups were conducted, aimed at discussing their opinions on the application of each technique as well as the clarification of research results.

4. Results and discussions The study shows a comparison between all three projective techniques. The first two: word association and sentence completion were favoured over cartoon completion. It is interesting to note that respondents with a science background tended to prefer sentence completion rather than word association, the reason given was that its structures helped stimulate ideas.

4.1 Word association For word association, in terms of method, research results showed that adapting word association to meet with the local context was very useful for novices. Respondents commented that if they needed to conduct qualitative research in the future, they would select a technique that allowed respondents to understand easily. Furthermore, they stated that from their experience of being a respondent, using things that they are familiar would eliminate their fear and confusion. Respondents shared the experience that it helped them feel more relaxed and therefore enjoyed this method much more.

When further detailed questions were asked about fear and confusion, the answer from the majority was that they were unfamiliar with qualitative techniques but more familiar with surveys. They felt more comfortable in answering with three words as they had often seen in the TV show. In addition, most respondents commented that when talking about research, they felt it was distant from the reality of the business world, and, in their thoughts, more theoretical and serious and may not be applicable in real business practice. However, with the technique, familiar from the TV show, they felt that this was something that could be easily used to conduct research in real business life.

In terms of research results obtained, this study found that a few student respondents answer only one word as they misunderstood three as three syllables for example “Quality” as it is pronounced “qual‐li‐ty”.

However, this study found that even though those students answered only with one word with three syllables, the word that they answered with was similar to those with three words.

Through word association, the study showed that the most selected words were relevant to the attributes of Japanese products for example: quality, durable, expensive. Second, the most frequently found was the product brand name such as Daiso, Glico, Sanrio, etc.

4.2 Sentence completion Although respondents expressed the view that this projective technique was one of the easiest to conduct, over half of sixty six respondents did not know that there was a duplicated question within the seventeen sentences. Those with awareness of a duplicated question provided the same answer to both questions, or either left it blank. Among those without awareness, most answered the same for both duplicated questions.

When the respondents had completed the incomplete sentences, it was observed that the majority began with the first question and followed it through to the end. Very few students started and went through the questions to find those they could answer easily first. In terms of the research results obtained, there was no difference in results no matter which method respondents completed in the test.

It is interesting to find that in the first incomplete sentence: “Japanese product is….” ; instead of mentioning the first thought of Japanese product attribute: “Quality”; like that found in word association. Most respondents answered were “product that made/manufacturer in Japan”. This reflected the country of origin.

This seemed surprising as for a projective technique; it would be expected that the first thought to mind would have been the same no matter what technique was applied.

When analyzing content, more details emerged. The results showed that respondents mainly described Japanese products rather than brand name. The outstanding attributes were high quality, innovation, high price, technology, identity, unique, and packaging. Many stated that Japanese product price should be reduced.

314

Phallapa Petison In terms of quality, the study found that few students compared Japanese product quality with European, Thai, Chinese, or Korean products. All believed in world class acceptable quality.

Another interesting point that this study uncovered is that although sixteen questions were asked differently one answer that almost all respondents indicated was the identity of Japanese products. The discussion on the issue of identity revealed that respondents view identity in a positive way, and appreciated its unique character. Several suggested that it would be an advantage if there were English instructions for the product.

During the focus group, the issue of the most famous and most preferred brands was raised. The results revealed that the most famous and most prefer is not always the same brand. The most famous brands found were Toyota, Sanrio, and Sony. Whilst the most preferred brands were Sanrio, Shiseido, and Sony.

Furthermore, those who gave the answer Sanrio and Shiseido were female while the Toyota brand was mentioned by both female and male. Likewise, Sony was mentioned by males. However, it should be taken into account that there were more female than male respondents in this study.

The brand that the majority considered the most famous and preferred was identified as Sanrio. Results from the focus group indicated that respondents had been familiar with Sanrio products since they were young. This was a surprise as Sanrio is mostly an accessory for the young, but working women still like it. Sanrio is an expensive brand which the majority stated they would like but sometimes their parents did not buy it for them due to the price. Presently, they are all working and so can afford to buy it for themselves.

Another point raised in a discussion was that there seems to be no new brand name mentioned. The majority of the mentioned brands were introduced some time ago. The newest brand found was Tokyo banana as this is the most popular souvenir from Japan. Although Tokyo banana was mentioned briefly in word association, it was found more in sentence completion.

4.3 Cartoon test From respondents’ point of view the cartoon test is regarded as the most difficult technique of all for implementation as well as interpretation of results. However, results from the cartoon test produce such richness of information that the effort is considered well worthwhile. Results reflect not only brand name and attributes but also their attitude, perception and concern about Japanese products. The cartoon dialogue showed that there was male awareness of such female brands as Sanrio, Kitty, and Shiseido unlike in the other two techniques, which failed to detect this. Furthermore, numerous brands and categories were found. Details of brand were also expressed such as quality, brand history, innovation, packaging, etc. Although food categories remain the most mentioned products, new brands are mentioned Ramen Champion, Yamazaki Bakery, etc. Moreover, new product categories such as robotics, fashion apparel, and games are mentioned.

When respondents mentioned quality, it was found that all who mentioned this point believed in the high quality of Japanese products. They somehow compared them with Korean, American, and European brands.

However, there were brand products that were not Japanese, and those mentioned were SKII (P&G), and Kitkat green tea (Nestle). Respondent reasoning was that although both products were not Japanese they were thinking of green tea because they often think about Japan. For SKII, advertising shows that SKII uses Japanese scenarios and mentioned a Japanese herb compound, so they misunderstood it to be a Japanese brand.

Further, via the cartoon test, it was found that respondents mentioned behavior/characteristics of Japanese people, which again went undetected using the other two techniques.

The issue of adopting the Doraemon cartoon was discussed during the focus group, and all respondents commented that it helped stimulate ideas because of its relevance to Japan and their familiarity with the cartoon, from their very early years. Furthermore, discussion revealed that applying another cartoon such as using an American cartoon, respondents suggested that with the same story and reference to a “Japanese product”, they would create a storyline with Japanese relevance, such as Batman having a Japanese girlfriend or Mickey Mouse and the gang going to Japan.

315

Phallapa Petison Using cartoons also somehow helps facilitate ideas and boosts the creative process. Content analysis showed that some “no” meaning words such as “Uhm….Arrh…..Oh…Yah…” prompted the respondents to be creative in order to fill the speech balloon completely by the invention of expressive nonsense words.

5. Conclusions and implications In terms of applying the techniques effectively, the research results revealed that the respondents had different perceptions on different techniques. With inexperienced respondents, different techniques provide different results. Therefore, the decision to select an appropriate technique should be made according to the purpose of the research. Although literature reviews showed that the more ambiguous the best to draw insightful (Sayre, 2001) however this may not best suit for novice level. Begin with the more the structure of the technique, the better the clearer the answers. Furthermore, allowing inexperienced respondents unlimited time for completing each technique test seemed to facilitate their idea simulation.

In addition, with the inexperienced respondent, application of more than one technique was found to be useful. Word association although provided less information than the other two techniques, but it is useful to begin with as it helped to stimulate the ideas of inexperienced respondents, then these were followed with more complication one.

An adaption of the three techniques to render them more familiar to inexperienced respondents is suggested in order to obtain greater participation and therefore expose more insightful, truer and accurate feelings.

However, adaption of techniques to local context has a few majors of concerns when implemented. Clear explanation is a requirement as they may misunderstand method and provide less information than it ought to be. At any given point, localization of projective technique like this study on word association may applicable and appropriate with a particular group of respondents. In addition, creativity in adopting techniques to local context and relevance to the study subject like a cartoon test are recommended. However, providing clear instruction prior applying each technique is suggested. This is one of the most key success factors particular with novice.

References Choo, K.S. (1966). The use of projective techniques in consumer attitude research. M.B.A. University of Singapore.

Burns, L.D. and Lennon, S.J. (1993). “Social perception: methods for measuring our perception of others”. International Textile and Apparel Association Special Publication, Vol. 5, pp.153‐159.

Dole, A.R., Levitt, D., Baggaley, A.R. and Stewart, B.L. (1986) “What do employers want: Functions, knowledge areas, and characteristics desired of professional psychologists”. Psychological Reports, Vol. 58, pp. 643‐658.

Donoghue, S. (2000) “Projective techniques in consumer research”, Journal of Family Ecology and Consumer Science, Vol. 28, pp. 47‐53.

Gupta, S.L. (2003) Marketing Research, Excel Books, Delhi, India Hofstede, A., Joris, H.F., Natascha, W. and Menno, J. (1997) “Projective techniques for brand image research: Two personification‐based methods explored”, Qualitative Market Research: An International Journal, Vol. 10, No. 3, pp.

300‐309. Özlem, H. S. (2009) “Projective techniques for social sciences and business research”, Journal of Consumer Marketing, Vol. 26, No. 3, pp. 223 Ramsey, E., Ibbotson, P., Bell, J. and Gray, B. (2004) “A projective perspective of international e‐service”, Qualitative Market Research: An international Journal, Vol. 7, No. 1, pp. 34‐47.

Reddy, N.P. and Achryulu, G.V.R.K. (2008) Marketing Research, Excel Books, Delhi, India Roininen, K., Arvola, A., and Lähteenmäki, L. (2006). Exploring consumers’ perceptions of local food with two different qualitative techniques: Laddering and word association, Fod Quality and Preference, Vol. 17 (2006), pp. 20‐30.

Sayre, S. (2001) Qualitative Methods for Marketplace Research, Sage Publication, Thousand Oaks California Stienman, B. R. (2008) “Projective techniques in consumer research”, Northeastern Association of Business, Economics, and Technology Proceedings, pp. 253‐261.

Vidal, L., Ares, G. and Giménez, A. (2013) “Projective techniques to uncover consumer perception: Application of three methodologies to ready‐to‐eat salads”, Food Quality and Preference, Vol. 28, pp. 1‐7.

Will, V., Eadie, D. and MacAskill, S. (1996) “Projective and enabling techniques explored”, Market Intelligence and Planning, Vol. 14, No. 6, 38‐43.

Zubin, J., Evron, L.D. and Schumer, F. (1965) An Experimental Approach to Projective Technique, Wiley and Son Inc, New York

316

Research Methodological Trends in Business and Management Studies in South Africa: An Exploratory Bibliometric Survey Magdal Pienaar‐Marais1 and Gerhard Moolman2 1 Strategic Planning and Reporting, Corporate Governance, National Research Foundation, South Africa 2 Business Intelligence, Corporate Governance, National Research Foundation, South Africa magdal@nrf.ac.za; gmoolman@nrf.ac.za

Abstract: It can be assumed that the attention paid to the research methodology of a particular discipline or subject field is an indicator of the salience of the research methodology with respect to the research dynamics of the field. The prominence of research methodology is reflected in, among other things, publications focusing on research methodology and relevant sub‐themes. This paper reports the results of a content analysis of abstracts of journal articles in business and management studies where at least one author had a South African address. The sample was drawn from the Thomson Reuter’s Institute for Scientific Information (ISI) Web of Knowledge database for the period 2007 to 2011. In the content analysis, methodological themes (i.e. quantitative, qualitative, mono‐methodological, multi‐methodological) that featured in the abstract of each publication were used as the unit of analysis, while the title and entire abstract of the publication served as the contextual unit. Standard reliability checks were done on the thematic coding. Against the background of an official review of the state of Master’s of Business Administration (MBA) degrees in South Africa, this study includes an indication of the share of business and management studies in South African knowledge production; the salience of methodological themes in business and management publications; specific preferences in the application of methodology in these fields, and selected demographic characteristics of authors. The study found, firstly, that multiple‐authored publications dominate, secondly, that there is a preference for mono‐methodological research designs and quantitative and empirical research methods, and thirdly that international collaboration did not meaningfully influence the choice of research designs and methods. The paper concludes by highlighting the implications of the findings.

Keywords: methodological themes, research methodology, business and management studies

1. Introduction This study was undertaken at a time that the research system of South Africa – and indeed many other countries across the world – is under great stress caused by the economic situation and a government committed to a monitoring and evaluation strategy that is increasingly asking for proof of return on investment. Against this background the present self‐initiated project was undertaken to evaluate the state of research methodology (in terms of both quantity and quality) in the important field of business and management studies. The rationale for investigating this topic is addressed below.

1.1 The importance of methodology Research methodology has become a dynamic field of research in which new approaches to uncovering the truth through scientific research are tested (e.g. the increase in the prominence of multi‐methodological research designs).

Also, business and management studies have grown to become a prominent field of teaching and research, especially in higher education (e.g. through business schools and the offering of master’s programmes in business administration), although the emphasis sometimes seems to fall more on transferring successful practices and skills development than on discovering new knowledge or the scientific confirmation of theories and/or modelling exercises.

By 2002, there were 37 programmes offered by 27 providers in South Africa. The Higher Education Quality Committee, (HEQC), a standing committee of the Council for Higher Education (CHE), has executive responsibility for quality promotion and quality assurance in higher education. In 2003 the CHE commenced with the process of reaccreditation of MBA programmes. The review followed on the proliferation of the MBA offerings in South Africa that was ascribed to the expressed need of government for an increase in postgraduate enrolments in the fields of business and management studies to support public service employees at the medium and upper management levels (CHE, 2004: 7).

317

Magdal Pienaar‐Marais and Gerhard Moolman As part of the review, a survey testing the perceptions of students indicated that students enrolled for MBA qualification to hone their skills (including entrepreneurial skills), to capitalise on prospects of access to better career development opportunities and earning higher salaries (CHE, 2004: 25). The review was however very critical of some of the offerings and stated that ‘ a master’s degree without sufficient research methodology, theoretical grounding and analytical skills seems misplaced…….The issue of research in the MBA is not limited just to a matter of standards but is fundamentally related to curriculum content and knowledge production’ (CHE, 2004:102). The review concluded that research methodology played a minor role in the curriculum of most MBAs (CHE 2004: 63). Following the review, several providers offering MBAs in South Africa had their programmes de‐accredited because they did not meet the HEQC criteria for quality. The HEQC Board granted full accreditation to 19 MBA programmes from 17 South African higher education institutions (CHE, 2004).

The past few years have also seen a few scientific South African reviews of the state of research in business and management studies, as well as in economics. One of these articles, by Botha, Lilford and Pitt (2011), assessed progress in business and management studies by means of a content analysis of articles in South Africa’s three top business and management journals over the previous 15 years, covering a broad range of content categories, ranging from the productivity of individual researchers, to the ranking of institutions, the methodological preferences shown in the journals, and the length of the journal articles. The methodological variable included an analysis of the research design of the papers (i.e. theory development, quantitative research, qualitative research and other designs) and reported that quantitative designs, found in 54% of the total articles, far outnumbered the other designs (theoretical development: 22%; qualitative designs: 11%).

This paper is especially commendable for differentiating between the various methods used in each of the designs. The authors stressed the importance of methodological development for the progress of disciplines.

Another recent paper by Kriek, Beaty and Nkomo (2009) analyses research methods used in international management literature for the period 1991‐2007 and found that a preponderance for the use of quantitative methods as opposed to qualitative methods.

An article by Luiz (2009), reviewed the state of economics departments in South Africa across a broad spectrum of variables, ranging from staff profiles, to course curricula and research output. Luiz used a questionnaire that was filled out by heads of economics departments. With respect to research, Luiz made the obvious but nevertheless very important remark, ‘Research is fundamental to the academic endeavour. Our function is both to engage with new knowledge with our students and also to develop this new knowledge through the publication of original research’ (2009: 596). The assessment was based, inter alia, on the number of publications in the databases of the Institute for Scientific Information (ISI) and the International Bibliography of the Social Sciences (IBSS), and in journals on the list of publications accredited by the South African government department responsible for higher education, as well as the responses of the heads of economics departments. Luiz concluded that the status of research in economics had improved in South Africa, as shown by growth in output and more publications in top journals.

These and other papers as well as textbooks on research methodology (e.g. Babbie and Mouton, 2001) emphasise the axiomatic importance of research methodology to the progress of a discipline ‐ and to the quality of high‐risk decisions in business, as Blumberg, Cooper and Schindler (2008: 5) point out. The quality of research is a co‐determinant of the scientific status of a discipline, and the methodological preferences within a discipline largely map the route that the discipline will follow in its evolution as a knowledge producer. This study explores some elements of the methodological road map of business and management studies.

1.2 Research quality assurance and assessment The role of the Council on Higher Education (CHE) as an independent statutory body responsible for quality assurance in higher education in South Africa has already been mentioned. Another measure of research quality in business and management studies is the National Research Foundation rating, which entails the assessment of the research outputs of individual researchers by their peers internationally. Based on the assessment of peers, a researcher is allocated a rating, which is valid for five years and can be used to leverage research funds. In this regard it can be noted that the ‘rated’ researchers in business and management studies and their related fields constitute approximately 4% of the total population of rated scientist in the country (NRF: 2013). As is shown in Figure 1, the proportion has been maintained at a fairly steady state over recent years.

318

Magdal Pienaar‐Marais and Gerhard Moolman

Figure 1: Number of rated researchers in business, economics and accounting disciplines In addition to these methods of quality assurance in research, the international practice of quantifying the relative performance of individual researchers, departments, disciplines and institutions through bibliometric indices is followed. Like all research‐evaluation methods, the quantitative analysis of bibliometrics is not without its limitations (Brownman and Stergiou, 2008; Pendlebury, 2008). Hartzing (2013), for example, cautions against the limitations in the data that disproportionally affect the Social Sciences, especially in terms of document categories. Citation analysis and content analysis are, however, frequently used bibliometric methods. An important premise in science is that only that which has been published (and has thus been peer evaluated) can qualify as scientific knowledge. It follows that an analysis of publications (whether the analysis covers only titles and other relevant bibliographic information, or abstracts, or preferably the entire publication) could uncover important ‘traits’ of the discipline, and over time the dynamics in the discipline.

This paper offers a modest contribution to mapping some of the traits of the field of business and management studies in South Africa.

2. Research questions and objectives The interrelated set of research questions addressed in this project can be formulated as follows:

What does the profile of research methodology in business and management studies, and closely allied fields, look like?

Who does the research?

Where are they located?

Are there meaningful signs of collaboration (between institutions or between countries, for example)?

What methodological preferences are manifested in research publications?

The paper addresses these research questions by means of a content analysis of ISI‐recorded publications with at least one author with a South African institutional address. The main objective of the study was thus to generate research information that would contribute to the methodological characterisation of research in business and management studies, and to a discussion of the strengths and vulnerabilities of this research area. Another objective was to offer suggestions on research methodological training in business and management studies.

3. Research design The key research question, which centred on the profile of research methodology in business and management publications, led to a bibliometric survey design. The components of the research design are discussed in the following sub‐sections.

3.1 Sample The Thomson Reuter’s Institute for Scientific Information (ISI) Web of Knowledge database was selected, since it is a global research resource that provides access to leading citation databases that include current and retrospective journals and proceedings data in the sciences, social sciences, arts and humanities.

The National Research Foundation is a subscriber to the Web of Science and has as stipulated in the terms and conditions, the right to use bibliographic and abstract contents for research publication, on condition that it is not for commercial use or sale.

319

Magdal Pienaar‐Marais and Gerhard Moolman In the present study, abstracts for approximately 78% of the sample were available for analysis.

3.1.1 Research areas Two research areas, Business and Economics (one area) and Management sciences (the second area) were selected to ensure the inclusion of all relevant South African publications.

3.1.2 Period The five‐year period of 2007 to 2011 was selected to ensure the fullest possible coverage of recent publications, and to enable the search for trends (the data base was accessed on 30 November 2012).

3.1.3 Sample The total sample consisted of 1 643 publication titles, 129 of which did not have an abstract and could consequently not be subjected to content analysis, and a further 137 of which could not be classified in terms of the category system (see 3.2.1), thus leaving a sample 0f 1 377 publications available for the full content analysis.

3.2 Process of content analysis Two approaches were used in the analytical phase of the project. In the first phase, already coded information that required no interpretation (e.g. the number of authors and their institutional locations) was recorded directly from the information in the relevant sections of the ISI database. In the second phase: where interpretation by the researchers was required (e.g. type of methodology used for the research), the researchers used the abstracts as the information source for the coding process (for an overview of the method, see, e.g., Krippendorff, 2004).

3.2.1 Categories Each publication was classified in terms of the following two sets of categories. Firstly, census‐type information was recorded, including authorship, and institutional and international location. Secondly, each of the publications with an abstract was assessed on the basis of the following two criteria:

Relevance to the scope of the present study: If the publication was not relevant to the scope of this study, e.g. dealing with an engineering subject without referring to the management thereof, it was classified as such and only analysed for authorship, institutional and country location.

Amenability to analysis: The abstract was assessed in terms of whether it contained sufficient information on the research design to allow reliable classification according to the selected categories (see next paragraph). If not, it was classified as such but only analysed for authorship, institutional and country location.

Those publications that could be included in the study were further classified in terms of the following methodological categories:

Quantitative: Studies whose research design included measures that would yield numerical data (nominal, rank‐order and/or interval scales) and normally be amenable to the application of statistical techniques (it should be noted that mathematical and statistical modelling studies were coded as Quantitative in this study).

Qualitative: Studies that took an insider’s perspective and used non‐numerical information for further analysis.

Mono‐methodological1: Studies that reported a research design using a single approach (conceptual or empirical), and within the chosen approach used only one method of information gathering:

Conceptual (here used interchangeably with ‘theoretical’): Studies that focused on the analysis of key concepts in a field and/or considered the theoretical basis of research; also included here were studies that merely set out project plans

1

‘Methodology’ is used here since the difference between ‘mono’ and ‘multi’ concerns more than mere methods, but includes broader issues such as research strategy and design.

320

Magdal Pienaar‐Marais and Gerhard Moolman

Empirical: Studies that involved some quantitative or qualitative form of observation/measurement that was used for information gathering, such as a Likert scale or a focus group (it should be noted that ‘empirical’ was not defined as being synonymous with ‘quantitative’).

Multi‐methodological: Studies that reported using a research design that included both conceptual and empirical approaches, or studies that used only one of these approaches, but utilised more than one method of information gathering:

Conceptual: Studies that considered the theoretical and methodological underpinnings of research or set out project plans

Empirical: Studies that involved some quantitative or qualitative form of observation/measurement that was used for information gathering, such as a Likert scale or a focus group.

3.2.2 Analysts The second author was responsible for the database management and the census part of the analysis. The first author, supported by a researcher with experience in content analysis and research methodology, was responsible for the methodological analyses. The two analysts each assessed and classified 50% of the publications.

Inter‐coder reliability was measured by having each of the two analysts independently re‐analyse ten randomly selected publications that had been coded and classified by the other; thus yielding 120 pairs of comparisons (20 publications times 6 categories). The analysts differed in eight pairs of codings, i.e. recording 94% agreement which suggested a high level of inter‐coder reliability.

3.3 Representation of the results The content analyses were done in Microsoft Excel spreadsheet format, and the results are represented in graphic form (bar charts and pie charts).

4. Results The results of the bibliometric survey are reported in the following sequence: disciplinary yield, authorship of the publications; description of the research methodological dimensions analysed; and cross‐tabulation of authorship and methodological dimensions.

4.1 Disciplinary yield Apart from the research area, Thomson Reuters also provides Web of Science categories which enable the separation of Business and Economics publications if necessary. The data base produced a total of 1 643 titles with the following distribution across the Web of Science categories (percentage of total in brackets): Business: 671 (41%); Management: 1 057 (64%); and Economics: 642 (39%). (It should be noted that one publication can be ascribed to more than one category). Where appropriate, statistical comparisons were made between business and management on the one hand, and economics on the other to ascertain whether the presence of economics‐specific topics, such as econometrics, might have influenced the profiles of methodology in business and management, However, it turned out that the patterns remained practically the same irrespective of whether economics was excluded or not. The results of the total sample, economics included, are consequently reported in the next sections.

4.2 Authorship Twenty nine percent of the 1 643 publications were single authored; 40% had two authors; and 31% had more than two authors (at least one of whom was institutionally located in South Africa); of the 21% of the multi‐ authored publications at least one author was based abroad. Figure 2 shows the country affiliation of the authors who co‐published with South Africans.

The largest group comprised institutional addresses in the USA, which can probably be accounted for, among other reasons, on the shape (prominence of business and management schools, for instance) and size of the US research system. The second highest percentage of institutional addresses abroad was for collaborators in the UK, which is not surprising given the historic links between South Africa and the UK. It is also worth noting

321

Magdal Pienaar‐Marais and Gerhard Moolman the sixth position of authors with Nigerian institutional addresses and the fact that Kenya is the only other African country among the top 23 shown in this figure. (The prominence of collaboration with Nigeria based authors can also be seen in the finding by Botha and her colleagues, namely that Nigerian researchers accounted for 56% of African researchers who had published in the three journals they content analysed in their study). The ‘other’ group of countries in Figure 2 consisted of 28 countries. Removing publications in economics from the pool caused a few rank changes, e.g. Nigeria moving up to third position ahead of the Netherlands and Botswana appearing in the 22 nd position.

Figure 2: International co‐authorship 2007‐2011 Against the background of the relatively high proportion of internationally co‐authored publications, the next issue to be addressed was the extent of inter‐institutional collaboration within South Africa. The data showed that South African interinstitutional co‐authorship was rather limited, namely 12% of the 1 293 multi‐authored publications by South African based authors.

4.3 Methodological dimensions The findings of the content analysis with respect to the methodological characteristics of the research, based on the categories (see section 3.2.1), are discussed next.

The analysis showed that 51% of the publications were quantitative in their research approach, 38% qualitative and 11% both quantitative and qualitative (e.g. in the case of some of the multi‐methodological designs).

These percentages did not significantly change when publications in economics were not included in the analyses, 48%, 42% and 9% respectively. In this regard reference can be made to Botha et al. (2011) reporting 64% and 11% for quantitative and qualitative articles in the journals they analysed.

The information in Table 1 shows that the majority of the publications (81%) reported using a mono‐ methodological design. Furthermore, there was a preference for empirical approaches (51%) over conceptual ones (32%) or combined conceptual and empirical approaches, e.g. in multi‐methodological approaches (17%), irrespective of whether economics titles were included or not.

Table 1: Publication preferences for research designs and approaches to information gathering Information approach Conceptual and empirical Conceptual only Empirical only

Design Mono‐methodological Multi‐methodological; n Col % n Col % 87 8 148 56 406 36 28 11 621 56 87 33 1 114 Row: 81% 263 Row: 19%

N

1 377*

Note: * Number of publications with sufficient information to allow content analysis (see section 4)

4.4 Cross‐tabulation of authorship and methodological dimensions The question addressed in this section is whether the characteristics of authorship (single or multiple; local or international) are associated with different choices of research methodology (mono‐ or multi‐methodological

322

Magdal Pienaar‐Marais and Gerhard Moolman designs; and conceptual or empirical approaches to information gathering).There was not a significant difference between single and multiple‐authored publications in the use of research designs (mono vs. multi‐ methodology) – 85% versus 15%, and 79% versus 21% respectively. The message so far is clear, namely the very strong preference for mono‐methodological designs.

Figure 3: Number of authors and choice of research methods Inspection of Figure 3 further shows reasonably similar patterns by the two groups (single and multiple authors) in choice of research methods, the only notable difference being single‐authored publications reporting a high usage of conceptual methods (42%) as opposed to the 28% of multiple‐authored publications.

Excluding economics titles resulted in virtually the same percentages, viz. 43% and 36%.The fact of international collaboration seems not to make a significant difference in either research design or published choice of research methods as can be seen in Figures 4 and 5. Figure 4 represents the percentage of publications that used either a mono or multi‐methodological design and were authored by research teams with international members (International, n=298) and teams consisting of researchers from different institutions in South Africa (n=135). The proportion of multi‐methodological publications relative to mono‐ methodological ones is only marginally higher in the case of international collaborative research.

Figure 4: International collaboration and choice of research design Inspection of Figure 5 shows empirical methods to have been more commonly used in both sub‐samples of publications (51 and 57%) while there was really no difference in the use of both empirical and conceptual methods. The only notable difference between international and national inter‐institutional collaboration reflected in Figure 4 seems to be the fact that international collaboration contributed to relatively higher utilisation of conceptual methods than did South African interinstitutional teams.

Figure 5: Collaboration and choice of research method

323

Magdal Pienaar‐Marais and Gerhard Moolman The analysis of the term frequency exercise yielded the following results: The term ‘mixed methods’ were only explicitly mentioned in the abstracts of nine articles, and the total number of articles that dealt explicitly with research methodology in Business and Management Sciences as topic amounted to 26.

5. Discussion The following sections address the more important findings of the study.

5.1 Reflections on the information base The main objective of this project was to give an overview of the researchers, their collaborative orientations, and their research methodological preferences in the field of business and management studies. The quantitative aspects of the research design comprised a content analysis of the abstracts of the 1 377 of an original sample of 1 644 business and management papers in the ISI database, published between 2007 and 2011, with at least one South African‐based author. It should be noted that, from a methodological perspective, the quality of the abstracts varied in important respects, which made accurate classification of some of the papers problematic (some abstracts, for example, referred to analyses of data, without any reference to the source or nature of such data). In this context, it is also worth noting that the content analysis used categories (cf Krippendorff, 2004) that could best be described as medium grained and did not allow drilling deeper into the potentially richer methodological reef (as Botha et al., 2011 could do). Moreover, the fields covered in this survey included economics and accountancy, which introduced a number of difficult‐to‐ classify papers, however ‐ and in contrast to earlier expectations ‐ excluding economics publications from the analyses had practically no effect on the results – at most a couple of percentage points either way in all cases.

5.2 Main findings It is perhaps not surprising, given the international character of business and management studies that about half the publications analysed were co‐authored by researchers from other countries. This is an important finding for at least three reasons. Firstly, the chances of intellectual and methodological cross‐fertilisation increase proportionately to the number of countries participating in a particular project, other factors being equal. Above all, there are opportunities for international cross‐validation of findings. Secondly, the potential for future exchange of researchers and students is bound to increase, with important consequences particularly for a small country such as South Africa. Thirdly, the future opportunities for funding from international agencies for the individual researcher or the institution are likely to improve under normal circumstances. The relative prominence of Nigerian co‐authors and the relative absence of co‐authors from other African countries should be highlighted, especially in the light of the growing internationalisation of research and the high priority the South African government put on intra‐African cooperation.

The results further showed that research in business and management studies predominantly used mono‐ methodological designs and empirical methods. A similar conclusion was reached by Botha et al. (2011) in their analysis of three South African business and management journals, although they did refer to a growth in the use of qualitative methods over the previous 15 year, albeit from a very low base line. They also showed that approximately a quarter of the publications they analysed could be classified as theory development which was somewhat lower than the result reported in the present study. In this regard, the analysis by Kriek et al.

(2009) of research methods in international management studies covering a period of 17 years further confirmed the preference for quantitative methods. These observations give rise to a number of interrelated questions, such as

Whether sufficient attention has been given to appropriate qualitative and conceptual approaches to research in research methodology courses?

Whether qualitative and conceptual approaches have been intentionally and appropriately designed to assist business and management in problem solving and decision making?

Whether training in mixed and multi‐methodological research designs focused on complementarity – rather than assumed irreconcilability ‐ between them receive sufficient attention?

Whether a combination of editorial policies of giving preference to shorter articles and institutional strategies focused on optimising output did not have the (unintended) consequence of promoting mono‐ methodological as opposed to multi‐methodological projects and publications?

324

Magdal Pienaar‐Marais and Gerhard Moolman

5.3 Thoughts on the research design The findings reported in this paper offer a medium‐resolution picture of the methodological landscape in business and management studies and should contribute to debates on ‘the state of the art’ in these important research fields in South Africa. It would be reasonable to suggest that the findings of this study reflect of the situation of research in business and management studies in South Africa, since all publications in the ISI database for the five‐year period 2007 to 2011 were covered, and ISI offers wide coverage and broad representivity.

The design of the study imposes some limitations on the findings, three of which are worth listing. Firstly, the resolution of the analysis was not as fine as it could have been, in that more detailed information on research methodology could perhaps have been extracted from the abstracts. Secondly, the inclusion of current projects in the analysis, especially but not exclusively research undertaken towards an advanced degree, would have enabled inferences on future developments.

6. Conclusions, implications and further research The findings of this study would allow a number of high‐level conclusions to be drawn.

6.1 Conclusions and implications Firstly, researchers and research in business and management studies compare favourably with other intellectual fields if their relative position on indicators such as the National Research Foundation’s Evaluation and Rating System is considered. Secondly, there are positive signs of international collaboration, but inter‐ institutional collaboration within the country can probably improve. The fact that international and inter‐ institutional collaborative research is high on the agendas of governments, national funding agencies and business itself should serve as a stimulus to researchers in this regard. Thirdly, the findings reported in this paper suggest that business and management research is predominantly empirically orientated, although theoretical and qualitative methodological approaches are reasonably well represented in the data base that was analysed. Fourthly, the findings of this project can with caution be generalised to the South African national system of business and management research, but the use of abstracts as base source of information introduced limitations on the depth of the content analyses – the findings should be seen as a medium resolution picture of the relevant landscape.

Taking a picture as metaphor further, it can be concluded that the research questions (section 2) were answered and as such the paper may serve to encourage dialogue on research in business and management studies.

6.2 Further research This was an exploration of a small part of the methodological profile of research in business and management studies in South Africa. At least two follow‐up studies are currently planned. The first study is intended to provide for a more detailed analysis of the methodological dimensions, e.g. specific methods used in each of the quantitative and qualitative research strategies; the extent to which the authors explicitly report on reliability and validity considerations; and the use of primary as opposed to secondary data in their analyses.

The second study would include an analysis of the projects covered in NEXUS (http://stardata.nrf.ac.za/), a database of current and completed South African projects, including studies for master’s and PhD degrees thus covering local research outputs that might not be included in international data bases, serving as a cross‐ validation of ISI‐based analyses and also offering an empirical basis for future planning.

Acknowledgements The authors wish to thank Faranah Osman, Director: Governance of the NRF for her encouragement, support and comments on the first draft. Henda van den Berg and Marna van Rooyen are thanked for providing essential research information. The assistance of Hendrik Marais (University of Pretoria) with the content analysis is gratefully acknowledged. Simon Gathua (Da Vinci Institute) is thanked for statistical support. The value‐adding copy editing of Robyn Arnold is highly appreciated. The ECRM review process is acknowledged – it added further value to aspects of the second version of this paper.

325

Magdal Pienaar‐Marais and Gerhard Moolman

References Babbie, E. and Mouton, J. 2001. The Practice of Social Research. Cape Town: Oxford University Press.

Blumberg, B., Cooper, D.R. and Schindler, P.S. 2008. Business Research Methods. London: McGraw‐Hill.

Botha, E., Lilford, N. & Pitt, L. 2011. South African management literature over the past fifteen years: Content analysis of the three top South African management journals, South African Journal of Business Management, 42(4): 89–98.

Brownman, H.I. & Stergiou, K.I. 2008. The use and misuse of bibliometric indices in evaluating scholarly performance, Ethics in Science and Environmental Politics, 8(1): 1–3.

Council for Higher Education (CHE HEQC) (South African Council on Higher Education, Higher Education Quality Committee). 2005 MBA Review. [Online] Available at: http://www.che.ac.za/documents/d000096/ . Accessed: 25 March 2013.

Chernyi, AI. 2009. The ISI Web of Knowledge, a modern system for the information support of scientific research: A review.

Scientific and Technical Information Processing, 36(6): 351‐358.

Harzing, A. 2013. Document categories in the ISI Web of Knowledge: Misunderstanding the social sciences? Scientometrics, 94: 23‐34.

Kriek, H.S., Beaty, D. and Nkomo, S. 2009. Theory building trends in international management research. An archival review of preferred methods. South African Journal of Economics and Management Sciences. 12(1):126‐135.

Krippendorff, K. 2004. Content Analysis: An Introduction to its Methodology. Thousand Oaks, CA: Sage.

Luiz, M. 2009. Evaluating the performance of South African economics departments, South African Journal of Economics, 77(4): 591–602.

National Research Foundation (NRF) 2013. NEXUS Database System. http://stardata.nrf.ac.za/ National Research Foundation (NRF) 2013. Rating History. Pretoria.

Pendlebury, D.A. 2008. Using Bibliometrics in Evaluating Research (White Paper). Philadelphia, PA: Thomson Reuters. 2012. Web of Science. [Online] Available via subscription: http://thonponreutrs.com/. Accessed: 30 November 2012.

326

Everyday Ethics for Student Research Projects Martin Rich Cass Business School, London, UK M.G.Rich@city.ac.uk

Abstract: This paper was partly prompted by an incident when the author received a questionnaire, emailed to a number of staff within an institution, from an undergraduate seeking information from an assignment. There were plenty of flaws in the questionnaire, although it is worth noting that an undergraduate in their second year of studying a technically oriented subject could not reasonably be expected to have much familiarity with the sort of research methods generally associated with the social sciences. However, it was surprising that the most vocal criticism that the student received from academics who had been sent the questionnaire was not a criticism of the questionnaire itself, but an observation that, because the assignment involved work with human subjects (the academics being asked to respond) it should have been checked with the university’s ethics committee. University have ethics policies which are principally designed for research projects, or at least for research conducted by students who have some training in research methods and for whom independent inquiry is a major part of their studies. However, as the above anecdote illustrates, elements of these research projects apply to a very wide range of student activity and can extend to work by students for whom gathering any sort of primary data is only a tiny element of their work. It is tempting to suggest that for these cases ethical considerations can be treated superficially. Certainly it could be argued that approving one questionnaire within an undergraduate course is not a particularly fruitful use of time for the members of an ethics committee. However the student in such a case, as part of the university community, still has an obligation to act in an ethical manner. Therefore it is interesting to raise the question of how students who are not principally researchers, but who are nevertheless in a position where they might need to collect primary data and to carry out interviews and surveys, can be encouraged and supported in ethical actions. It is worth discussing whether students might be deterred from seeking original data because of the ethical issues, and whether there is a tendency to ignore ethical issues either through ignorance – as appeared to be the case with the anecdote above – or through wilful attempts to avoid any barriers. But the focus of the discussion will be on the sort of teaching, coaching, and support which would be most effective in providing students carrying out small‐scale research, which does fall into the category where ethical considerations become significant, with the confidence to deal with ethical issues effectively.

Keywords: research ethics, taught students, research training, research approval process

1. Background This paper sets out to discuss issues around the need to provide a level of ethical approval when students in higher education carry out an element of research – for example an undergraduate dissertation – as part of a course where the principal mode of learning is through formal instruction. The paper sets out a number of principles and the examples cited refer to experiences within business and management (and to a small extent finance) undergraduate degrees at a British business school. The first example referred to, however, which was significant because of its contribution to the motivation for this paper was not a business or management student. Instead this was a student who sent an inquiry, with no covering note and no explanation, to a mailing list covering staff across the university asking for their response on a particular subject. So this request was received by academics with a responsibility for business, but it was not at all clear what the student was studying, or why they might value a business academic’s viewpoint.

Nevertheless the author was able to determine rapidly that the query originated with a second year undergraduate taking a computer science‐related course. Before being overly critical of the student concerned, it is worth noting that an undergraduate in their second year of studying a technically oriented subject could not reasonably be expected to have much familiarity with the sort of research methods, and especially with the characteristics of sound research governance, generally associated with the social sciences.

It is also possible that the student sent the query to a staff mailing list, sincerely believing that the message would be circulated only to staff in one department who were likely to know the student personally. In that case, the issue becomes as much one of email behaviour as of research methods.

However, it was surprising that the most vocal criticism that the student received from academics who had been sent the questionnaire was not a criticism of the questionnaire itself, but an observation that, because the assignment involved work with human subjects (the academics being asked to respond) it should have been checked with the university’s ethics committee. The originator of this criticism also chose to share it with other people to whom the original request had been sent. Suddenly a student, who thought that collecting

327

Martin Rich results from a quick questionnaire would be a simple way to gain a few extra marks in an essay assignment about use of a particular online service, found themselves in a position where they were expected to follow a procedure that was principally intended to be carried out by staff responsible for research. But the tone of one piece of vociferous criticism raised questions of how important the process of ethical behaviour should be.

The author of this paper is directly involved in the supervision and training of undergraduate students carrying out projects, which introduce an element of independent research into students’ final year. Similar training issues arise with taught postgraduate students and, to a lesser extent, when students carry out extended coursework in earlier stages of their studies. While the project handbook for the undergraduate course, on which the author teaches, includes the guidance that ethical approval may be necessary for projects involving human subjects, the exact scope of this is not made clear. In practice very few of the undergraduates do seek ethical approval, and even though the institutional procedures are such that most cases can be dealt with on a light‐touch basis, it is highly unlikely that the staff responsible for administering the approval process would be able to cope with large numbers of undergraduates and taught postgraduates contacting them.

In preparation for this paper, the author sought ethical approval, on the basis that the use of vignettes from real student experiences, however brief, and the discussion of the experiences of students and supervisors, constituted the use of human subjects. The research officer who handled the author’s initial query immediately suggested that there was no need for any further discussion, on the basis that this was straightforward, that as a member of staff the author could be trusted, and consequently this case merited a very light‐touch approach.

2. Why research ethics?

Within the last two decades there have been significant drivers to include ever firmer governance processes into research. For those working in social sciences – including business and management – in the UK the principal impetus has come from the Economic and Social Research Council through whom a lot of research funding flows. Over the years they have developed a ‘framework for research excellence’ (Economic and social research council, 2012). Within the framework there is an expectation that any research involving humans should be subject to some sort of approval process, where any ethical considerations might be discussed.

Underlying the ESRC framework are six principles:

Integrity and quality

Fully inform researchers and participants

Confidentiality and anonymity

Voluntary participation, free from coercion

Avoiding harm

Independence and impartiality of the researchers.

As outlined in the Ethics Guidebook (no date) different institutions have different processes for securing approval, and the onus is on the student or the member of staff to navigate the processes and to seek approval. Within the current author’s institution, the research officers responsible for the approval process see their interactions as being with academic staff, but usually not with students directly. This implies a delegation of some responsibility at least, from the research officer to students’ supervisors, to ensure that procedures are followed where necessary.

In the context of educational research – directly relevant to the author’s experience in gaining ethical approval for this paper, Hammersley and Traiano (2012) outline a set of five principles not dissimilar to those listed by the ESRC: for example their first principle is ‘minimising harm’ which closely mirrors the ESRC requirement to avoid harm. Significantly, they emphasise ‘offering reciprocity’ which implies that in some way the subjects of research should be able to gain some benefit from it.

While their focus is on medical research, and they trace the impetus to formalise approval processes to European Union requirements for clinical trials (a very different research setting from that encountered by most business and management undergraduates), Dixon‐Woods et al (2007) discuss the significance of the letters sent by research ethics committees as a means of understanding the research approval processes. They

328

Martin Rich explore the dichotomy between the requirement to formalise the process and the intrinsic uncertainties surrounding the subject.

Returning to the business and management field, ethical leadership (Brown et al, 2005) which focuses both on appropriate conduct, and on the promotion of such conduct to others, offers one possible template for students’ behaviour, particularly in cases where student research projects include direct contact with individuals. In such contexts students should behave in a manner which is trustworthy, predictable, and inspires others to follow their example. In the context of academic research, Macfarlane (2009) proposes an approach based on a set of virtues to which researchers might with to aspire. In contrast to the bureaucratic view of research ethics which a focus on the approval process is liable to foster, these sources deal with the intrinsic desirability of ethical behaviour.

Anecdotal evidence, gleaned from informal conversations with academics responsible for supervising projects on undergraduate and taught postgraduate courses, and also from a cursory survey of material available on the web as guidance for students, reinforces the view that there is considerable variation between institutions in the interpretation of ethical policies. For instance at least one set of guidelines within an institution sets out to distinguish between interviews and surveys carried out face‐to‐face, and those carried out electronically, with face‐to‐face contact being somehow more human and therefore requiring more rigorous approval. But this distinction becomes problematic as soon as one considers interviews carried out through Skype, which are electronically mediated but where the personal dynamics may well be closer to those experienced in a face‐to‐ face interview. It also fails to recognise that with electronic communication there can be more scope for causing offence than in a face‐to‐face discussion, because of the lack of visual clues and body language that can mitigate any unfortunate or offensive remarks.

This is not presented in any spirit of criticism of those responsible for drawing up institutional guidelines, but is merely included to give an indication of how difficult it is. Other examples include questionnaires which are valuable, and which can be used to determine whether a light touch is sufficient, although these can fall into the trap of focusing on the mechanics of the research to be carried out at the expense of addressing the principles.

Another set of guidelines around ethics, relevant to researchers in business and management, is offered by the association of business schools (2010). This addresses a number of practical points, which overlap with the issues raised by the ESRC (for instance the ESRC places great store on the voluntary nature of participation and the association of business schools observes that consent should never be presumed) but is perhaps more manageable as a document to which students might turn for an indication of how to operate ethically.

There are significant criticisms of the influence of research ethics frameworks. Haggerty (2004) looks at the Canadian case, and while he sets out his own position as a strong advocate of behaving ethically as a researcher, he also expresses deep reservations about the value of formal ethics procedures, and fears that they will merely encourage researchers to pursue bland activities which carry limited risks. Stanley and Wise (2010) robustly criticise the ESRC approach, again expressing concern about the extent to which formalising procedures can stifle original thought. But they also dismiss the ESRC framework as the prompt for a set of excessively unwieldy approval processes, in their view implemented with complete disregard to researchers’ professional knowledge and judgement.

In practice ethics policies do appear to be applied with a level of pragmatism. What might be described as the norms of many institutions (Crawford and Ostrom, 1995), in that these are behaviours that are tacitly understood and define the character of the organisation, include an understanding that much taught student work, if not exempt from the requirement to secure ethics approval, should be handled with the lightest of touches. Indeed the ESRC framework does include the distinction between a light touch review and a full approval process, depending on the nature of the research, as a key element in the process.

3. Illustrative vignettes from undergraduate projects Three brief accounts of undergraduate projects in business and management which raised ethical issues – not necessarily ones which would have been easily resolved through strict adherence to the ESRC framework – are given below. All of these, in the context of the range of research carried out by the business school in which

329

Martin Rich they were locating, constituted small scale and low risk research, and it would have been grossly inappropriate to apply anything beyond a light‐touch review. But there are subtleties in each of them which deserve some consideration. To reiterate, these examples are purely illustrative and have been chosen to provide an indication of the complexities which can arise in this area. They are not presented as being in any way representative of undergraduate students’ approach to individual and independent study.

3.1 Participant observation of a website acting as an exchange for short‐term employment This student carried out a project based around a website that they had been involved with, as a part‐time business partner, for some time. The aim was to explore whether this site, which matched people looking for short‐term work in various professional fields with potential employers, could contribute to disruptive innovation (Christensen, 1997) within one of the business sectors for which the website offered opportunities.

At the supervisor’s behest, the student adopted the term ‘participant observer’ which is recognised as a valuable way of generating data for qualitative research (Iacono et al, 2009).

While the student, by virtue of having experience of working closely with the site, and having followed its success in attracting both employers and employees, brought an immense amount of insight to the project, it is open to question whether they could really act as an impartial and independent observer. This characteristic, superficially at least, is in conflict with the ESRC conditions specifying participants’ impartiality.

In practice it is unrealistic to expect most undergraduates carrying out projects to be impartial – after all they are encouraged to work on topics that capture their imagination and that frequently implies that they already have strong views. In fact the ESRC framework does allow for potential conflicts of interest to be recognised and notes that these should be made explicit.

This project used observations and background information, often gained through privileged access to the business, in tandem with a series of structured interviews with key players. As part of the participant observation process, and as an interesting and valuable reflective exercise, the student prepared answers to the same set of interview questions to which the other key players responded.

3.2 Interviewing shoppers in selected supermarkets on green consumer habits This student had studied some of the literature, particularly that around emerging parts of the marketing discipline, relevant to the propensity among consumers to favour products that were promoted as being ‘green’ or environmentally responsible purchases. Like many projects, this had an element of determining the match between theory and practice, and whether the dynamics posited in the marketing literature could be mirrored in a set of practical settings. Inevitably the student could not obtain a large enough sample, nor a sufficiently sophisticated sampling mechanism, to draw any very general conclusions, But the student did have the opportunity to carry out face‐to‐face interviews in a number of local supermarkets, and to regard these as case studies. Moreover there were several supermarkets close to the university which catered for a middle‐ class and environmentally conscious demographic, whose views would be worth considering as they were likely to be early adopters of such projects. Carrying out these live interviews appealed to the student, and also constituted an excellent training in research methods at the undergraduate level. One significant element in this training was the experience, for the student, of considering the limitations of the chosen approach to data collection and acknowledging them before drawing any conclusions.

Because this project was built around data that was explicitly collected from participants, most of whom were unknown to the student, this was an example where the standard view of ethics agreement might have been appropriate. In practice, and perhaps with hindsight incorrectly, this project was completed and submitted without a formal ethics approval process. The student was anxious to behave in an equitable and transparent manner while working on the project, and as a result, despite having little knowledge of the ESRC framework, in fact followed most of its principles. Notably the student was scrupulous in explaining to participants the background and motivation for the research, and stressing the confidentiality of individual data and that there was no compulsion to take part. This is an instance where following the principles defined by the ESRC makes for a more realistic aim than a more detailed and prescriptive approvals process.

330

Martin Rich

3.3 Recommending a new business model to reduce exploitative practices This is an example where there were slightly unexpected reasons for taking factors around confidentiality and intellectual property into account as the project progressed. The student proposed a new business model in an industry where the current players were perceived as being resistant to change. Some of the motivation for this was related to ethical issues, as the student had expressed a concern that the usual current practices could encourage exploitative actions at some points in the supply chain. But it was also based on recognition that the new business model could become very influential indeed, and could spread rapidly through the sector along the lines discussed by Brynjolfsson et al (2008).

Therefore a major concern for the student was that one of the existing participants in the industry sector might replicate the idea. While at one level this might appear to validate the model, it would also have threatened the model’s originality which was both the prime motivation to pursue the project, and also a reason that the student considered setting up within this business after graduation. This meant that the student needed to take precautions to avoid publicising the idea, and in particular to prevent it from falling into the hands of existing players. At the same time the student was reliant on interviews with people currently working in the sector, and it would have been inappropriate to tell the interviewees in any detail of the aims of the project. It is perhaps putting it too strongly to call this a minor deception, but it could certainly be argued that this works against the aim of telling all participants clearly about the research and the reasons behind it.

Interviewing experts, as this student did, constitutes an approach to data collection that does clearly involve human subjects and for which discussion of ethical approval is appropriate. But in this case the interviews were balanced with detailed accounts of the business sector based on the student’s observation over a period of some months. This was primary data and involved the effects of human agency within the business, but whether it counts as research with human subjects remains debatable.

3.4 Some observations on training and trust In the first section of the paper, there was an observation that the author was encouraged to proceed with this paper, with light touch ethical approval essentially coming down to an exchange of emails, and a brief conversation, on the basis that the author was considered trustworthy within the institution. This is sound enough reasoning when research is being carried out by university staff, but it begs the question of whether students on undergraduate courses particularly should be accorded the same level of trust. Maybe the former assumptions about whether ethics approval is needed for students or staff should be reversed, and approval should be regarded as more important for students, who are less qualified to exercise judgement, than staff.

Or maybe this is a training issue that students need to learn to make ethical judgements as part of any research training that they receive before embarking on projects.

Certainly an argument for treating undergraduate students’ ethical requirements superficially, is that approving one questionnaire within an undergraduate course is not a particularly fruitful use of time for the members of an ethics committee. However the student in such a case, as part of the university community, still has an obligation to act in an ethical manner, and may well ignore that obligation while preoccupied with other aspects of preparing for their project.

Not all professional researchers need to deal with primary data collected from humans. Within the business and management domain, one category of researchers for whom there is no need to collect such data includes those working in finance who mine data that is in the public domain to determine patterns. But such researchers should at least be aware of their limits, and perhaps more sensitised to most that if they are to deal with data from interviews, surveys, or observation they will need to address a set of issues which might be new to them.

4. Discussion and thoughts for the future Reflecting on the vignettes in the previous section, it is significant that the issues covered do map onto the six principles identified by the ESRC. For instance the participant observation case covers issues of impartiality, and of fully informing researchers and participants. For the supermarket survey, these are also issues (and not only ones covering data collection from humans, because in the field of green consumption it is very difficult to

331

Martin Rich write an impartial literature review) as are integrity and quality, and also possibly voluntary participation. The innovative business model raises issues of fully informing participants, and while this is perhaps not the sort of case considered by the ESRC when highlighting ‘covert research’ as an area where exceptions might need to be made to a rigid implementation of the framework.

This is an amorphous area and there are limits in how much students can learn from didactic material, or how much they can resolve uncertainties by working through a standard set of questions about ethical issues. If undergraduate students are to build up enough of an understanding of the connection between their project work and the ethical requirements that they can be trusted to judge whether they are working ethically or not, they need to appreciate the complexity of the issues and find ways to deal with them.

Two possible approaches which could contribute to this would be the use of case studies, such as those touched upon in the vignettes given here, along with discussion of the ethical challenges raised within each, and the inclusion of a coaching element in students’ preparation to work on projects entailing individual research, so that they would be offered some support in dealing with the ethical issues. None of this is a substitute for more straightforward and didactic teaching of research methods, which still needs to address research ethics, but it has the potential to be a useful supplement. It would also be valuable to draw links between this training process and the process for continuing, if appropriate, to a light touch approval for the students’ activities.

Finally it is worth returning to the distinction between the ethical approval processes and the benefits of ethical behavirour.and emphasising that in the context of student projects it is important to promote ethical behaviour as an intrinsically worthwhile endeavour, and not merely as a series of hurdles to be negotiated.

References Association of business schools (2010): ABS ethics guide retrieved 15th February 2013 from http://www.associationofbusinessschools.org/sites/default/files/2010_ethics_guide_agm_version.pdf Brown M, Trevino L and Harrison D (2005): Ethical leadership: a social learning perspective for construct development and testing. Organizational behaviour and human decision processes 97 (2) 117‐134 Brynjolfsson E, McAfee A, Sorrell M and Zhu F (2008): Scale without mass: business process replication and industry dynamics. Harvard Business School working paper Christensen C (1997): The innovator’s dilemma. Harvard Business School Press.

Crawford S and Ostrom E (1995): A grammar of institutions. American political science review 89 (3) 582‐600 Dixon‐Woods M, Angell E, Ashcroft R and Bryman A (2008): Written work: the social functions of research ethics committee letters. Social science and medicine 65 (4) 792‐802 th Economic and social research council (2012): Framework for research ethics retrieved 15 February 2013 from http://www.esrc.ac.uk/_images/Framework‐for‐Research‐Ethics_tcm8‐4586.pdf Haggerty K (2004): Ethics creep: governing social science research in the name of ethics. Qualitative sociology 27 (4) 391‐ 414 Hammersley M and Traianou A (2012): Ethics and educational research. British educational research association online resource retrieved 30th April 2013 from http://www.bera.ac.uk/resources/ethics‐and‐educational‐research Iacono J, Brown A and Holtham C (2009): Research methods – a case study of participant observation. Electronic journal of business research methods 7 (1) 39‐46 retrieved 15th February 2013 from http://www.ejbrm.com/issue/download.html?idArticle=200 Macfarlane B (2009): Researching with integrity: the ethics of academic enquiry. London: Routledge. th Research ethics guidebook (no date) Institutional requirements retrieved 15 February 2013 from http://www.ethicsguidebook.ac.uk/Institutional‐requirements‐77 Stanley L and Wise S (2010) The ESRC’s 2010 framework for research ethics: fit for research purpose? Sociological research online 15 (4) 12

332

Research Methods in Management Academic Programs: From Where we are to Where we Want to go Silvia Silva and Sara Ramos ISCTE‐IUL University Institute of Lisbon; BRU‐IUL Business Research Center, Portugal silvia.silva@iscte.pt

Abstract: The purpose of management research is to develop valid knowledge to support thoughtful practitioners and organizational problem solving in the field (Huff et al., 2006; Saunders et al, 2009). Business and management research has a distinctive focus once management knowledge is sustained by contributes from other disciplines and research should be problem/practice oriented completing a cycle of theory and practice (e.g. Easterby‐Smith et al., 2008; Tranfield & Starkey, 1998). Therefore, in the last years several authors emphasized the relevance of evidence‐base management (e.g., Rousseau, 2006; Saunders et al., 2009). Considering these assumptions research methods courses assumes particular significance in Management Academic Degrees and it is important to characterize it current status. Departing from a curriculum analysis of management programs in different national and international universities, the present work intends to contribute for a reflection on the role of research methods courses in these programs. This analysis has two main purposes: to explore the presence and number of research methods courses and to explore its contents (which methods and techniques are covered). Management academic programs (bachelor and master) where examined, considering top management schools (according to Financial Times 2012 ranking) and national schools (the biggest and with better position on the national ranking). Specifically, 18 programs where covered (6 Portuguese universities, 8 European universities and 4 North American universities). Each program was analyzed focusing research methods courses presence and content.

Analysis covered all the topics that are globally shared and that constitute the Contents of most referenced books.

Preliminary results show a general low presence of methods’ courses on curricula, both in undergraduate and master level.

It was also observed that the methods courses available to students show a clear predominance of quantitative methods over qualitative ones, with a high prevalence of statistics. The presence of methods on management teaching leads us to questioning its role, which acquires particular relevance within the functions of management. Indeed, the underrepresentation of methods is a problem if they are taken not only as useful research tools in strictly academic context but rather as fundamental instruments for decision making, development of critical thinking, understanding relevant scientific information and designing projects to solve real problems in organizations.

Keywords: methods, management, high education, quantitative, qualitative

1. Introduction The general aim of our study is to develop a critical reflection about the presence of research methods courses on academic management programs, connecting this fact with the importance given to methods and the role that they have to managers and to academics on management and business training and education.

It is recognized that we are living a “knowledge economy” era that relies on graduates ability to work effectively across complex and highly diverse contexts (e.g., Blumberg et al., 2008; Jackson, 2009). Due to this graduate skills gap and managing it is a critical point.

1.1 Management education and general relevance of research methods A debate has been raised by Jackson (2009), who questioned if the management education should aim to develop “job‐readiness” or to enhance higher order intellectual and moral skills like criticism and inquiry? For instance, some authors (Candy & Crebert, 1991, cited by Lancaster, 2005) suggested that university education should contribute for developing higher order capabilities (for example, critical and lateral thinking). Other authors have emphasized the impact of management education on business world (e.g., Ghoshal, 2005 cited

by Lancaster), namely a self‐fulfilling prophecies effect that link theory to practice.

It is interesting to see that this kind of debate has not been widely developed for undergraduate education although it has been deepened for Masters and MBAs. Moreover, modern business educators seem to focus more the “how” than the “why” or the “what”. In fact, according to Blumberg et al. (2008, p.4) business research may be defined as a systematic inquiry whose objective is to provide the information that will allow managerial problems to be solved. In this sense, the purpose of management research is to develop valid knowledge to support thoughtful practitioners and organizational problem solving in the field (Huff et al., 2006; Saunders et al, 2009). Business and management research has a distinctive focus once management

333

Silvia Silva and Sara Ramos knowledge is sustained by contributes from other disciplines and research should be problem/practice oriented completing a cycle of theory and practice (e.g. Easterby‐Smith et al., 2008; Tranfield & Starkey, 1998).

Additionally, business research programs recognize that students preparing to manage business, non‐ profit and public organizations – in all functional areas – need training in a disciplined process that will enable them to investigate and solve a research or management dilemma (i.e. any problem or opportunity that requires a management decision) (Blumberg et al., 2008).

1.2 Information complexity and decision‐making IT changed the way we collect and use information in organizations and it is relatively simple to use some software’s for data analysis. But better decisions require better knowledge and using well this kind of tools will be critical for developing useful knowledge for decision making.

To succeed in such an environment, we need to know how to identify high‐quality information and how to recognize relevant (solid and reliable) research to support decisions (Blumberg et al., 2011). Research turns information into knowledge (Hair et al., 2007). This means making better decisions by using knowledge that only research can create. Fortunately, while the decision‐making environment has become more complex, business research tools have become more refined and improvements in information technology have served to simplify the research process (Blumberg et al., 2008).

Considering this context Hair et al. (2007) emphasize that managers do need business research skills. Decision‐ making is a central activity of managers and the amount of information available nowadays is huge.

Information gathering activities are critical as well as adequate analytical tools that can be more or less complex, anyway, this information it is supposed to be used to sustain decisions. As it is stressed by these authors “without knowledge of these business research tools managers and entrepreneurs simply cannot benefit from the intelligence emerging from this expanded database information. Knowledge is power but managers must convert the increasing amount of information into knowledge before its power can be tapped.

Business that are best able to harness this power will be those that are successful in the long run” (pp 16‐17).

In fact, there are several examples about errors that could be prevented by research so it is important that managers understand information collection, evaluation, understanding and use. Currently, there are already some companies whose main activity involves providing research services that help to answer key strategic, tactical and operational business questions.

1.3 Evidence‐based management In the last years several authors emphasized the relevance of evidence‐base management (e.g., Pfeffer & Sutton, 2006; Rousseau, 2006; Saunders et al., 2009; Tranfield et al., 2003). Evidence based practice started in medical care being evidence‐base medicine defined as “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients” (Sackett, cited by Pfeffer & Sutton, 2006,p.2). For instance in Human Resources Management, evidence‐based is defined as a “decision‐making process combining critical thinking with use of the best available scientific evidence and business information” (Rousseau & Barends, 2011, p.211).

These authors emphasize that critical thinking on what works, what does not and why is a critical step for improving practice. These imply actively exploring alternatives, seeking understanding and testing assumptions related to our decisions and activities. Therefore Rousseau & Barends (2011,p.224) establish 4 main activities that are very important: i) the use of the best available scientific evidence from peer‐reviewed sources; ii) systematic gathering of organizational facts, indicators and metrics to better act on the evidence; and iii) practitioner judgment assisted by procedures, practices and frameworks that reduce bias, improve decision quality and create more valid learning over time.

Considering all this, it is clear that making decisions informed by scientific evidence is and will be critical for management practice. This means that managers should be able to read directly scientific evidence to fulfill knowledge gaps in organizations or searching for information to sustain a specific decision. This will require knowing how to formulate an answerable question, search for evidence and critically appraise the evidence found (Rousseau, 2006). Therefore it seems clear that learning research methods will be critical for supporting evidence‐based management.

334

Silvia Silva and Sara Ramos

1.4 Management research and qualitative research methods It is recognized that qualitative methodology play an important role in management and business research both in general and for specific subjects like: general management, leadership, strategy, etc. (e.g., Gummerson, 1991). Although used very frequently (e.g. case studies and ethnographic studies) in the early beginnings of this field in the last decades there was a strong decrease in its use prevailing a dominant quantitative paradigm maybe due to researchers pressured by competitiveness and rankings and it started to loose impact and are underrepresented in the management high top journals.

Fortunately a scientific movement that was firstly initiated by Van Maanen in 1979 (cited by Bluhm et al., 2011) encouraged the development and use of qualitative approaches aiming to change the current situation.

This is now very visible at one of the best journals: Academy of Management Journal. In the last years several editorials were dedicated to this issue like the 2011 “the coming of age for qualitative research: embracing the diversity of qualitative methods”.

Moreover, all textbooks on research methods cover qualitative methods and there was an increase in the number of specific books on qualitative research. However, qualitative methods are still used less frequently and connoted with less “quality” and this is evident considering the type of training that students receive in different graduation levels. It is our experience and expectation that qualitative research methods are generally under‐represented in management degrees.

1.5 Specific goals of the present study These considerations explained above led us to go to the field trying to observe how we are in terms of research methods training on management and business degrees. For this aim we contemplate several programs of different levels of education (bachelor, master and MBA) in different countries and we performed a content analysis of its curricula.

2. Methodology In this study we adopted a curricular search strategy in the official websites of higher education institutions. As the aim was to make an international comparison, six Portuguese institutions were selected, 8 other European institutions and 4 American institutions. For each of these 18 institutions (see table 1), we analysed the curricula of undergraduate and master's degrees in management, a total of 37 programs.

Table 1: Sample of higher educations institutions Portugal FEP‐Porto University

Europe IE Business School, Madrid ‐ Spain

FEUC‐Coimbra University ISCTE‐IUL University Institute of Lisbon Nova University of Lisbon ISEG Technique University of Lisbon Catholic University of Lisbon

HEC Paris ‐ France London Business School ‐ UK INSEAD ‐ France ESAD Business School, Barcelona ‐ Spain St. Gallen University ‐ Switzerland ESSEC Business School ‐ France ESCP Europe

USA Stanford Graduate School of Business Harvard Wharton Pensylvania Columbia

Portuguese institutions were selected considering its dimension on the area of management teaching and also the grade of the last student that was selected to the program. The criteria used on the selection of European and American institutions were its classification on the Top Five of the Financial Times ranking 2011, considering the score obtained in three indicators: European Business School Rankings 2011, Masters in Management 2011 and Global MBA 2011.

Curricula were subjected to a thematic analysis (Braun & Clarke, 2006) in which it was identified the courses that can be retained in the specific area of research methods. This analysis was performed considering regarding the domain characterization of the Academy of Management Research Methods Division stating that it comprises areas such as philosophy of science, research design, qualitative and quantitative research methods and design. They also describe the general aim of this area: to support the process of doing

335

Silvia Silva and Sara Ramos organizational research (qualitative, quantitative, and multi‐faceted) and the major topics that can be included (but are not limited to), for instance: epistemology, theory development, experimental and nonexperimental research design, survey research, ethnography, evaluative research, cross‐cultural and comparative methods, measurement development and evaluation, construct validation, statistical modelling, casual mapping, historical analysis, discourse analysis, textual analysis, scholarly writing and publication, and other methodological topics.

Moreover, it was also considered the topics covered by some of the most quoted management research methods text books (for instance: Saunders, Lewis & Thornill, 2009; Bryman & Bell, 2007 and Cooper & Schindler, 2003).

For analyzing the contents of the programs, a system of categories was created (see table 2).

Table 2: Category system Mandat ory courses

Elective courses

Category name and brief definition General research design, evaluative research, cross‐cultural and comparative method Foundations epistemology, theory development, philosophy of science Specific Data Qualitative collection interviews, focus group, observation, ethnography Quantitative survey research, construct validation, sampling Data Qualitative analysis historical analysis, discourse analysis, textual analysis, content analysis Quantitative statistical analysis, statistical modelling, maths General Foundations Specific Data Qualitative collection Quantitative Data Qualitative analysis Quantitative

Examples Research Methods in Management Sciences Foundations of Qualitative Research ‐‐‐

‐‐‐

‐‐‐

Statistics

Research Methodology

Qualitative Techniques on Sociological Research Sampling and Survey Qualitative data analysis Multivariate Statistics

3. Results and discussion The content analysis of curricula reveals the following results (see table 3).

In a general overview, the results show the slight presence of research methods courses in the majority of the management programs from our sample (52 courses in 37 programs). This low presence seems to decrease even more when we compare undergraduate programs to master and MBA programs (37 courses on undergraduate and 22 on graduate), which can be due to the gradual specification of the contents throughout the different levels of training.

Another result is the clear dominance of quantitative methods in the course’s contents, in particular quantitative data analysis techniques (39 courses and 9 elective courses) namely statistics. In fact, results show a clear predominance of data analysis courses and an absence of data collection topics, both for qualitative and quantitative methods (only one course for qualitative data collection and another one for quantitative data collection, both optional courses).

Concerning qualitative methods, the results confirm that they are really sub represented with only two courses. They are only represented in bachelor degrees and are elective or optional courses that don’t make part of the mandatory curriculum.

336

Silvia Silva and Sara Ramos

Elective courses

Mandatory courses

Table 3: Frequencies table by category, country and degree level

Degree

Category name General courses Foundations courses Specific courses: data collection, qualitative Specific courses: data collection, quantitative Specific courses: data analysis, qualitative Specific courses: data analysis, quantitative General courses Foundations courses Specific courses: data collection, qualitative Specific courses: data collection, quantitative Specific courses: data analysis, qualitative Specific courses: data analysis, quantitative Total

Portugal Bachelor Master n=6 & MBA n=11 0 3 0 0 0 0

Europe Bachelor Master n=5 & MBA n=10 1 0 0 1 0 0

USA Bachelor Master n=1 & MBA n=4 0 0 0 0 0 0

Total n=37 4 1 0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

18

6

5

5

2

3

39

1 0 1

2 0 0

0 0 0

0 0 0

0 0 0

0 0 0

3 0 1

1

0

0

0

0

0

1

1

0

0

0

0

0

1

7

2

0

0

0

0

9

29

13

6

6

2

3

52

We can also observe that fundamental knowledge about methods (e.g.: philosophy of science, epistemology, theory development) is almost not considered in the programs. Exception made by ESADE Barcelona that has the course “Foundations of Qualitative Research” as a mandatory course of its Master in Management Sciences.

Regarding the diversity of methods, we can also say that the frequencies table demonstrate an obvious disequilibrium between the number of courses concerning data collection and data analysis, particularly in the group of quantitative methods. Quantitative methods courses are actually focused on data analysis training much more than on development and selection of instruments and surveys, sampling, variables construction and research design, ethics on surveys and other topics related to data collection.

The results show some differences between Portuguese programs, other European programs and American ones, but we must highlight the fact that the majority of the American programs are situated at MBA level so, some differences can be due to this fact. We found that there are some institutions that really have more research methods courses than the others and that could be an evidence of some difference between institution’s cultures. In Portugal the Coimbra University has the most elevated frequency of research methods courses (2 mandatory courses on bachelor degree, 2 mandatory courses in MBA and 4 more elective courses in bachelor and master degrees) and in Europe is the ESAD Business School of Barcelona‐Spain that offers the programs with more research methods courses (4 mandatory courses in the undergraduate program in management and 2 mandatory courses in the master in Management Sciences).

4. Conclusion From these results, we can stress some general findings:

low presence of research methods courses in the majority of the management programs

even lower presence of research methods in master and MBA programs

prevalence of quantitative methods in the course’s contents

sub representation of qualitative methods courses

clear focus on data analysis than data collection

fundamental knowledge is almost not considered in the programs

337

Silvia Silva and Sara Ramos These findings reflect some arguments that can enrich our discussion. Regarding the lower presence of research methods in master and MBA programs we can question how to assure that students have had this kind of training before (namely on undergraduate education) and have developed the required competences to understand literature and to use methods in the field?

Concerning the prevalence of quantitative methods and the emphasis on data analysis we think is really important to better explore what are the most used and useful methods in management practice.

The absence of fundamental topics on methods in most of the programs raises the question of how to assure that managers can develop or select the adequate methods or research design for studying real problems in their organizations, how to make them aware of the limits and the implications of different methods; how to develop critic skills to make them question the methods and instruments that are used in organizations)

Management requires scientific based knowledge and research methods are a powerful tool to fulfill the gap between theory and practice. The decreasing number of research methods courses offered on Management Academic Degrees programs demands a large reflection about the impact on student’s training and future practice.

Considering that the current times are more demanding, that decision making is a critical competence for managers that evidence based management is critical for organizations efficiency and efficacy, and richness of diverse research methods. It seems urgent to start a wider academic debate to think about where we want to go?

References Bansal, P. and Corley, K. (2011) The coming of age for qualitative research: Embracing the diversity of qualitative methods.

Academy of Management Journal, Vol. 54, No 2, pp 233‐237.

Bluhm, D., Harman, W., Lee, T. and Mitchell, T. (2011) Qualitative research in management: A decade of progress. Journal of Management Studies, Vol. 48, pp 1866‐1891.

Blumberg, B., Cooper, D.R. and Schindler, P.S. (2008) Business Research Methods. McGraw‐Hill Education, Maidenhead.

Braun, V. and Clarke, V. (2006) Using thematic analysis in psychology. Qualitative Research in Psychology, Vol. 3, pp 77‐101.

Bryman, A. and Bell, E. (2007) Business Research Methods. Oxford University Press, Oxford.

Cooper, D. R. and Schindler, P. S. (2003) Business Research Methods. McGraw Hill International Edition.

Easterby‐Smith, M., Thorpe, R. Jackson, P. and Lowe, A. (2008) Management Research (3rd edn). Sage, London.

Gummerson, E. (1991) Qualitative Methods in Management Research. Sage, Newbury Park, CA. Hair J.F.R., Money, A.H., Samuel, P., & Page, M. (2007) Research Methods for Business . John Wiley & Sons, Ltd, England.

Lancaster, G. (2005). Research Methods in Management. A Concise Introduction to Research in Management and Business Consultancy, Elsevier.

Pfeffer, J. & Sutton, R,I. (2006). Hard Facts, Dangerous Half‐Truths, and Total Nonsense: Profiting from Evidence‐Based Management. Harvard Business School Press, Harvard.

Rousseau, D.M. & Barends, E.G.R., (2011). Becoming an evidence‐based HR practitioner, Human Resource Management Journal, Vol. 21, No. 3, pp 221‐235.

Rousseau, D.M. (2006). Is there such a thing as evidence‐based management? Academy of Management Review, Vol. 31, pp 256‐269.

Saunders, M., Lewis, P., & Thornill, A. (2009). Research methods for business students. Prentice Hall, Harlow.

Tranfield, D., Denyer, D. and Smart, P. (2003). Developing an evidence‐informed approach to management knowledge by means of systematic review, British Journal of Management, Vol. 14, No. 3.

Tranfield, D., Starkey, K. (1998). The nature, social organisation and promotion of management research: Towards policy.

British Journal of Management, Vol. 9, No. 4, pp 341‐353.

338

Co‐Word Mapping Using Correspondence Analysis Anthony Stacey Graduate School of Business Administration, University of the Witwatersrand, Johannesburg, South Africa Anthony.Stacey@wits.ac.za

Abstract: Co‐word analysis is a method of identifying and illustrating thematic content of textual data. Correspondence analysis is an exploratory and descriptive quantitative multivariate technique for analysing and illustrating associations or correspondence between categorical variables, which is traditionally based on contingency tables or cross‐tabulations of data. There is no definitive method for generating co‐word maps and in this paper it will be proposed that Correspondence Analysis can be used to identify common themes in textual data by mapping word associations and creating co‐word maps.

Using responses to open‐ended questions from surveys with large sample sizes, it will be shown that text can be suitably filtered to eliminate common words, plurals, punctuation, spelling errors, and the like. A cross‐tabulation can be created of the frequencies of use of keywords against associated or qualifying keywords that survey participants have juxtaposed in their responses. Ideally, although not a prerequisite, the rows and the columns are grammatically distinct, with for example rows comprising nouns and columns comprising verbs and adjectives. Conventional Correspondence Analysis can be applied to the resultant contingency table of keywords. Correspondence Analysis produces bi‐dimensional plots simultaneously positioning both the rows (e.g. nouns) and the columns (e.g. adjectives and / or verbs) of the contingency table on the same set of axes. Therefore all keywords and associated keywords can be plotted on what is in essence a co‐ word map. Each of the dimensions identified in the Correspondence Analysis is a potential common theme in the textual data, defined by the keywords. The advantage of this method is that it is suitable for analysing open‐ended responses from large samples using readily available statistical software. Unlike other co‐word analytical techniques, this technique can identify orthogonal thematic content.

Keywords: textual data analysis, correspondence analysis, co‐word analysis, co‐word mapping

1. Introduction Content analysis is arguably the most widely used qualitative research method (Zhang & Wildemuth, 2009).

One of the challenges of the method is the presentation of the results in an efficient manner such that they may be unambiguously and rapidly assimilated.

Primary and secondary textual data is available from many sources, particularly with the easy accessibility of electronic sources. Typically however, the data is voluminous and the analysis of the data can be resource intensive and time consuming. Fortunately, a plethora of computer software exists for content and qualitative data analysis (K. G. Hill, 2008; Romppel, 2012).

In this paper, it will be illustrated how generic office and statistical software can easily be used to analyse textual data, potentially making textual data analysis accessible to a wider group of researchers. Primary data from a large survey which included a question with an open‐ended response format will be analysed.

2. Theoretical background 3. Co‐word analysis In their seminal work on co‐word analysis Callon, Courtial, Turner, and Bauin (1983) proposed the method to identify and visualise networks of relationships between research problems “be they scientific, technical, political, economical or other” (Callon et al., 1983, p. 193). He (1999) describes co‐word analysis as a content analysis technique that uses the frequencies of occurrence of patterns of words or phrases in textual data, in order to identify the relationships between ideas. Similarly co‐word analysis has also been described as a class of techniques that analyses co‐occurrences of words or groups of words in textual data (Kopcsa & Schiebel, 1998). A comparable technique involving the extraction of “maximal frequent sequences” from a document collection is described by Ahonen (1999, p. 160) as a means of knowledge discovery. Co‐word analysis has been used extensively in meta‐analyses of research literature, data mining, bibliometrics, artificial intelligence, and qualitative trend analysis.

Typically the output of co‐word analysis is a co‐word map illustrating the relationships between keywords. It is visualisation technique that position keywords with the strongest linkages together and thereby forms clusters

339

Anthony Stacey of related words (Engelsman & van Raan, 1994). Semantic mapping (Leydesdorff & Welbers, 2011) is a similar visualisation technique for presenting the results of latent semantic analysis, which is a technique for identifying latent dimensions in a word‐document matrix.

3.1 Correspondence analysis The history of Correspondence Analysis is somewhat convoluted (M. O. Hill, 1974) but it is now widely used in a variety of disciplines and applications. Correspondence Analysis is a special case of analysing the association between two datasets, and is “a technique for displaying the rows and columns of a data matrix (primarily, a two‐way contingency table) as points in dual low‐dimensional vector spaces” (Greenacre, 1984, p. 54). The technique illustrates the fundamental relationships between two sets of categorical, indicator or dummy variables. The input to the analysis is generally a two‐way cross tabulation or contingency table, while the principal output from the analysis is a two dimensional chart, known as a Correspondence Analysis bi‐plot (Greenacre, 2007).

In the Correspondence Analysis bi‐plot, all rows of the contingency table (representing a set of variables in the original dataset) and all columns of the contingency table (representing a different set of variables in the original dataset) can be displayed on the same axes. In some ways Correspondence Analysis is a technique analogous to principal component analysis where the input data is categorical, typically in the form of frequency counts, rather than numerical.

Two properties of Correspondence Analysis suggest that it could form the basis of co‐word mapping. Firstly, the input format of the data for Correspondence Analysis is a cross tabulation or contingency table; the occurrences of adjacent and semi‐adjacent keywords in textual data can be counted and summarised in a cross tabulation of frequencies. Secondly, a co‐word map comprises a chart in which keywords are positioned on a common set of axes in such a way that the relative location of the keywords represents some aspect of the relationship between them, which is a defining property of the Correspondence Analysis bi‐plot.

What follows is a description of the method and interpretation of the results of analysing a textual dataset comprising the responses of survey participants to the open‐ended question: “In your view, what could the company do to improve its reputation?” The company which was the subject of the survey is a provider of products and services in the information and communication technology industry. Interviews were carried out telephonically, and participants comprised a variety of stakeholders, including business customers, residential customers, employees, and potential customers. The overall sample size was approximately 2 600.

4. Co‐word mapping methodology The novel method of co‐word mapping proposed here involves three phases of analysis: the first phase would typically be carried out using conventional word processing software (such as Microsoft Word), the second phase would conceivably be carried out using conventional spreadsheet software (such as Microsoft Excel) and the third phase would involve applying statistical analysis software (in this instance SPSS). It is important to recognise that in common with all qualitative research techniques, this method is not carried out in a vacuum.

It will be shown that specific choices and decisions require some familiarity with and knowledge of the research context.

4.1 Pre‐processing of textual data The format of the raw textual data is not crucial to the analysis, provided it can be imported or copied into a text processing package. The initial pre‐processing of the textual dataset is primarily a filtering and cleaning process in order to prepare the dataset for numeric processing ambiguities and redundancies.

The raw data for co‐word analysis are keywords in the textual dataset that occur adjacent or close (semi‐ adjacent) to one another. What defines a keyword depends on the context and will become evident later in the analysis.

Where there are numerous records in the textual dataset, for example successive survey respondents or different documents, it is necessary to maintain the separation of the records in order to avoid spurious counts

340

Anthony Stacey of adjacent or semi‐adjacent words across record boundaries. Therefore the first step in the processing of the data is to ensure that there is a unique character or character set that separates each record.

In English grammar, nouns represent things or ideas. However, sometimes a thing or an idea comprises more than a single word, in which case the consecutive words are referred to as a compound noun (Downing, 1977).

In this method it is necessary to identify those consecutive words that form compound nouns. Table 1 illustrates the formation of compound nouns based on the wording of the survey that constitutes the context of the textual dataset. In order to treat compound nouns as a single entity, the individual words therefore have to be joined using a suitable character, such as a nonbreaking space or underscore character. Henceforth, the compound nouns are treated as a single unit in the subsequent data analysis.

Table 1: Examples of formation of compound nouns Nouns

Compound Noun

Cable

theft

Cable theft

Call

centre

Call centre

Call

centres

Call centres

Call

outs

Call outs

Call

rates

Call rates

Cell

phone

Cell phone

Cell

phones

Cell phones

Customer

service

Customer service

Customer

services

Customer services

Good

work

Good work

Good

working

Good working

Response

time

Response time

Response

times

Response times

Service

delivery

Service delivery

Staff

training

Staff training

Top

management

Top management

Turnaround

time

Turnaround time

Turnaround

times

Turnaround times

The final processes in the preparation of the textual data are:

Removal of all punctuation marks and non‐textual data (e.g. full‐stops, commas, hyphens, colons, semi‐ colons, brackets, numbers, and exclamation marks).

Changing the case of words so that all have the same capitalisation (either lowercase, uppercase or capitalised).

Optionally changing plural nouns to singular nouns.

Changing spaces between words into paragraph marks or line break characters to ensure that each word is a separate record in the textual data.

The cleaned and filtered textual data is then exported using a file format that is appropriate for importing into the spreadsheet software.

4.2 Compilation of data table The first operation that is carried out towards the compilation of the data table is a straightforward word count. Clearly there are a plethora of words that are not substantive to the analysis, and the grammatical parts of speech given in Table 2 are a convenient means of filtering those words that are meaningful for further analysis from those that are not. Generally words that are used as pronouns, prepositions, conjunctions or interjections can be omitted from the co‐word analysis.

341

Anthony Stacey Table 2: Functions of parts of speech in co‐word analysis Part of speech Noun Verb Adjective Adverb Pronoun Preposition Conjunction Interjection

Function A thing or an idea An action word or something being done Modifies or describes the noun Modifies or describes the verb or adjective Omitted from analysis Omitted from analysis Omitted from analysis Omitted from analysis

Therefore the second process is the classification of each of the frequently used words into one of the grammatical parts of speech. At this stage a choice is made of which keywords are substantive to the analysis with a focus of attention being verbs, adjectives and nouns (including compound nouns). Clearly some knowledge of the context of the textual data is necessary to reliably identify those keywords. For example, the word “Service” could be used as either a verb or a noun, but even a cursory scan of the raw textual data reveals that in this context, the word is used almost exclusively as a noun.

The cardinal process in the compilation of the data table is counting co‐occurrences of the key verbs and adjectives that are adjacent or semi‐adjacent to key nouns and compound nouns in the textual data. For the purposes of the count, semi‐adjacent refers to co‐occurrence of the keywords within a specified span of words within the textual data. In this instance, keywords were regarded as semi‐adjacent if they occurred within three words of one another. Table 3 gives examples of the use of co‐word pairs to demonstrate that the semantic association between the keywords is essentially the same, whether the keywords are adjacent or semi‐adjacent. The cross tabulation of co‐occurrences of key‐words with rows corresponding to verbs and adjectives, and columns corresponding to nouns and compound nouns, forms the co‐occurrence frequency count matrix.

Table 3: Examples of adjacent and semi‐adjacent use of co‐word pairs Co‐word pair Better & Service Train & Staff New & Products

Example of adjacent use “Concentrate more on giving better service to customers.”

“Train staff on how to deal with customers.”

“Be quicker to take new products to the market.”

Example of semi‐adjacent use “Provide better after sales service.”

“They must train their call centre staff.”

Offer new and exciting products to attract more clients.”

The data in the resultant co‐occurrence frequency count matrix is somewhat sparse as some of the rows and columns have few non‐zero elements. A pragmatic approach to militate against the sparse data is to aggregate those rows and columns corresponding to unambiguous and unmistakeable synonyms. This was the rationale for aggregating the following columns of data:

“Employees” aggregated into “Staff”

“Clients” aggregated into “Customers”

“Call centres” aggregated into “Call centre”

“Call rates” aggregated into “Rates”

The final co‐occurrence frequency count matrix on which the analysis is based is given in Table 4.

4.3 Correspondence analysis The co‐occurrence frequency count matrix constitutes the input data for the Correspondence Analysis. For the purposes of entering the data into SPSS, the co‐occurrence frequency count matrix is transformed to a database with three variables: the column variable (corresponding to the nouns and compound nouns), the row variable (corresponding to the verbs and adjectives) and a count (corresponding to the co‐occurrence frequencies).

342

Anthony Stacey Table 4: Co‐occurrence frequency count matrix

Verbs and Adjectives

Communication

Customer

Customer service

Customers

People

Products

Rates

Reputation

Response

Service

Service delivery

Services

Staff

Technology

Training

Nouns and Compound Nouns

Call centre

Better

9

22

3

12

18

1

0

7

0

2

83

18

18

10

2

10

Faster

0

0

0

1

2

0

1

0

0

7

6

0

4

0

0

0

Focus

0

0

2

3

10

0

0

0

0

0

4

4

0

1

0

1

Give

0

0

1

2

16

0

0

3

0

1

21

0

6

2

0

3

Good

1

1

0

15

10

1

7

0

24

1

64

1

10

1

0

1

Improve

27

12

6

23

8

0

2

1

6

2

54

13

14

11

2

0

Keep

0

0

0

0

6

0

5

1

0

0

8

0

1

2

1

0

Lower

0

0

0

0

0

0

0

28

0

0

0

0

0

1

0

0

More

3

3

24

1

24

14

9

0

0

0

9

7

5

28

3

11

New

0

0

0

0

7

0

30

0

0

0

3

0

3

1

17

1

Offer

0

0

1

2

5

2

13

4

0

0

19

0

5

0

1

0

Provide

0

0

0

11

1

0

0

0

0

0

24

2

7

0

0

2

Quicker

6

0

0

2

2

0

0

0

0

21

14

2

0

1

0

0

Reliable

1

0

0

0

0

0

2

0

0

0

10

0

8

0

0

0

Service

11

3

3

0

24

5

7

3

1

3

2

0

0

6

0

1

Train

5

0

1

0

0

1

2

0

0

0

0

0

0

46

0

0

5. Results The scree plot (Cattell, 1966) presented in Figure 1, which gives the proportion of inertia (i.e. degree of variation) accounted for by successive dimensions, and the cumulative inertia are used to determine how many dimensions there should be in the Correspondence Analysis model. A two dimensional Correspondence Analysis model is often preferred as the results of the analysis can then be presented in a single two dimensional bi‐plot. However, in this instance, the degree of variation (cumulative proportion of inertia) accounted for by the first two dimensions is only 45.4%, less than the 60% often regarded as the minimum satisfactory proportion (Hair, Black, Babin, Anderson, & Tatham, 2006). Two dimensions are therefore deemed insufficient to adequately present the results of the analysis, and a four dimensional Correspondence Analysis model has been selected.

For obvious practical reasons, only two dimensions of the Correspondence Analysis model can be presented at one time. Therefore, for a four dimensional model there are six permutations of two dimensions and therefore six possible bi‐plots that can be presented. In this instance, the two bi‐plots shown in Figure 2 and Figure 3 are considered adequate to interpret the results of the analysis as each of the four dimensions is presented.

The co‐word bi‐plot positions all of the keywords (nouns, compound nouns, verbs and adjectives) on a common set of orthogonal or perpendicular axes. In Figure 2 and Figure 3 the row points (representing verbs and adjectives) are charted using a cross and the column points (representing nouns and compound nouns) are charted using circles. For clarity, labels are given only to those variables that are heavily loaded onto and contributing substantially to the respective dimensions.

343

Anthony Stacey

Figure 1: Scree plot The co‐word bi‐plot for dimensions 1 and 2 is given in Figure 2. The key themes emerging from the textual data can be identified by the position of the verbs and adjectives in relation to the nouns and compound nouns.

Therefore, based on the open‐ended responses in the survey, key themes regarding how the company can improve its reputation that emerge from the first two dimensions of the Correspondence Analysis model are:

Lower rates,

Faster quicker service / Faster quicker customer service, and

Train staff / New staff / More staff.

At this juncture it is relevant to highlight that the issues of “Lower rates” is orthogonal to, and therefore completely independent of the other two reputational issues identified here. The points corresponding to “Lower” and “Rates” are strongly related to one another and not to other keywords. Therefore they occur relatively close to one another on the co‐word bi‐plot and relatively distant from all other points. In Figure 2 the points corresponding to “Service” and “Customer service” are similarly located on the co‐word bi‐plot. This is an indication that similar verbs or adjectives occur adjacent to or semi‐adjacent to both these two nouns.

Consistent with the objectives of the co‐word mapping, they are similarly positioned on the co‐word bi‐plot.

On the same figure the points corresponding to the adjectives “Quicker” and “Faster” as closely positioned, indicating that similar nouns occur adjacent to or semi‐adjacent to these adjective. This is entirely plausible and logical given that the two words are essentially synonymous. It is noted that the keywords “Quicker”, “Faster”, “Customer service” and “Service” are antithetical in the second dimension to the words “More”, “New”, “Staff” and “Train”. This positioning suggests that these groups of keywords are not mutually independent, but rather are incongruous. In other words, the adjectives “Quicker” and “Faster” were generally not found adjacent to the noun “Staff”, and the similarly the verb and adjectives “Train”, “More” and “New” generally did not arise adjacent to the nouns “Customer service” and “Service”. Again, this positioning of the keywords is consistent with the objectives of the co‐word mapping.

Figure 2: Co‐word bi‐plot of dimensions 1 and 2

344

Anthony Stacey The co‐word bi‐plot for dimensions 3 and 4 is given in Figure 3. Key themes regarding how the company can improve its reputation that emerge from the third and fourth dimensions of the Correspondence Analysis model are:

Train staff,

New technology / New products, and

Quicker response / Faster response.

Here it is noted that the verb “Train” and the adjective “New” were similarly positioned in the first dimension Correspondence Analysis. However, the different co‐word associations of the two words are manifest in this third dimension, with “New” now associated with the nouns “Technology” and “Products” , while “Train” remains associated with and in close proximity to the nouns “Staff”. It is also noteworthy that in Figure 3 the adjectives “Quicker” and “Faster” are positioned close to one another, as they were in Figure 2. However in the fourth dimension which is at independent of the second dimension, the adjectives are associated with the noun “Response”. This suggests that on the basis of the survey, the construct of “Response” is quite distinct from the construct defined by “Service” and “Customer service”. This is consistent with the multidimensionality of customer service as found by, for example, Parasuraman, Zeithaml, and Berry (1988).

Figure 3: Co‐word bi‐plot of dimensions 3 and 4

6. Discussion A three stage method involving screening of the textual data, compilation of a co‐occurrence frequency matrix, which was then subjected to Correspondence Analysis, has been described above. Applying the method to a textual dataset comprising responses to the question “In your view, what could the company do to improve its reputation?” the following issues were extracted:

Rates: lower rates;

Service: faster, quicker service; faster, quicker customer service;

Staff: train staff; new staff; more staff;

Technology: new technology; new products, and

Responsiveness: quicker response; faster response.

These issues were extracted by interpreting the co‐word bi‐plots which were the output of the final stage of the analysis in which all of the keywords were positioned on a set of mutually independent (i.e. orthogonal) axes. This orthogonality of the axes is both beneficial and disadvantageous. The advantage is that the method has superior discriminant power; that is, the ability to identify and differentiate between distinct thematic content in the textual data. In other words, the method minimises any ambiguity and vagueness, and enhances the specificity of the emergent themes. An illustration of this is the distinction between the constructs of “Response” and “Service” / “Customer service” in the above example. The disadvantage of orthogonality is that, as in the example given in this paper, it may require more than two dimensions to express all the thematic content adequately. Therefore this method does not necessarily produce a parsimonious representation of the thematic content of the textual data. While this is a drawback when

345

Anthony Stacey reporting the results in a hard copy format, creative graphical techniques including animation can be used when presenting the results interactively.

There are some parameters implicit in the method of co‐word mapping using Correspondence Analysis, and there is the opportunity to optimise the method by varying these parameters. The following are examples of varying the method parameters:

In this example, the frequencies of co‐occurrence of keywords have only included those responses in which the verb or adjective precedes the noun or compound noun. The rationale for this is that the sentence structure used by respondents generally placed the keywords in this order, with the nouns or compound nouns being the object of the sentence, rather than the subject of the sentence. Depending on the source of the textual data, it may be appropriate to include in the frequency counts those occurrences in which the nouns or compound nouns precede the verbs and adjectives.

The span within which key‐words are regarded as semi‐adjacent and therefore included in frequency counts can be varied. More complex sentence structures and richer textual data might suggest that the span of semi‐adjacent words should be greater than three that was used in this analysis.

There is an argument to suggest that the association between adjacent words is somehow greater than that between semi‐adjacent words. That being the case, it would be appropriate to weight the counts of adjacent co‐occurrences greater than the counts of semi‐adjacent occurrences. The results of the analysis of this textual dataset were not notably different when weighting adjacent and semi‐adjacent co‐ occurrences differently.

7. Conclusion There is no definitive method for co‐word mapping and the approach presented here has effectively distilled thematic content from the open‐ended responses to a specific survey question. The appeal of the method is enhanced by the use of conventional desktop office and statistical software. While the method has been used here to analyse open‐ended responses to a single question in a substantial study of corporate reputation, it is recommended that the method be validated by analysing other types of textual data. It is intended to use this method to analyse transcripts of in‐depth interviews and corpuses of literature across various management disciplines.

References Ahonen, H. (1999). Knowledge Discovery in Documents by Extracting Frequent Word Sequences. Library Trends, 18(1), 160‐ 181.

Callon, M., Courtial, J. P., Turner, W. A., & Bauin, S. (1983). From translations to problematic networks: An introduction to co‐word analysis. Social Science Information, 22(2), 191‐235.

Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245‐276.

Downing, P. (1977). On the Creation and Use of English Compound Nouns. Language 53(4), 810‐842.

Engelsman, E. C., & van Raan, A. F. J. (1994). A patent‐based cartography of technology. Research Policy, 23(1), 1‐26.

Greenacre, M. (1984). Theory and Applications of Correspondence Analysis. London: Academic Press.

Greenacre, M. (2007). Correspondence Analysis in Practice (2nd ed.). Boca Raton: Chapman & Hall/CRC. Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2006). Multivariate Data Analysis. Upper Saddle River: Pearson Prentice‐Hall.

He, Q. (1999). Knowledge Discovery through Co‐Word Analysis. Library Trends, 48(1), 133‐159.

Hill, K. G. (2008, 2008‐12‐02). Software for Content Analysis: Links to external sites Retrieved 2013‐01‐28, from http://courses.washington.edu/socw580/contentsoftware.shtml Hill, M. O. (1974). Correspondence Analysis: A Neglected Multivariate Method. Journal of the Royal Statistical Society.

Series C (Applied Statistics), 23(3), 340‐354.

Kopcsa, A., & Schiebel, E. (1998). Science and technology mapping: a new iteration model for representing multidimensional relationships. Journal of the American Society for Information Science, 49(1), 7‐17.

Leydesdorff, L., & Welbers, K. (2011). The semantic mapping of words and co‐words in contexts. Journal of Informetrics, 5(3), 469‐475.

Parasuraman, A., Zeithaml, V. A., & Berry, L. L. (1988). Servqual. Journal of retailing, 64(1), 12‐40.

Romppel, M. (2012). Resources related to content analysis and text analysis Retrieved 2013‐01‐28, from http://www.content‐analysis.de/software/qualitative‐analysis Zhang, Y., & Wildemuth, B. M. (2009). Qualitative analysis of content. Applications of social research methods to questions in information and library science, 308‐319.

346

Using Knowledge Cartography Techniques for Conducting a Realist Synthesis on Collaborative Teaching Leonel Tractenberg Faculty of Management and Finance, UERJ, Brazil leoneltractenberg@gmail.com

Abstract: The astonishing growth of academic publications worldwide and the increasing access to online bibliographic databases of recent decades represent a challenge to researchers and professionals concerning the integration of findings on their area of expertise. As qualitative management studies multiply, the importance of using new methods of qualitative research synthesis increases. New methods of qualitative synthesis have been recently developed, such as qualitative meta‐synthesis and realist synthesis (or realist review). Yet, these qualitative syntheses methods remain relatively unknown by management researchers. Objectives ‐ The purpose of this paper is to briefly present the realist synthesis method, and to show how knowledge cartography techniques and tools can be used in realist synthesis in order facilitate the process of theory building. Design/methodology ‐ Underpinnings and method of realist synthesis are described, followed by a discussion on knowledge cartography and its applications to qualitative research. A realist synthesis on collaborative teaching serves as an illustration of how knowledge mapping tools that can facilitate the review process. Findings – Cartographic techniques and tools can facilitate organizing and analysing studies, arranging and re‐ arranging concepts and, thus, can help designing theoretical frameworks for the social intervention in focus.

Originality/value – The rapid growth of qualitative research demands new and improved efforts towards synthesising literature. This paper can contribute to the instrumentalization of the realist review method, and to disseminate this method of research synthesis in Management Research.

Keywords: qualitative research synthesis, realist synthesis, knowledge cartography, knowledge maps

1. Introduction Science is a cooperative and cumulative endeavour. Knowledge building in any field requires trustworthy accounts of past research. But making sense of a growing mass of research evidence has become progressively harder. The astonishing growth of academic publications worldwide and the increasing access to online bibliographic databases of recent decades represent a challenge to researchers and professionals concerning the integration of findings on their area of expertise (Cooper 1998).

Research synthesis can be defined as the systematic effort of finding, selecting, organizing, evaluating and integrating research findings on a given topic. Until the 1960s, most syntheses were written on an essayistic or narrative fashion (Cooper et al. 2009). Those traditional ‘narrative’ reviews frequently suffered from researcher bias and lack of rigour, and, in many cases, were not considered as genuine pieces of scientific investigation. On the 1970s, standardized protocols of literature review were developed in order to provide increased reliability and consistency in synthesising the literature. The 1980s and 1990s saw the proliferation of systematic reviews and meta‐analysis, especially in medicine and in social sciences (Crag 1996, Littel et al.

2008).

Despite this growth, the increasing dissemination and refinement of quantitative methods of research synthesis contrasts enormously with the little attention given to qualitative research synthesis (Sandelowski et al. 1997:365). As a result, findings of qualitative studies end up getting scattered or lost, and end up having little impact on professional practices and policies (Matheus 2009). To address this problem, new methods of qualitative synthesis have been recently developed, such as qualitative meta‐synthesis (Sandelowski et al.

1997) and realist synthesis (or realist review) (Pawson et al. 2004). Still, most of these syntheses have been focusing on health interventions.

As qualitative management studies multiply, the importance of using these new methods to provide insights and guidance for base for evidence‐based policies and practices increases. The purpose of this paper is to contribute to the dissemination and development of these methods in social science, by discussing how knowledge mapping techniques and tools can be used in the realist review process in order facilitate theory building.

347

Leonel Tractenberg

2. Background 2.1 Realist synthesis A social intervention is considered to be of increasing complexity as it involves a growing number of processes or mechanisms which are intertwined with contextual variables (environmental, historical, political, social, economic, institutional, etc.). Realist review (or realist synthesis) is a kind of qualitative research synthesis with the interpretive and explanatory purpose of understanding the processes or mechanisms underlying complex social interventions. It aims to provide an explanation of how and why an intervention works (or not) in a given context (Pawson et al. 2004). It seeks to identify principles, concepts, models, explicit or implicit theories allegedly responsible for the operation of a particular intervention; and look for evidence in a systematic way to test and refine the understanding of these elements. Instead of seeking for universal principles or a final verdict about the intervention (effective vs. ineffective), realist synthesis recognizes the fact that the success of an intervention depends on complex interactions, negotiations and conflicts between individuals, relationships, institutions and infrastructures through whom and for whom the intervention is directed.

Therefore, a realist review seeks to answer is: what works, for whom, in what circumstances and issues, and how (Pawson 2006).

The ultimate goal is to provide input for the design of new interventions, programs or policies. The review seeks to synthesize findings in terms of a theory of intervention built from a “mosaic” of evidence‐based principles and concepts extracted from selected studies. Final conclusions should be capable of expressing the complexity and dynamicity of the intervention, program or policy under study, and recommendations should be given with great care and being transparent about the review limitations.

The review method follows a sequence of six main steps, but instead of a fixed linear sequence, it is an iterative and recursive process, allowing revisions and refinements of previous steps. A generic template for realist reviews proposed by Pawson et al. (2004) is shown in figure 1. This template should not be interpreted as a rigid scheme. What matters is to keep the logic and transparency in the process of discovery and theory building and refinement (2006:102).

The reviewer begins setting goals and questions that will guide the review. If the goal is to evaluate the integrity of a theory of intervention, the reviewer will seek to identify weaknesses and barriers in the chain of processes of the intervention in a story of successive similar interventions. If the aim is to evaluate rival theories of intervention, the reviewer will seek to synthesize the evidence in favour of one or another theory and explanation of the mechanisms, benefits and shortcomings of each. If the objective is to analyze how the same theory of intervention is implemented in different contexts, the reviewer will try to identify factors that best explain this or that situation. If the target comparison between official expectations or initial and current practices, the review will seek to identify the elements of reality that are not being neglected or are in contradiction with the official expectations and beliefs. This process of goals and questions setting is open to further refinement.

The search for references is similar to the traditional systematic reviews: it is necessary to define the sources, search expressions used, the selection/exclusion criteria, and quality standards for selection of studies.

However, there are some points of differentiation: while traditional systematic reviews focus on an increasingly limited number of references, search strategies in realistic revisions may involve several cycles filtering and funnelling (narrowing down the scope), scope expansions, new bottlenecks and repeated searches for "convenience samples" (purposive sampling) of studies in order to investigate specific aspects of the intervention in focus. That is, the universe of searchable references may suffer expansions and contractions along the route. Distinct niches literature can be scrutinized. These strategies are applied iteratively and recursively insofar as understanding the phenomenon expands until the reviewer identify a saturation point, no longer adds new information to the structure.

Experimental studies, case studies, ethnographic research, theoretical discussions and other types of research can be included. Different from systematic reviews, the criteria of selection and quality evaluation are not necessarily the same to all studies. They are established ad hoc. Instead of establishing an absolute filter to separate all usable studies, different filters are used to check the partial contributions of each study to the review questions. In later stages, partial assessments can be made on the relevant information to be extracted.

348

Leonel Tractenberg On the other hand, some studies selected for in‐depth analysis can be assessed more comprehensively. This “just‐in‐time assessment” makes the process considerably more complex and time‐consuming. Pawson (2006) warns about the limitations of this strategy in terms of the amount of studies that the reviewer is able to encompass. Given the inherent complexity of interventions and their variability, it is necessary that the reviewer prioritize certain contexts and certain sources of information in order to limit the amount of data to a manageable volume. All these decisions should be clearly justified.

The elements that compose theories of intervention are not always explicit. They often appear in the form of conjectures, expectations, explanations and rationalizations about how the intervention works. They can be extracted from several parts of the study (background, methodology, discussion etc.) and not only from the conclusions, as it is done in other types of research synthesis. Realist synthesis is an interpretative approach, which considers each study as a particular “narrative”.

Figure 1: The process of a realist review (Pawson et al. 2004:29)

2.2 Knowledge cartography Knowledge cartography is the art, craft and science of mapping cognitive domains. Knowledge maps are graphical representations of knowledge structures (concepts and relations between them), and can help organizing, synthesizing and communicating ideas developing thinking skills and improve learning (Okada 2007, 2008).

349

Leonel Tractenberg There are different types of knowledge (also called cognitive) maps, each one better suit for specific applications: mindmaps are useful for brainstorming ideas, problem‐solving and creative writing (Buzan 1991); concept maps are suitable to represent knowledge domains or cognitive structures, by making concept relations and hierarchies explicit (Novak and Cañas 2006); argument maps are ment to visually represent the components of an argument, such as main contention, principles, premises, co‐premises, objections and rebuttals, that can be applied to decision‐making, legal and political analysis (Okada 2008); hypermedia maps and webmaps are graphical and hypertextual representations of knowledge domains which incorporate different digital media files (texts, images, audios, videos etc.), software and/or weblinks (Okada 2007, 2008, Tractenberg et al. 2009). While mindmaps usually have a radial‐branching structure, concept maps have a more strict representation in terms of hierarchy, putting higher‐order concepts on the top, over subordinated concepts, and links between concepts must identify the nature of their relation. Argument maps should clearly identify issues, arguments or questions, pros and cons, premises, counterarguments, and conclusions.

Hypermedia and webmaps have a free hypertextual structure that can hold all kinds of maps and enrich them with all kinds of digital media.

Knowledge mapping can be applied to qualitative research. Okada (2008) identified seven moments of an investigation process in which maps can be used: 1) problematizing research topics and defining a research project (research project map); 2) organizing a literature review (map of references); 3) analysing and interpreting studies (map of readings); 4) building a conceptual framework (maps of concepts); 5) organizing and analysing fieldwork data (notes, documents, observations etc.) (Map of field research); 6) systematizing and writing research reports (map of writing); and 7) managing the whole investigation process (map of research process). From these, maps 2, 3 and 4 can be of special interest to researchers conducting literature reviews and qualitative research synthesis. As put by Vasconcellos, Mapping ideas, arguments and concepts of a body of literature are important techniques for literature reviews. Mapping allows the researcher to find his way into a set of the existing literature, identify key studies and key concepts, and, at the same time, build up a framework of relationships between each study. (2008:170) In the following section we use a realist review as an illustration of how knowledge mapping techniques and tools can facilitate some steps of the realist review process. This work is in progress and we are currently finishing the third step of the review process (see figure 1).

3. Using knowledge cartography techniques and tools on a realist review on collaborative teaching Nature and context of the intervention: teamwork and collaboration are substantive ideas in post‐fordist organizations. Virtues of collaboration are constantly extolled in management articles and business magazines.

Yet, forging an authentic and sustainable collaboration culture within the organization is not an easy process.

This is particularly true in educational institutions that commonly reinforce isolation, individualism and balkanized forms off collaboration among teachers (Fullan and Hargreaves 2000). Collaborative teaching, our intervention in focus, is a form of teacher collaboration in which two or more teachers work together in planning, development, evaluation, and especially in the implementation of instructional activities directed to the same group of students. Different forms of collaborative teaching, such as team teaching, co‐teaching, and shared teaching, have been studied and practiced since the 50s. Despite of growing cumulative evidence suggesting that collaborative teaching can contribute to interdisciplinary education, to initial and continuing professional development of teachers, and to reinforce teachers’ collegiality, relatively few studies focus on the understanding of mechanisms underlying collaborative teaching interventions, particularly in higher education settings (Tractenberg 2012).

Purpose of the review: we began sketching our realist review building up from a previous systematic review in which we analyzed peer‐reviewed articles on collaborative teaching in higher education published between 1988 and 2010. Our final objective is to build a model or a conceptual framework to help understand what works in collaborative teaching practices in higher education settings.

Finding and articulating programme theories: to help finding explicit and implicit pieces of programme theories of collaborative teaching interventions, instead of drawing up a “long list” of theories we initially created a map of references with the articles found in our systematic review. To create this map, we used a

350

Leonel Tractenberg knowledge mapping software tool called Compendium, developed by the Open University (UK) 1 . As shown in figure 2, this tool allows attaching files to the map, making easy to organize, arrange and rearrange and access full‐text articles in PDF format.

Compendium supports hypermedia maps. We used this capability when creating maps of readings for each reference. Figure 3 shows the analysis of an article – Barron (2002) “Problem‐solving and English for Academic Purposes (EAP)” – in terms of research problems, hypothesis and objectives, background ideas, research methods and instruments, results and conclusions, and other elements relevant to the review. The map of readings’ structure takes the form of a mindmap in which each node is clickable and allows annotations. Figure 3 shows “background to research” annotations.

After all references were analysed, we began building a map of concepts of each study we find theoretically relevant. Figure 4 shows a map of some concepts we “extracted” from the study of Barron (2002). It is important to note that these are just some of the ideas discussed by the author in his empirical case study.

Different from quantitative research synthesis we are not looking for summing up results of effectiveness of interventions. We are looking for theoretical pieces of a mosaic that help understanding the facets of collaborative teaching.

In Barron´s study we found of central relevance the idea that collaborative teaching efforts can fail when there are philosophical differences among teachers and refusal to address them in a symmetrical; and that if we try to understand the collaborative process just in terms of degree of negotiations and trade‐offs we might fail to understand collaborative teaching in depth.

Figure 2: Map of references

4. Discussion Here we briefly described some underpinnings and steps of the realist synthesis method, and showed how knowledge mapping can be applied to the realist review process. The creation of maps of readings can be very time‐consuming, but this analysis can help the reviewer getting familiarised with the studies, synthesising their

1

See: http://compendium.open.ac.uk/institute/

351

Leonel Tractenberg main elements, appraising their rigour and identifying their contributions to the review. It also helps further with the categorisation of studies in terms of research type, research design, context and subject characteristics (e.g. EAP course, science teachers, language teachers), characteristics of the intervention (e.g. co‐teaching, team teaching, parallel teaching) etc. This can be especially useful when the purpose of the review is to understand how does the intervention work on different settings or which mechanisms work similarly in different types of intervention.

Figure 3: Map of readings

Figure 4: Map of concepts The maps of concepts took shape of a mix between concept and argument maps, thus facilitating both the identification of concept relations and hierarchies, and the identification of arguments, premises, counterarguments and conclusions extracted from each study. It is important to note that this extracted data

352

Leonel Tractenberg is not “pure”. Although these maps should be as self‐explained as possible and anchored on the ideas presented by the authors, they are always a snip set of understandings and interpretations from the reviewer.

Realist synthesis is not a mere abridgement process, but rather a reflexive and creative one.

Instead of creating a “long list” of intervention theories, and then grouping and categorising these theories, the use of a mapping software can provide the reviewer with a greater flexibility to work (compared to using pencil and paper, a spreadsheet or a text editor software). It is an invitation to “play” with concepts, moving, cutting, pasting and (re‐)combining them, facilitating the tasks of grouping, categorising and synthesizing ideas.

And it also can help with creating higher order or integrative maps that will help building a theoretically based framework or model “to be ‘populated’ with evidence” as the review work goes on.

References Barron, C. (2002) Problem‐solving and EAP: themes and issues in a collaborative teaching venture. English for Specific Purposes, No. 22, pp.297‐314.

Buzan, T. (1991). The Mind Map Book. New York: Penguin.

Cooper, H. (1998) Synthesizing research: a guide for literature reviews. 3.ed. Sage, London.

Cooper, H.M., Hedges, L.V. and Valentine, J. (eds.). (2009) Handbook of research synthesis. 2.ed. New York: Russell Sage Foundation.

Critical Reviews Advisory Group – CRAG (1996) Introduction to systematic reviews. School for Health And Related Research (ScHARR), The University of Sheffield, Sheffield, UK. Fullan, M. and Hargreaves, A. (2000) A escola como organização aprendente. 2.ed. Porto Alegre: Artmed.

Littel, J.H., Corcoran, J. and Pillai, V. (2008) Systematic reviews and meta‐analysis. Oxford: Oxford Univ. Press.

Matheus, M.C.C. (2009) Metassíntese qualitativa: desenvolvimento e contribuições para a prática baseada em evidências.

Acta Paulista de Enfermagem, Vol 22, pp.543‐545.

Novak, J. D. and A. J. Cañas (2006) The Theory Underlying Concept Maps and How to Construct Them. Technical Report IHMC CmapTools 2006‐1, Florida: Florida Institute for Human and Machine Cognition.

Okada, A. (2007) Using knowledge maps applied to open learning to foster thinking skills. Paper presented at the 12th Cambridge International Conference on Open and Distance Learning, 25‐28 Sept 2007, New Hall, Cambridge, UK. Okada, A. (2008) O que é cartografia cognitiva. In: Okada, A. (org.) Cartografia cognitiva: mapas do conhecimento para pesquisa, aprendizagem e formação docente. Cuiabá: KCM. p.37‐65 Pawson, R. (2006) Evidence‐based policy: a realist perspective. London: Sage.

Pawson, R., Greenhalgh, T., Harvey, G. and Walshe, K. (2004) Realist synthesis: an introduction. ESRC Research Methods Programme. University of Manchester, RMP Methods Paper No.2.

Sandelowski, M., Docherty, S. and Emden, C. (1997) Qualitative metasynthesis: issues and techniques. Research in Nursing & Health, Vol 20, pp.365–371.

Tractenberg, L. (2012) Ensino colaborativo e ensino colaborativo online na educação superior: uma revisão sistemática de artigos internacionais publicados ‐ 1988 a 2010. In: Leite, C., Zabalza, M. Ensino Superior: Inovação e qualidade na docência. Porto: CIIE ‐ Centro de Investigação e Intervenção Educativas. p.8623‐8643.

Tractenberg, L., Struchiner, M. and Okada, A. (2009). A case of web‐based collaborative inquiry learning using OpenLearn technologies. Paper presented at the V International Conference on Multimedia and Information and Communication Technologies in Education, 22‐24 April, Lisbon, Portugal.

Tranfield, D., Denyer, D. and Smart, P. (2003) Towards a Methodology for Developing Evidence‐Informed Management Knowledge by Means of Systematic Review. British Journal of Management, Vol 14 pp.207–222.

Vasconcellos, M. (2008) Mapeamento cognitive para revisão bibliográfica e análise conceitual. In: Okada, A. (org.).

Cartografia cognitive: mapas do conhecimento para pesquisa, aprendizagem e formação docente. Cuiabá: KCM. p.155‐173

353

Developing Knowledge Sharing Partnerships in the SME Sector: An Action Research Approach Teresa Waring, Lorraine Johnston, Angela McGrane 1 , Thuyuyen Nguyen and Peter Scullion Newcastle Business School, Faculty of Business and Law, Northumbria University, Newcastle upon Tyne, UK teresa.waring@northumbria.ac.uk lorraine.johnston@northumbria.ac.uk angela.mcgrane@northumbria.ac.uk* thuyuyen.nguyen@northumbria.ac.uk peter.scullion@northumbria.ac.uk

Abstract: The economic and social prosperity of the UK depends upon a healthy knowledge‐based economy. In a globally competitive economic environment what is needed is a talented, enterprising workforce, constant innovation in product and service development, a thriving culture of entrepreneurship, dynamic leading‐edge scientific and technological development and world‐class research that attracts investment. Lord Dearing in 2002 recognised that in collaboration with business, and with the support of government, the UK university sector has the capability to fulfil this role. This message has been echoed by successive UK Governments who have made it quite clear through reports such as the 2003 Lambert Review and more recently the 2007 Sainsbury Report that businesses and universities should be working together more closely. Wilson (2012) argues that universities are an integral part of the skills and innovation supply chain to business yet this supply chain is not a simple linear supplier‐purchaser transaction; it is not the acquisition of a single product or service.

This supply chain is multi‐dimensional, it has to be sustainable, and it has to have quality, strength and resilience. These attributes can only be secured through close collaboration, partnership and understanding between business and universities. Working in partnership with universities does not come naturally to business organizations particularly to the Small to Medium (SME) sector as there are challenges on both sides. The aim of this paper is to explore such challenges as it presents the development of a knowledge partnership between academics at Newcastle Business School (NBS) and a manufacturing company, ManCo, in the North West of England which customises vehicles for use by disabled individuals and their carers. The company approached NBS in 2010 when they were experiencing business challenges and entered into a research partnership with the university. The purpose of this partnership was to utilise university skills in business research to inform ManCo’s future business strategy. The methodological approach adopted by the two organisations was action research (Coghlan and Brannick, 2010) which involved a democratic, participative generation of knowledge ‐ something that in business research is highly unusual. Action Research (AR) has its origins in sociology, social psychology, psychology, organisational studies, education and health studies and can be distinguished from applied research.

Researchers working on AR projects, such as in organisational or social planning, have realised that the notion of ‘expertise', as pre‐supposed by applied research, is highly questionable. AR is said to involve generation of situation‐specific knowledge, not mere application of some pre‐existing knowledge. During the course of twelve months the company and academics worked together and went through three cycles of AR. The outcomes of this project are: better insight into how SME business‐university collaboration might work; generation of new customer data; and an improved business for ManCo.

Keywords: action research, knowledge partnership, business‐university collaboration, small medium enterprise

1. Introduction The economic and social prosperity of the UK depends upon a healthy knowledge‐based economy. In a globally competitive economic environment what is needed is a talented, enterprising workforce, constant innovation in product and service development, a thriving culture of entrepreneurship, dynamic leading‐edge scientific and technological development and world‐class research that attracts investment. In collaboration with business, and with the support of government, the UK university sector has the capability to fulfil this role (Dearing, 2002). This message has been echoed by successive UK Governments who have made it quite clear through reports such as the Lambert Review (2003) and Sainsbury Report (2007) that businesses and universities should be working together more closely. Wilson (2012) argues that universities are an integral part of the skills and innovation supply chain to business yet this supply chain is not a simple linear supplier‐purchaser transaction; it is not the acquisition of a single product or service. This supply chain is multi‐dimensional, it has to be sustainable, and it has to have quality, strength and resilience. These attributes can only be secured through close collaboration, partnership and understanding between business and universities.

1

For correspondence

354

Teresa Waring et al.

Working in partnership with universities does not come naturally to the SME sector despite a history of successful initiatives such as Knowledge Transfer Partnerships (KTPs). The UK Government motivation for knowledge transfer partnerships has emerged from the Lambert Review (2003) and points out that much more needs to be done to persuade business of the economic benefits to be gained from innovation and collaboration with university departments/schools. From a business‐academic perspective, researchers who work with businesses gain access to new resources, conduct applied research and develop new perspectives on existing problems; businesses get access to new ideas and can improve their chance of better performance.

Nevertheless the challenges to these knowledge partnerships are still great and as shown in Table 1 include: Table 1: The challenges of working on academic‐business knowledge transfer partnerships A Business Perspective An Academic Perspective Short termism – not in it for the long term.

The need for academic freedom Research seen as a cost rather than an investment.

Difficult to calculate economic returns The capacity within the firm to absorb research and innovation Different ideas of rigour Hard to identify potential academic partners Governance and managing conflict of interest Difficulties in managing the relationships Intellectual Property Rights Need to generate return for shareholders

Differing perceptions of time Speed Robustness of the research

However if universities and in particular business schools are to demonstrate the impact of their research and make a difference within the business community it is essential that they endeavour to overcome these obstacles and consider more innovative ways of doing so.

The aim of this paper is to explore the development of a knowledge partnership between academics at Newcastle Business School (NBS) and a manufacturing company, ManCo, in the North West of England which customises vehicles for use by disabled individuals and their carers. The company approached NBS in 2010 when faced with business challenges and after detailed discussions with one of the authors decided to enter into a research partnership with the university. The purpose of this partnership was to utilise university skills in business research in order to inform ManCo’s future business strategy. The methodological approach adopted by the two organisations was action research due to its focus on reflection as the research progressed. The desire to work in a collaborative way and to involve company representatives in all stages of decision making also leant itself to an action research approach. The outcomes of this project are: a better insight into how SME business‐university collaboration might work; generation of new customer data; and an improved business for ManCo.

2. Background to the project ManCo is a manufacturing company based in the north west of England, UK. The company employs one Executive Director and approximately forty‐five highly skilled workers including sales professionals, draft engineers, operations staff, designers and fitters. The company converts vehicles for a diverse mobility impaired customer group and acts on the principles of customer ‘freedom’ and ‘choice’ in product design. In recent years, ManCo have taken part in a KTP and another small government backed scheme which have enabled it to work in a multi‐disciplinary way with university engineers and computer science departments to enhance the company’s products and designs. Pre‐recession, the company sold 420 vehicles per year. During 2009 and faced with a difficult economic climate and global recession the company’s sales figures fell. The ED believed that ‘… the company has lost sight of its customer value requirements’. Customer value he argued ‘is created through offering differentiated products and services’. Faced with the challenges of recession and lack of understanding of his customer values, the ED made a conscious decision to seek out support at a university.

The ED of ManCo was clear that he did not want research ‘done to the company’ and needed to capture the knowledge embedded within the company.

3. Action research as a methodology Action Research has its academic origins in sociology, social psychology, psychology, organisational studies, education and health studies. The term Action Research (AR) has a long history (Lewin, 1948); and has continued to gain credence in management research (Coghlin and Brannick, 2010). Lewin (1948) described Action Research (AR) as an iterative process in which practitioners plan for action, act and then perform

355

Teresa Waring et al. reconnaissance. In his model of AR shown in Figure 1 there is a recursive relationship between thought and practice which stands in sharp contrast to standard western educational methods and traditions.

Action

Research

Reflection

Figure 1: Action research model (Lewin, 1948) Action Research can be distinguished from applied research and practitioners realise that the notion of ‘expertise', as pre‐supposed by applied research, is highly questionable. Action‐oriented research involves the generation of situation‐specific knowledge, not mere application of some pre‐existing knowledge. Approaches have emerged which include Action Learning, Action Science, Action Inquiry and Participatory Action Research (Coghlin and Brannick, 2010; Moggridge and Reason, 1996). Denscombe (1998) highlights that AR leads to practical outcomes as well as theoretical knowledge, contributing to social practice as well as theory development and bringing theory closer to practice. Achievement of change, not just knowledge acquisition, as well as a rigorous process of data generation and analysis, is essentially what differentiates AR from action learning (Clarke et al., 2006). O’Leary (2005:190) describes action researchers as working on ‘real‐world problems’ at the ‘intersection’ of the production of knowledge and a ‘systematic approach to continuous improvement’ which she argues is part of management. AR is grounded in real world problems and real‐life situations and this approach sought to satisfy the particular needs and aims of the research situation described here since, as shown in Table 1, there were perceived challenges in integrating work from two different perspectives within the project. Whilst the academic researchers might be seen as sitting outside the company there was a need to work as equal partners with participants inside the business to generate knowledge and facilitate change.

3.1 The methodological process adopted by this study There are many forms of AR only one of which is participatory AR (PAR). According to McTaggart (1991) for authentic participation to occur participant practitioners must set the agenda of inquiry, participate in collection and analysis of data and have control over the outcome of the research. Udas (1998) believes that PAR must be underpinned by some fundamentals. The first is that PAR questions the nature of knowledge, research and methods. Secondly, the nature of knowledge in PAR is for improvement of practice not for the construction of an abstract theory‐base. The PAR assumption of the nature of knowledge is that it is created by local practitioners, environments and historical factors. Thirdly, the findings and value of research are retained locally and the researcher must be prepared to be flexible and creative. Here the research team adopted the model utilised by Coghlan and Brannick (2010) which, like other variants of AR, is distinguished by a pre‐step and four stages as shown in Figure 2. The pre‐step is an important function in defining the context and purpose of the research. Avison et al. (1999) point to the need for determination of power over the structure of the project and process for renegotiation and/or cancellation. ‘Diagnosing’ is a collaborative act and seeks to identify provisional issues. ‘Planning action’ follows on from the diagnosis and is consistent with it. Taking action implements the planned interventions and ‘evaluating action’ examines outcomes intended or otherwise and links in to the next cycle of action research.

356

Teresa Waring et al.

Figure 2: The action research framework used in this research The research within this paper was designed around three action research cycles, two of which lasted approximately twelve months (Sept 2010‐Sept 11) as shown in Figure 3.

Figure 3: The three cycles of action research within this project

3.2 Involving the company and generating the data Stringer (1993:35) suggests that an authentic socially responsive AR methodology must enable participation, acknowledge people’s equality of worth and is most effective when it facilitates significant levels of active involvement, provides support for all stakeholders to learn and deals personally with people rather than with their representatives or agents. In adopting a PAR approach the project team actively encouraged the company to be participants in the research. A consistent member was the ED, Robert, and during the research period a number of staff took part in the research at appropriate times. The project team also engaged other stakeholders and this was reflected in the data generating methods as shown in Table 3. Coghlan and Brannock (2010) argue that it is more appropriate to discuss data generation rather than data collection because AR data exists through engagement with others and attempts to collect data are themselves interventions.

3.3 Survey data: recording, analysis and reporting At every stage of the research notes were taken. Thus when looking at ManCo’s company documentation e.g. strategy, promotional material, website etc., comments and questions were recorded and then explored with company staff for further insight. Where interviews were conducted or observation undertaken the team

357

Teresa Waring et al. ensured that recognised good social science research practice was adopted. From an analytical perspective the AR research team constantly engaged in discussions about the data and its interpretations with stakeholders at ManCo ensuring that the principles of participatory AR were observed (Stringer, 1999).

Table 2: Data generation methods within the research study Cycle Teasing out the Issues

Understanding the customer and the product

Acting on the knowledge

Data Generation Methods Diagnose the problem Document analysis Interviews with company staff Observation in the workplace Trade conference observation Fieldwork with customers Phone interviews with clients or their carers Pilot questionnaire data Interviews with company staff Survey plans Survey data: recording, analysis and reporting ‘get inside the heads of customers’ Strategy development meetings Dissemination of research findings Plan ‐ let’s do it?

Decide course of action to be taken Communicate action/Implement changes to employees Monitor effects of action ‐ Reflectivity of findings and improvements: What worked? What did not work? Future direction – where next?

Learning from know‐how – Identify new issues Press Release to media

3.4 Quality and rigour in action research Reason and Bradbury (2001) argue that AR should not be judged by the criteria of positivist science and requires its own criteria as shown in Table 3. Rigour in AR refers to how data are generated, gathered, explored and evaluated, how events are questioned and interpreted through multiple AR cycles.

Table 3: Criteria for quality and rigour in action research Criteria for quality in AR (Reason and Bradbury, 2001) The AR should be explicit in developing a praxis of relational participation – there should be evidence that there was cooperation between the action researcher and members of the organization.

The AR should be guided by a reflexive concern for practical outcomes.

The AR should include a plurality of knowing which ensures conceptual‐theoretical integrity; extends ways of knowing and has methodological appropriateness.

The AR should engage in significant work

The AR should result in sustainable change.

Rigour in AR (Reason, 2006) Show how there was engagement in the steps of multiple repetitious AR cycles and how these were recorded to reflect a ‘true’ representation of what was studied.

Show how assumptions and interpretations were continuously challenged and tested throughout the project to expose them to critique.

Show how different views of what was happening were accessed.

Show how interpretations and diagnoses are grounded in scholarly theory, rigorously applied and how the project outcomes are challenged, supported or disconfirmed in terms of theories underpinning those interpretations.

4. Findings This section outlines the action research project undertaken at ManCo and answers the three questions posed

by Coghlan and Brannick (2010): What happened?; How do you make sense of what happened?; So what?

358

Teresa Waring et al.

4.1 Cycle 1: Teasing out the issues The initial discussions with the ED and sales staff at ManCo were concerned with diagnosing the key research problem: ‘What do ManCo really want to know about their customers?’

Diagnosing: First steps included initial desk research of secondary data around the mobility car industry sector.

Researchers found this useful to gain background knowledge into the industry and ManCo’s customer base.

One of the researchers attended a trade fair where she informally met customers and discussed mobility issues with them. This was followed by a company site visit to meet with employees including sales, engineering, administrative and production staff. This initial meeting with employees was important in order to understand the nature of the problem. Researchers then conducted a conference call with the wider AR team and the ManCo employees to ensure that relations on both sides were cementing. This period was also characterised by many email and telephone calls. Thus during the first two months of diagnosis (Sep 2010‐Nov 2010) the following issues from the university perspective emerged in cycle 1:

How well do ManCo understand their customers and customer needs?

How do ManCo communicate with their customers?

Why do customers remain loyal and come back to ManCo?

Similar discussions were taking place in ManCo. For example ‘what does the business need to do to remain profitable?’; issues around design – ‘what works and what should be discontinued?’; ‘what do customers value in a product?’; ‘What decision making process does a customer go through when making a purchase?’; ‘Why do they choose a ManCo brand over a competitor brand?’ Other research challenges included the currency of the company customer database needed to support the research and the involvement of ManCo staff in the knowledge development and embedding process. This was a sensitive time as the company were facing economic difficulties and the company needed to change some of its business practices in order to remain competitive.

Planning action: In order to tease out these problems further it was clear that more in depth research needed to be done. This would involve conducting a pilot survey and interviewing customers. As many of the customers were disabled interviews with them would need to be done in a sensitive manner recognising that disability comes in many shapes and forms. There were also customers who were not disabled but cared for a disabled person. Discussions were held with the company and a pilot survey was planned by the AR team.

Some challenges included the nature of the survey – online, telephone or postal; the questions that would be asked; the size of the survey; who would complete it and how it would be analysed. A telephone pilot study was decided on as it could reach a national clientele and it was agreed that ten disabled customers or carers would be contacted. Once again the integrity of the ManCo customer database gave cause for concern as the AR team wanted to protect customers’ right to privacy and confidentiality. ManCo agreed that the database would be cleansed prior to the survey going out.

Taking action: This part of the cycle included developing the questions for the survey, carrying out the pilot study and constructing the questionnaire. This was done with the AR team involving ManCo’s ED and employees. A key consideration here was addressing carefully who might complete the questionnaire. The researchers also needed to pilot the survey and began a dialogue with ManCo’s customers. At this stage the researchers learnt that customers were wider‐ranging and included the disabled customer groups as well as able‐bodied carers, guardians and spouses and those who may have been employed as professional workers.

Other issues surrounding the interest group were a consideration that the customer may have impairment in motor and sensory functions. Customers may experience reading and literacy difficulties and have mobility issues including mild visual impairments and difficulty walking; they may find it easier to listen to rather than read a survey. If a postal or online survey was to be conducted careful consideration needed to be given to design to acknowledge these issues.

During this period ManCo staff cleaned their customer database. This took about 3‐4 weeks. The AR team developed a draft of the questionnaire and at this point the team invited a research specialist in disability studies to comment on the draft questionnaire. This exercise ensured that the questionnaire design was appropriate. The pilot questionnaire was tested with ten customers and explored how customer attitudes had changed buying behaviour. The administration was pre‐arranged and agreed in advance with respondents –

359

Teresa Waring et al. e.g. time agreed to telephone. One customer could not participate due to personal circumstances and another because of work commitments. The rationale used to identify customers for the pilot came from the ED ‘we need to get customers that didn’t buy from us which resulted in lost sales’. Other customers were contacted simply because they had requested product information or had a more general enquiry about a product and ‘received valuable advice’. It was important to gain new insights into changing customer habits and purchasing needs and to find out what these customers wanted from products. Once the pilot survey had been completed the data were analysed using appropriate statistical techniques.

Evaluating the action: The pilot survey provided further insight into ManCo target customers and the questionnaire. For example certain questions were more appropriate than others in terms of their relevance, clarity, length and order. Further insights were gained into the profile of the respondents including classifications of occupation, status and relationship to the customer, age and lifestyle. The participants were highly receptive to telephone interviews and referred to themselves and their condition in non‐politically correct terms e.g. ‘I’m a spastic’. This was in contrast to researchers who had been more circumspect when expressing a disability. The ED of ManCo had thought that an incentive might ensure a ‘higher response rate’ for completed and returned questionnaires. Yet respondents’ feedback from the telephone survey suggested that a registered charitable donation was preferred. Others preferred ‘personal gain’ by means of a shop voucher or a bottle of champagne. The final decision was left to ManCo.

A key outcome was that participants were pleased that ManCo was interested in their ‘opinions’ and for ‘taking the time to communicate that their views on customer value were important’. One argued that ‘the research is demonstrating that someone is actually listening to our underrepresented group’. Another respondent thanked the researcher ‘for raising awareness of their needs through the project’. Additional respondents noted that ‘…your research has identified a problem for disabled people like me and that is that companies do not consider our needs and will try to sell you anything. We [disabled] are vulnerable’. Overall this process elicited a highly positive response.

4.2 Cycle 2: Understanding the customer and the product During March 2011 a tsunami in Japan halted car and parts production in some Japanese firms. This impacted badly on ManCo whose business relied on this supply chain.

Diagnosing: Going into this Cycle provided an opportunity for ManCo and their employees to provide more comprehensive comments on the draft survey instrument. Answering the company’s specific concerns and questions around customer value was paramount. It was important to the company to maintain the inclusive nature of the research as they wanted ‘buy in’ from their staff. Comments were emailed to the researchers in Newcastle for consideration. Communication with the ED and the AR team continued as the survey instrument was examined for minor errors. At this part of the cycle there was an opportunity for researchers to reflect on the discussions and emails, tighten the focus of the questionnaire and reduce the length to six pages. With final feedback from ManCo some small changes were incorporated to focus the detail of the questionnaire.

Planning the action: At this point in the cycle some survey questions were reconfigured and prioritised against ManCo’s strategic plan. The strategic decision was based on company objectives. For example, this generated new questions around: what design aspects customers really value in a vehicle; what type of decision making processes do customers experience when purchasing? ManCo wanted to inform the future marketing and product development and understand what customers may require in future. ManCo requested more information about who customers talk to about products, which magazines they read and how much they are influenced by brands. This had to be balanced by the known theory on survey design if ManCo wanted to get a good response rate to inform their business strategy. In terms of planning for the data collection issues emerged over who would administer the survey ManCo suggested they should log responses and send out a reminder letter. The ED made a decision not to incentivise on the first mailing of the survey, however, they ‘might on the second' depending on response rates. The company also decided that they would meet the cost of mailing the survey out to customers but the accompanying personal letter (on ManCo headed paper) would come from university researchers and would explain the purpose. Researcher contact details were made available for further explanation. Ethical considerations were discussed and it was agreed that researchers would not require any personal information but would deal only with coded questionnaires.

Freepost envelopes were provided to all customers with clear instructions on how to return completed

360

Teresa Waring et al. questionnaires. In this stage ManCo’s ED aspired to target the survey towards 1000 randomly selected customers (600 existing and 400 potential customers). The company sought a 50% response rate.

Taking action: The final survey instrument was agreed jointly by the AR team and company with 800 questionnaires sent out in March 2011 to the customer sample identified by ManCo. Completed surveys were returned to the company by 31 March 2011. To minimise error, a ManCo employee was assigned to record the number of surveys returned. Out of 800, 120 completed surveys gave a response rate of 15%. ManCo transported completed surveys in batches of 20‐25 to the AR team at the university. Data entry was undertaken by a skilled research assistant who used a pre‐designed coding plan for speed and accuracy. This process took two weeks and entries that did not tie in with coding were cross‐checked by a senior researcher before data were imported into the Statistical Package for the Social Science (SPSS) software. A report was prepared by the AR team.

Evaluating the action: At this point in the Cycle analysis and reflection was undertaken by the research team.

Preliminary findings were discussed with the AR team and a meeting was arranged with the ED. He was initially overwhelmed by the result. Having taken time to consider some of the outcomes the ED sought further clarifications and more analysis. The AR team continued to meet to discuss these additional findings. In October 2011, these results were presented back to ManCo in a Supplementary Data Analysis report.

4.3 Cycle 3: Acting on the knowledge Diagnosing: In this phase of the AR cycle, ManCo’s ED and his sales team evaluated the report. They then ‘discussed and highlighted the results of the report to the management team’. A ManCo management meeting discussed the key findings emerging from the report and this was then used to guide the strategic decisions of the business moving forward. Discussions ensued around how this might be undertaken, how to remove obstacles and who would be responsible to ensure tasks were carried out. This was challenging.

Planning action: Based on the survey results, the management team considered which business processes needed to change. The ED commented that ‘it took 3‐4 meetings over a six month period before we began to identify specific actions, customers’ communication before we fed into and produced our strategic plan’.

According to the ED the research enabled ManCo to plan key actions.

Taking action: The company maintained an open dialogue with their customers. For example they are convinced that trade events worked and these should be maintained. They have introduced a more tangible communication and reward system to retain high value customers and have made a conscious effort to attract more customers like that. This was implemented through different communication channels i.e. telephone, email and postal. ManCo identified the high value customer group to nurture. They have looked more at potential customers who are in a position to purchase ManCo products and have become more strategic in the use of their resources. This has brought another dimension to the role of the sales professional within ManCo. Evaluating the action: The collection and analysis of the customer data focused the minds of the senior staff in ManCo. Over the next few years knowledge acquisition about their customers they believe is more fully informed and their strategy is clear. They know the areas where they are going to build innovations into the product design and at what cost to the customer. They reflected on how the research project impacted upon employees and believe that, by sharing the findings of the research with employees, they are better informed about customer needs and the importance of good design to inform the company strategy. At the time of writing in terms of the impact of the research on customers it is too early to evaluate. It is possible that another survey could be undertaken in 12 months’ time to see if problems identified by customers within the survey have been resolved. Although the UK is still in recession the fortunes of the company have been reversed. ManCo may not have initially recognised the value of the research. The payback is there yet, according to the ED, there is ‘work still to do’.

5. Conclusion and reflections The university‐ManCo relationship worked extremely well because university researchers were prepared to engage with the company at every level. By using participatory AR (Udas, 1998; Stringer, 1993) everyone not just the ED was included in the project. The research team engaged in a meaningful way with the company and encouraged co‐construction of knowledge. From the company perspective the relationship was a great example of a real partnership between academic researchers and business. According to the ED this was a

361

Teresa Waring et al. unique contrast to his previous experience of consultancy ‘which has a why don’t you do this attitude’. ManCo are going into production with a new model and this has been influenced by the results of the survey. For example sound insulation will be offered after they spent a lot of time looking at sound–deadening materials and vehicles are much quieter. This has been done in–house rather than by the manufacturer and has given the company a competitive advantage. The ED reflected that the company has gone from strength to strength.

The findings informed the company’s future strategic direction.

Beyond the application to the single company involved in the research project, the findings have implications for policy, academic practice and the SME community. The work described here demonstrates the value of Action Research in developing ‘bottom up’ models of partnership between universities and SMEs. The findings in relation to the usefulness of the method to promote such partnerships are not peculiar to the sector described, and should be of value to anyone seeking to promote SME engagement with the HE research community.

References Avison, D.E., Lau, F., Myers, M., Nielsen, P.A. (1999), Action Research. Communications of the ACM. January 1999.Vol.42.No.1.pp 94‐97.

Clarke, J., Thorpe, R., Anderson, L. and Gold, J. (2006) It’s all action learning: action learning in SMEs, Journal of European Industrial Training, 30(6), pp441‐455.

Coghlan, D and Brannick, T (2010) Doing Action Research in your own organization. 3rd edition. Sage publications. London.

Dearing (2002) National Committee of Inquiry into Higher Education, Report of the National Committee, Norwich, HMSO. Denscombe, M. (1998), The Good Research Guide for Small Scale Research Projects. Open University Press, Buckingham.

Lambert Review (2003) Lambert Review of Business‐University Collaboration: Final Report, Norwich:HMSO. Lewin, K. (1948), Action Research and Minority problems in T G W Lewin (ed) Resolving Social Conflicts. Selected Papers on Group Dynamics. Harper and Row. New York. p201‐216.

McTaggart, R. (1991), Principles for Participatory Action Research. Adult Education Quarterly. 41(3) p168‐187.

Moggridge, A. and Reason, P. (1996), Human Inquiry: Steps Towards Emancipatory Practice. Systems Practice, Vol. 9, No. 2. p.159‐175.

O’Leary, Z. (2005) Researching Real‐World Problems: A Guide to Methods of Inquiry. Sage Publications. London.

Reason, P (2006) Choice and Quality in Action Research Practice. Journal of Management Inquiry. Vol. 15 (2). 187‐203.

Reason, P and Bradbury, H (2001) Handbook of Action Research: Participative Inquiry and Practice. Sage Publications.

London.

Sainsbury Report (2007) The Race to the Top: A Review of Government’s Science and Innovation Policies, Norwich:HMSO. Stringer, E. T.(1993), Socially responsive educational research: linking theory and practice in D. J. Flindes and G.E. Mills (eds.). Theory and Concepts in Qualitative Research. Perspectives from the field. Teachers College Press. New York. pp141‐162.

Udas, K. (1998) Participatory Action Research as Critical Pedagogy. Systemic Practice and Action Research.

Vol.11.No.6.p599‐628 Wilson, T. (2012) A Review of Business‐University Collaboration, London:BIS.

362

PhD Research Papers

363

364

Empirical Examination of Internal Dynamics of Software Project Management: Mixed Methods Approach Zana Ahmedshareef, Miltos Petridis and Robert Hughes University of Brighton, Brighton, UK Z.Ahmedshareef@brighton.ac.uk M.Petridis@brighton.ac.uk R.T.Hughes@brighton.ac.uk

Abstract: In today’s increasingly competitive business environment, managing software engineering projects has become more challenging and complex. Project managers are expected to deliver high quality products in a shorter time with fewer or less expensive resources. Despite the availability of well‐established project management frameworks and innovative tools and techniques, software projects still fall short of meeting their time, cost and quality objectives. While there are many studies that treat projects as ‘black boxes’ and retrospectively analyse the externally apparent characteristics of projects that might influence their success, this work addresses the internal processes of project management. This paper describes the methods developed to conduct an empirical study into the internal dynamics ‐ often reacting to external events ‐ of software project management during project execution focussing on the Design, Build, and Test (DBT) stages of the application development lifecycle. The data used comes from a case study carried out at a large software development company which uses an incremental delivery model with intensive parallel development in a multi‐location team environment. The main research concerns of the overall study are the extent to which the project performance indicators currently used provide enough information to project managers to make good decisions, and the range of issues that occur during project execution that affect the project’s ability to achieve its time, cost and quality objectives. This paper discusses the research methods that were eventually adopted. The original intention was to build numerical models using the largely quantitative data generated by the project management processes. The study now employs a mixed methods approach using a ‘convergent parallel’ model (Creswell & Clark 2011). Quantitative and qualitative data are collected in parallel, analysed separately, and then merged for interpretation. This approach provides a more complete understanding of the project management processes and issues. A Grounded Theory approach was used to analyse the textual progress reports generated on a weekly basis during the execution of the project. As well as giving insights into the obstacles to progress experienced, this also provided a view of how the progress reports were used by individuals to influence perceptions of the individual elements of the project. This paper focuses on the research approach adopted in the study and its suitability for use in empirical software project management research. The study findings demonstrate that considerable benefit can be gained from using mixed methods approach to obtain a fuller understanding of the management dynamics of project execution.

Keywords: empirical software project management, empirical research methods in software engineering, grounded theory, mixed methods in software engineering

1. Introduction Developments in frameworks, tools, and techniques have not been decisive in overcoming threats to software development project success ‐ see for example Ahlemann et al. (2012), Rabbi & Mannan (2008), Rose et al.

(2007), Bannerman (2008), and Taylor (2006). Whilst much existing research has attempted to address these challenges, they tended to treat these projects as a black box looking at external influences affecting project success ‐ see for example Cerpa & Verner (2009) and Verner & Evanco (2005). There is a need for getting insight into the internal factors affecting project success and developing a fuller representation of the examined area.

This study examines the internal dynamics – particularly when reacting to external events ‐ of software project execution during the Design, Build and Test (DBT) stages of the application development lifecycle. The study employed a mixed methods approach to enquiry utilising what Creswell and Clark (2011) call the ‘convergent parallel’ design model. The decision to use mixed methods design emerged during the course of the study, rather at the outset. Early work revealed that adopting a quantitative method alone provided only a partial understanding of the area; hence a qualitative strand of enquiry was added to strengthen the study.

The original research motivation was to better inform project managers of the individual challenges that emerge during project execution and to reduce the uncertainties associated with software development projects. The research was based on the following principles:

365

Zana Ahmedshareef, Miltos Petridis and Robert Hughes

An empirical study of real project data to generate knowledge that ‘supports the advancement of the software engineering field’ (Basili et al. 1986, page: 733).

An examination of past project data: this followed the advice that organisations should conduct post‐ mortem examination to learn from their mistakes (Cerpa & Verner 2009, Verner & Evanco 2005).

The research examines extant project data from ABC ‐ a fictional name of a real software development firm.

The project data was generated and used at the time of project execution by project participants rather than being collected retrospectively. The research used the numeric data to investigate the project’s schedule performance over time, and the textual information to discover emergent phenomena during project execution. Thus, most of the data has been used twice (i) as part of the management process that managed the project (ii) as data for the research carried out after the completion of the project. The data was processed/analysed differently on each occasion.

The research questions of the study are:

RQ1: To what extent do the performance measurements, collected during project execution, provide enough information to project managers to make good decisions?

RQ2: In what ways the project data, made available to the research, can be used to increase our understanding of software project execution delays?

RQ3: What theoretical frameworks help us to understand the practices of the studied organisation, and what constructs are unique to this case?

RQ4: What phenomena emerge during software project execution that contributes to the projects becoming late?

2. Related literature review In seeking to answer the research questions; relevant literature has been examined, whereas the available organisation’s data have been used to provide practical answers to help inform future research and guide management decisions. The following sections examine related literature to help in making sense of ABC’s work practices (see RQ1 and RQ3).

ABC used two key ‘best practice’ techniques: an incremental delivery model to enable intensive development by parallel teams to shorten lead time, and a modified version of Earned Value Management (EVM) to enable managers to have a realistic and timely view of project progress.

2.1 Incremental delivery model ABC used an incremental model that delivered software products over a number of development cycles building chunks of software functionality in each increment. While this model has been widely and successfully adopted (Larman & Basili, 2003), complexity increases in such projects and they require more management control and monitoring during project execution compared to traditional approaches – see Redmill (1992) and Graham (1992).

2.2 Earned value management The success of delivering software engineering projects is typically evaluated by the degree to which they deliver on schedule, within budget and to quality specifications. A number of frameworks have been proposed to assist the project managers in measuring, controlling, and monitoring project performance, for example, the Quantitative Project Management and Measurement and Analysis process areas of CMMI (Chrissis et al. 2011); the Executing the Strategy discipline area from APM (2006); and Earned Value Management (EVM) as proposed by PMI (PMI, 2004).

EVM, which employs the Earned Value Analysis (EVA) technique, is a management approach used to monitor and control project execution. It integrates measures of the scope, schedule and cost for a unit of work as a basis for comparing actual progress and cost performance against a baseline plan (PMI 2004, Fleming & Koppelman 2010, Budd & Budd 2010).

EVM has a set of base measures which include: Planned Value (PV), Earned Value (EV), and Budget at Completion (BAC). PV is analogous to an agreed price for a unit of work to be completed, and when the work

366

Zana Ahmedshareef, Miltos Petridis and Robert Hughes has actually been completed, the amount of PV is said to have been earned and becomes EV (PMI 2004, Fleming & Koppelman 2010). Among the different conventions used in assigning EV to completed work is 0/100 where EV is assigned value of 0% until the task is 100% complete (Anbari 2003, Fleming & Koppelman 2010). BAC is the overall estimated project cost, that is, the total PV for the project (PMI 2004, Fleming & Koppelman 2010, Budd & Budd 2010).

Performance indicators are used to assess a project’s performance. Schedule Performance Index (SPI) is used to assess the state of the project schedule by calculating the ratio of EV to PV (SPI = EV divided by PV). An SPI value greater than 1 means that project progress is ahead of schedule; an SPI less than 1 indicates that progress is behind schedule. Hence an SPI of 1 indicates that the project is on schedule (PMI 2004, Chrissis et al. 2011). Percentage Complete (PC), expressed as a percentage value, is an estimate of the amount of work that has been completed relative to the overall estimated project cost (PMI, 2004), that is EV divided by BAC. For example, if the overall estimated cost of a project is £100,000, and in the first quarter of its progress £25,000 worth of work was completed, then PC is 25%.

Project performance can be evaluated at progress review meetings using the information derived above.

Techniques such as variance analysis, variance percentages, and performance indices can be employed to monitor progress (Chrissis et al. 2011, PMI, 2004, Anbari 2003). As we will see, ABC project control processes used a terminology and model based on conventional EVM but differing significantly from it.

2.3 Risk management Managing project risks is integral to project management. Project management frameworks have proposed standards for handling such uncertainties effectively ‐ see PMI (2004), APM (2006), and Chrissis et al. (2011).

Once again, we will see how ABC adapted these recommendations to best meet their needs.

2.4 Approaches to research In order to answer the research questions asked above, ways needed to be established of analysing the documentary records left after the completion of a project. These details contained both numeric and textual data and so both quantitative and qualitative approaches were examined (RQ2).

2.4.1 Quantitative method The quantitative approach to enquiry aims to explain and quantify variation in the phenomena under investigation. Common quantitative designs include: longitudinal, retrospective, and non‐experimental designs. Examples of quantitative method in empirical studies include Lipke et al. (2009) and Kim et al. (2003).

The ABC data contained planned and actual completion dates for individual activities to quite a detailed level which could be subjected to numeric analysis. This analysis would clearly be retrospective. It will became clear, however, that the numeric data provides partial information to project managers as it only shows the extent to which projects are late but not why and how they became late (RQ1 and RQ2).

2.4.2 Qualitative method The qualitative approach to enquiry aims to explore and describe variation in the phenomena under enquiry.

Qualitative study designs include: observations, interviews, examining extant documents, ethnography, case study, and grounded theory.

Grounded Theory Method (GTM) has gained particular interest in empirical software engineering in recent years ‐ see for example Lehtinen & Mäntylä (2011), Coleman & O’Connor (2007), Balaji, et al. (2006), Montoni & Rocha (2010), Rose et al. (2007), and Hoda (2010). GTM aims to construct theoretical frameworks of latent patterns from the data focusing on the participants’ perspective of the problem domain, in contrast to observational studies which focus on providing description of what the participants do – see for example Hughes (2011). The process of constructing a grounded theory involves: data collection, coding, developing categories, memo writing, theoretical sampling, sorting, and constructing theory (Charmaz, 2011). Grounded Theory is usually associated with interviewing, but the mechanisms for gathering data observations and composing data documents are largely ignored in most discussions of the approach (Locke 2001, page: 44).

367

Zana Ahmedshareef, Miltos Petridis and Robert Hughes Using GTM to analyse the textual data of ABC reports appeared to be particularly appropriate for the research to be undertaken (RQ2 and RQ4).

2.4.3 Mixed Methods The mixed methods approach to enquiry combines quantitative and qualitative design characteristics aiming to provide the most complete understanding of the phenomena under examination. Mixed Methods designs include convergent parallel model where the quantitative and qualitative methods are implemented concurrently but independently during data collection and analysis, then the results are merged during the overall interpretation (Creswell & Clark, 2011).

Mixed methods research has gained growing interest and has been used by researchers in many disciplines (Creswell & Clark, 2011). Examples of mixed methods research in empirical software engineering include Lehtinen (2012) and Seaman (1999).

Given the mixed nature of the data available from ABC, a mixed methods approach seemed very attractive (RQ2).

3. The case study Empirical research in software engineering should provide sufficient contextual information to ensure that particularities of the studied area are taken into consideration by other researchers and practitioners (Kitchenham et al. 2001). The next sections provide contextual detail on ABC (RQ3).

3.1 Project delivery model ABC developed customised large systems software for external clients. The company ran multiple projects simultaneously with Design, Build, and Test (DBT) activities each carried out in different locations. The company followed an established proprietary incremental method, where a project consisted of multiple releases ‐ a release is a version of the software that is released to users. A release consisted of multiple increments ‐ an increment represents one development cycle comprising the three DBT stages which delivers a portion of the software functionality (Figure 1).

Legend

Design

Design

Design

Increment 1

Design

Project execution

Increment 2 Increment 3

Build

Build

Test

Build

Test

Project progress

Build

Test

Increment n

Test

Software released to users

Figure 1: Release lifecycle in ABC In Figure 1, the vertical arrow ‘Project execution’ shows the increment cycle, where workflow moves across stages; from design to build to test. The horizontal arrow ‘Project progress’ shows workflow moving to the next increment of functionality within a particular functional team (stage); for example, following completion of design work in increment 1; the team started to design increment 2 of the functionality and so on, the same principle applied to the build and test stages. In some ways this can be seen as a production line model.

A problem controlling this type of project is that at any one time, the functional project teams will be working on different increments of the same project.

A stage comprises the phases:

Design: comprising Functional Design (FD) phase.

368

Zana Ahmedshareef, Miltos Petridis and Robert Hughes

Build: comprising FD/TD Transition, Technical Design (TD), and Code phases.

Test: comprising Assembly Test (AT) and Integration Test (IT) phases.

Thus, an increment encompassed the following phases: FD, FD/TD Transition, TD, Code, AT, and IT. Each phase comprises a set of events marking the occurrences required to complete the phase. Each event had two attributes: ‘Planned occurrence date’ and ‘Actual occurrence date’ indicating the planned finish date and actual finish date.

3.2 Measuring project performance Operationally, the DBT stages had separate teams responsible for delivering artefacts of each stage, each with a team lead reporting to the project manager. The project leads led their teams to complete scheduled activities by the ‘Planned occurrence date’ of the corresponding event to track progress.

ABC used a modified version of EVM. It focused on events rather than on activities as in standard EVM. Events became the unit of measurement. Planned value (PV) became a count of the events scheduled to be completed at a certain date. EV was a count of the events that were actually completed by that date. Thus, on a particular day the units that should have been completed (PV) could be compared with those actually completed (EV). The EV and PV units were accumulated over the period of the phase allowing the team leads and project manager to ascertain the degree to which completed work matched to planned.

The leads used the base data above to calculate phase performance indicators in order to establish the efficiently of their team’s progress. They used the event‐based SPI and percentage completion (PC) of work activities. ABC calculated SPI in the same way as the standard EVM measure (described in section 2.2); however they used cumulative SPI at phase level rather than on the overall project as in standard EVM. ABC also calculated percentage completion of work in the same way as the standard measure.

3.3 Monitoring project performance The team leads collated project information into weekly project performance reports. These were talked through in the weekly performance meetings with management. Schedule performance was considered the key measure of delivery success. Besides indicating timely delivering, it was a sign of cost overrun; because delays in schedule increased effort and thus costs. These costs would be incurred by the company and/or charged to the client.

3.4 Managing project risks ABC handled uncertainties in two complementary ways. Depending on their need for urgent attention they were (i) logged, if less urgent, in a standard risk register, or (ii) if more urgent identified in the weekly performance reports for immediate tackling. Both these practices were used as defence mechanism/self‐ protection against being blamed for the failure of meeting planned schedule dates.

4. Conducting the study 4.1 Design Section 2.4 has described how the decision was made to use mixed methods research design combining the strengths of quantitative and qualitative approaches. The quantitative strand used longitudinal, retrospective, and non‐experimental designs to investigate the extent of variation in schedule performance (using SPI data) over time, for past projects. The qualitative strand used Grounded Theory Method to explore the narrative information of the project performance reports and the risk entries of the project risk register for emergent phenomena.

Creswell & Clark’s (2011) convergent parallel model was adopted in this study (Figure 2). Both the numeric and textual source data, which were originally produced as parts of one unit within a weekly cycle where the later expanded and interpreted the former, were separated for independent strands of analysis in the study.

369

Zana Ahmedshareef, Miltos Petridis and Robert Hughes

Figure 2: The research design of the study – convergent parallel model

4.2 The research process The source data was for five completed software engineering projects; comprising 11 releases containing 46 increments. The progress data for each release was entered into a spreadsheet after removing references to the original organisation and to identifiable projects and individuals.

Quantitative analysis was based on dividing the number of tasks actually completed for each phase by a count of all the tasks were due to be completed to give an ‘event SPI’ for the phase in each week – see 3.2 above.

Grounded Theory Method (GTM) techniques were used for the qualitative analysis. The following example demonstrates the steps followed to surface the emergent categories:

Step 1: Initial coding ‐ the source information was coded by labelling segments of text that summarise and account for what is happening in the statement (Figure 3).

Figure 3: Initial coding Step 2: Focused coding ‐ the initial codes that capture and synthesise the main themes in the statement were selected for subsequent analysis (Figure 4).

Figure 4: Focused coding Step 3: Categories development ‐ focused codes were analysed for what they best represent happening in the text; and mapped to higher‐level categories (Figure 5).

370

Zana Ahmedshareef, Miltos Petridis and Robert Hughes

Figure 5: Categories development Step 4: Memo writing ‐ categories were elaborated in early memos to determine common themes and patterns they subsume in several codes, and to discover their properties and relationships (Figure 6).

Figure 6: Memo writing The next steps of the GTM; theoretical sampling, sorting, and constructing theory will be addressed in the full study.

Table 1 illustrates a preliminary list of the categories emerged from the GTM process along with their description (RQ4).

Table 1: Emergent categories cat #

Categories

1

Late events

2

Struggling to obtain input

3

Living through uncertainty

4 5 6 7

Grappling with time Raising concerns Hinting of breaking commitment Revising schedule

8

Late phases

9 10 11 12

Justifying breaking commitment Portraying powerlessness Tracing causality of failure Delivering news

Description Failing to meet planned occurrence date for an event within a phase. Late events are recoverable and might not impact project end date.

Describing lack of/delays in receiving input from dependent teams in order to complete own deliverable.

Operating in uncertain/fluctuating settings, for example constant changing of scope or requirements.

Lacking sufficient time to meet deadline.

Voicing uneasiness towards a particular situation.

Making tentative suggestions of inability to meet deadline.

Amending the planned occurrence date of an event or the planned finish date of a phase, to a later date.

Failing to meet planned finish date for a phase within a release. Late phases are not recoverable and will impact release end date.

Explaining internal circumstances leading to inability to meet deadline.

Suggesting onus on others to complete dependent activities.

Outlining external factors causing failure to meet own deadline.

Informing of some news of interest.

371

Zana Ahmedshareef, Miltos Petridis and Robert Hughes

4.3 Interpretation Grounded theory method facilitates understanding of latent patterns in a situation from the participants’ perspective, and allows for the generation of theories of process, sequence, organisation change, positions, and participant’s interaction (Glaser & Strauss 1967, page: 114). Based on this premise, the quantitative and qualitative results relating to each week were matched to get the complete picture of what was happening in the particular week (RQ2).

To illustrate this, the ‘Late events’ category in Table 1 is used to interpret what happened within a functional design phase. Figure 7 shows the schedule performance of the design phase where, in some weeks, the SPI threshold value is 1 signifying being on schedule. However, as we can see that progress SPI in some other weeks fell below threshold of 1, meaning that progress is behind schedule because of late completion events.

Figure 7: Event performance analysis Late events (cat #1 in Table 1) emerged in Wk1 due to the design team’s struggle to obtain customer requirements from the client (cat #2), uncertainty about the requirements and scope (cat #3), a third‐party (supplier of specialist software components) being late in providing high level designs (HLDs) (cat #2), and lack of time to get functional knowledge from the client (cat #4). During Wk2, progress stayed behind schedule with delays in receiving critical information related to architecture and process (cat #2) and the continued uncertainty about requirements and scope (cat #3) leading to events to be late (cat #1).

The phase progress came back on schedule in Wk3 as the designers were able to catch up on earlier delays, however, they raised concerns (cat #5) over scope increase and uncertainty about requirements (cat #3). The design lead foresaw a struggle to meet phase milestone (cat #6). As the design team further grappled with time (cat #4), they revised the activity schedule (cat #7). In Wk4, the phase became late (cat #8) because the project scope was changed without following the change control procedures (cat #9) and late receipt of HLDs from the same third‐party (cat #2). This resulted in re‐planning of the phase (cat #7) with senior management approval to allow for scope increase; meaning that (i) the original planned completion date for the remaining activities in the phase were amended (ii) some of the earlier activities had to be revisited to take account of scope change. Despite the schedule changes, the team still anticipated difficulty in meeting the revised schedule (cat #6).

Although progress in Wk5 was on, the new, schedule, the design team claimed they were faced with obstacles preventing progress (cat #10); blaming late receipt of HLDs from the third‐party above once more (cat #2) and continuing lack of communication from consumer teams on requirements definition (cat #3). Consequently the team anticipated difficulty in meeting the planned end date of the phase (cat #6). Wk6 exhibited client unavailability and late receipt of client feedback (cat #11) causing events to be late (cat #1). This led to continued concern that the phase would be late (cat #6) as dependent input was received late from the third‐ party above (cat#9); a sentiment persisted in Wk7 (cat#9) until the phase’s closure in Wk8 (cat #12).

On the whole, there were times when progress was late but the phase was still completed on schedule because of the re‐planning of the phase in Wk4 – i.e. the goal‐posts were moved.

372

Zana Ahmedshareef, Miltos Petridis and Robert Hughes

5. Conclusion Section 1 has outlined the research questions addressed in this paper. The study used a mixed methods approach to examine the internal dynamics, often when reacting to external features, of software project management. The approach demonstrated effectiveness through presenting a more complete understanding of the challenges faced project managers. Through combining the strength of quantitative with qualitative methods using the convergent parallel model, it was possible to establish: what problems impacted project progress, when in the lifecycle they occurred, and why these challenges emerged. The matching of the numeric data and textual information resulted in each providing a side of the story that is different but complementary to the other (RQ2 and RQ4).

The ABC’s methodology to control projects varies from the standard model to best meet their needs; ABC (i) focused on events delivery in performance meetings as in this way the cost will take care of itself (ii) focused on speed of delivery using incremental delivery model with intensive parallel development (iii) used narratives to explain performance indices since the quantitative data alone was insufficient to guide project participants of what is happening (RQ1 and RQ3).

The findings, thus far, have been drawn from analysing one project; however, a full study is planned in order to arrive to more conclusive findings.

References Ahlemann, F., Arbi, F. E., Kaiser, M. G. & Heck, A. (2012) 'A process framework for theoretically grounded prescriptive research in the project management field'. International Journal of Project Management, Article In Press, 14.

Anbari, F. T. (2003) 'Earned Value Project Management Method And Extensions'. Project Management Journal, 34 (4), 12‐ 23.

APM (2006) APM Body of Knowledge, UK, Association for Project Management.

Balaji, S., Ahuja, M. K. & Ranganathan, C. (2006) 'Offshore Software Projects: Assessing the Effect of Knowledge Transfer'.

Proceedings of the 39th Hawaii International Conference on System Sciences ‐ 2006. IEEE Conference Publications.

Bannerman, P. L. (2008) ‘Risk and risk management in software projects: A reassessment’. The Journal of Systems and Software, 81 (12), 2118‐2133.

Basili, V. R., Selby, R. W. & Hutchens, D. H. (1986) 'Experimentation in Software Engineering'. IEEE Transactions on Software Engineering, SE‐12 (7), 733‐743.

Budd, C. I. & Budd, C. S. (2010) A Practical Guide to Earned Value Project Management, USA, Management Concepts, Inc. Cerpa, N. & Verner, J. M. (2009) 'Why Did Your Project Fail?'. Communications of the ACM, 52 (12), 130‐134.

Charmaz, K. (2011) Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis, London, SAGE Publications Ltd. Chrissis, M. B., Konrad, M. & Shrum, S. (2011) CMMI for Development: Guidelines for Process Integration and Product Improvement, USA, Addison Wesley.

Coleman, G. & O’connor, R. (2007) 'Using grounded theory to understand software process improvement: A study of Irish software product companies'. Information and Software Technology, 49 (6), 654–667.

Creswell, J. W. & Clark, V. L. P. (2011) Designing and Conducting Mixed Methods Research, USA, SAGE Publications Inc. Fleming, Q. W. & Koppelman, J. M. (2010) Earned Value Project Management, USA, Project Management Institute.

Glaser, B. G. & Strauss, A. L. (1967) The Discovery of Grounded Theory: strategies for qualitative research, USA, AldineTransaction.

Graham, D. R. (1992) 'Incremental Development and Delivery for Large Software Systems'. IEE Colloquium on Software Prototyping and Evolutionary Development. London.

Hoda, R., Kruchten, P., Noble, J. & Marshall, S. (2010) 'Agility in Context'. OOPSLA '10 Proceedings of the ACM international conference on Object oriented programming systems languages and applications. Reno/Tahoe, Nevada USA, ACM. Hughes, R. T. (2011) 'Observing Dispersed Agile Project Processes'. 10th European Conference on Research Methodology for Business and Management Studies. Normandy Business School, Caen, France.

Kim, E., Jr., W. G. W. & Duffey, M. R. (2003) 'A model for effective implementation of Earned Value Management methodology'. International Journal of Project Management, 21 (5), 375–382.

Kitchenham, B. A., Pfleeger, S. L., Pickard, L. M., Jones, P. W., Hoaglin, D. C., Emam, K. E. & Rosenberg, J. (2002) 'Preliminary Guidelines for Empirical Research in Software Engineering'. IEEE Transactions on Software Engineering, 28 (8), 721‐ 734.

Larman, C. & Basili, V. R. (2003) 'Iterative and Incremental Development ‐ A Brief History'. Computer, 36 (6), 47‐56.

Lehtinen, T. O. A. (2012). 'Perceived Feasibility of Using Root Cause Analysis in Post Project Reviews: an Empirical Investigation' [Online]. Available from: http://esem.cs.lth.se/esem2012/idoese/pdf/137_IDoESE_Lehtinen.pdf: Paper presented to International Symposium on Empirical Software Engineering and Measurement at Lund University.

[Accessed 24 Oct 2012].

373

Zana Ahmedshareef, Miltos Petridis and Robert Hughes Lehtinen, T. O. A. & Mäntylä, M. V. (2011) 'What Are Problem Causes of Software Projects? Data of Root Cause Analysis at Four Software Companies'. 2011 International Symposium on Empirical Software Engineering and Measurement (ESEM). IEEE Conference Publications.

Lipke, W., Zwikael, O., Henderson, K. & Anbari, F. (2009) 'Prediction of project outcome: The application of statistical methods to earned value management and earned schedule performance indexes'. International Journal of Project Management, 27 (4), 400–407.

Locke, K. (2001) Grounded Theory in Management Research, London, Sage Publications Ltd. Montoni, M. A. & Rocha, A. R. (2010) 'Applying Grounded Theory to Understand Software Process Improvement Implementation'. 2010 Seventh International Conference on the Quality of Information and Communications Technology.

PMI (2004) A Guide to the Project Management Body of Knowledge, USA, Project Management Institute Inc. Rabbi, M. F. & Mannan, K. O. B. (2008) ‘A Review of Software Risk Management for Selection of best Tools and Techniques’. Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, 2008.

SNPD '08. Ninth ACIS International Conference on 6‐8 Aug. 2008. IEEE Computer Society.

Redmill, F. (1992) 'Incremental Delivery ‐ not all Plain Sailing'. IEE Colloquium on Software Prototyping and Evolutionary Development. London.

Rose, J., Pedersen, K., Hosbond, J. H. & Kræmmergaard, P. (2007) 'Management competences, not tools and techniques: A grounded examination of software project management at WM‐data'. Information and Software Technology, 49 (6), 605–624.

Seaman, C. B. (Jul/Aug 1999) 'Qualitative Methods in Empirical Studies of Software Engineering'. IEEE Transactions on Software Engineering, 25 (4), 557‐572.

Taylor, H. (2006) ‘Risk Management and Problem Resolution Strategies for IT Projects’. Project Management Journal, 35 (5), 49‐63.

Verner, J. M. & Evanco, W. M. (2005) 'In–House Software Development: What Project Management Practices Lead to Success?'. IEEE Software, 22 (1), 86‐93.

374

Grounded Theory Application in Doctorate Research Sjoerd Gehrels Academy of International Hospitality Research, Stenden University of Applied Sciences, Leeuwarden, The Netherlands sjoerd.gehrels@stenden.com

Abstract: Researchers working towards a doctorate need to make choices and defining the appropriate methodological approach is one of them. Usually, the choices are made based on the suggestions of a supervisor. In other cases, however, such as in a highly reflexive set‐up required for an educational doctorate, there is more space for personal initiatives by the candidate. In this paper a case of doctorate research methodology is presented, which can serve as contemplation for fellow researchers who are preparing their study. An interaction of reflexive narrating and grounded theory methodology is applied. This approach incorporates personal and professional reflexivity to prelude a rigorous field study. The starting point of the research consisted of constructing narratives based on the researcher’s personal experiences and observations from ten years of professional practice. Next, the scarce specific literature about the phenomenon and the wider connected literature on entrepreneurship and leadership were reviewed. In the following field study, ten successful small upper segment culinary restaurant owners were asked in depth interviews about their experiences and perceptions of successful entrepreneurship. To validate the practitioners’ interviews, a research journalist and expert connected to the entrepreneurs’ community was interviewed. A constructivist grounded theory approach as proposed by Charmaz (2006) was employed, involving analysis moving from interview to interview. The information obtained from the first eight interviews subsequently formed the basis for defining code labels and categories that captured the entrepreneurs’ worlds.

After tentatively formulating a social construct, three additional interviews were done to engage in theoretical sampling, comparing and validating the findings. Revisiting the literature, while analysing the findings supported the refining of the concepts and emerging themes. The last three interviewees were selected because of their specific knowledge about the substantive field and/or higher level of abstraction. Although the chosen approach is neither the most efficient nor easy one for applying in doctorate research, it does provide an interesting alternative for more regularly applied designs. The findings of this study provided valuable insights into the nature of entrepreneurship in the culinary restaurant sector and generated the definition of a social construct, which attempts to capture the complex process leading to success. The distinctive nature of this research design lies in starting from a reflexive practitioner’s position and subsequently applying systematic grounded theory procedures in order to produce findings that enlighten the practices in a particular professional sector. This research example can serve as helpful for doctorate researchers who would aspire to explore and explain phases of their own personal and professional life with the aim to develop knowledge that can support practitioners.

Keywords: grounded theory, methodology, doctorate research

1. Introduction It is paramount in the life of a doctorate researcher to provide and apply an appropriate academic rigorous research design in order to progress in producing a high quality thesis. (Phillips & Pugh 2010) Making choices in terms of the methodological approach is part of the doctorate process. In situations where the doctorate candidate is directly connected to a research programme, the research design will in most cases be based on the format suggested by the professor as supervisor. There are, however, other types of doctorate tracks that allow the candidate to be more influential in the set‐up of the research. An example is the Educational Doctorate programme such as experienced by the author of this article. One of the attractive points for me as starting doctorate researcher was the structured starting two years of course work in which a lot of research input was given. Shaping the research was a process that gave me a lot of opportunities to contemplate about what would be yielding most value in terms of process and desired outcomes.

The set‐up required for the educational doctorate made me reflect on my practice in education and the possible research to be built on it. There was the possibility to set a personal direction in the research. From the ‘journey’ of designing and implementing the research many valuable insights emerged that I would like to share with my fellow researchers who are planning to start on doing doctorate research or who have already started but are struggling with design issues. The case of doctorate research methodology presented here might serve as contemplation for fellow researchers when preparing their study.

This research paper only deals with the research design but cannot be understood without accessing the original overall research format for which purpose I provide here the abstract of the doctorate research (table 1).

375

Sjoerd Gehrels Table 1: Abstract of original thesis research Abstract The aim of this research is to explore the contextual characteristics of a particular group of Dutch restaurant owner’s (the SSUSCROs 1 ) and practitioners, to examine how these contextual characteristics might be used in a professional hospitality education programme. This very small segment of the Dutch restaurant business (0,2‐0,5% of the total restaurants) is known for its strong commitment to competitiveness, in delivering quality service and products. No previous research in The Netherlands had embarked on a search for connecting this specific category of practitioners to education. As owners of their restaurants, the SSUSCROs were aware of the potential contribution that participating in this research would make. The research was designed from a constructionist epistemological point of view. This means that the data supplied by the respondents, and the background and vision of the researcher provided an interplay.

By using grounded theory methodology, theory is constructed from the empirical data. The main instrument for the primary research was in‐depth, interviewing. Six retired and four practising restaurant owners, and a connoisseur of the business were interviewed in one to three hour depth interviews that were digitally recorded. The transcripts of the recorded interviews were analysed, applying the specific constructivist version of grounded theory methodology as described by Charmaz’s (2006). The research generated a grounded theory in the form of a narrative about the SSUSCRO social construct and its central theme ‘Living the Business’. The narrative informs future practitioners i.e. students, about how they can prepare for possible future business ventures in the culinary restaurant business.

Furthermore, it confronts future practitioners with the notion of particular contextual characteristics and value systems that need to be incorporated in order to successfully engage in and sustain a career in the culinary restaurant sector. Elements of the narrative, connected to Covey’s 7‐Habits of Highly Effective People framework for personal leadership. The findings from this research confirmed the importance of providing students in hospitality management education with a approach towards professional development that is grounded in the social construct of a remarkable group of entrepreneurs such as the SSUSCROs. The conclusions suggested that faculty and academic management of hospitality management programmes need to become more knowledgeable about the particular nature of the discipline, and the specific category of practitioners researched here.

The research was initiated to see how the contextual characteristics of an extraordinary group of practitioners that I had experienced in the first ten years of my professional life could be analysed and explained in order to potentially influence education. In other words, the research was about how the ‘good’ and the ‘bad’ of a remarkable practice in hospitality management can be used to better prepare students, faculty and management of hospitality management educational programmes. To start on the research it would be crucial to engage in an interaction of reflexive narrating of my experiences in order to explain why there was merit to be found in the exercise. In the next sections I will present the design and analysis considerations.

2. Designing the research The successful small upper segment culinary restaurants presented a very dynamic segment within the hospitality industry that has traditionally been outside the academic research world. It was crucial to find and apply the appropriate research design to “delve” into the minds and practices of the entrepreneurs and then to use the findings to define ways in which hospitality management education potentially could be influenced.

A useful and straightforward framework to build the research process is offered by Crotty (2003). Crotty suggests to look at four elements or levels that constitute a research and therefore should be taken into account when doing research (figure 1).

Figure 1: Four basic elements research process (Crotty 2003)

1

Successful Small Upper Segment Culinary Restaurant Owners

376

Sjoerd Gehrels The four elements are: ‘epistemology’, ‘theoretical perspective’, ‘methodology’ and ‘methods’. In this model, epistemology deals with the nature of knowledge or as Rosenthal & Rosnow (2008) define it: ‘the origin, nature, methods, and limits of human knowledge or human understanding of the world’ (p. 744). The theoretical perspective deals with “how we know what we know” and is the philosophical stance informing the methodology. The methodology is the strategy, plan of action, process, or design that lies behind the choice and use of particular research methods. The methods are the techniques or procedures used to gather and analyse data in relation to some research question or hypothesis. Other authors have made similar suggestions in providing a conceptual and philosophical structure to the research process. Fisher (2004), in talking about the ‘isms’ and ‘ologies’, offers a framework in which the dimensions of the‘nature of knowledge (epistemology)’ and the relation between ‘knowledge and reality’ are plotted. DePoy & Gitlin (1998) make a distinction between philosophical foundations, research traditions, and design strategies. Flick (2006) compares the qualitative research perspectives ‘theoretical positions’, ‘methods of data collection’, ‘and ‘fields of application’, while Hennink, Hutter, & Bailey (2011) try to capture the approach to research under the term ‘paradigm’. Creswell (2009) speaks about three components involved in a research design: ‘philosophical worldview’, ‘selected strategies of inquiry’ and ‘research methods’.

In essence, the authors all suggest taking a reflexive approach to designing and implementing research in order to be able to properly argue why choices are made. In the reasoning for this specific thesis research, both the subject of research and the position and world view of the researcher were included. Although Crotty’s (2003) approach may be highly modelled with a clear cut and identifiable reality, this is not how it actually is because there is no strict linear relationship between the elements of the model. Easterby‐Smith, Thorpe, & Jackson (2008) suggest using a similar modelling as Crotty where they refer to ‘ontology’, ‘epistemology’, ‘methodology’, and ‘methods’. Easterby‐Smith et al. strongly alert the researcher to the issue of research design by saying: ‘that failure to think through philosophical issues … while not necessarily fatal, can seriously affect the quality of management research’ (p. 56. Although warning against firm and fixed picking of an epistemological stance from the beginning, Dyson & Brown (2006) acknowledge the potential advantage of approaching research by starting from a clear paradigm or epistemology that then drives decisions at the level of strategy or methodology, methods and analysis. Consequently there will at least be consistency imposed onto the research problem.

Miles & Huberman (1994) offer an interesting notion to loosen up any debate about epistemology where they assert that it is tempting in epistemological debates to operate on the poles, while in the actual practice of empirical research all researchers are closer to the center with multiple overlaps. By adhering to a certain model including an epistemological choice, the purpose is to provide framing and structuring to the research.

Before defining the four elements as defined by Crotty (2003), the aim, objectives and from these the resulting research questions were connected to the information required to answer the questions and to realize the objectives of the research. From that point on it was possible to properly define methodology and methods that were appropriate for the implementation of the research.

2.1 Epistemology and theoretical perspective My research nested in constructionism, which has been developed differently in many disciplines. Crotty (2003) defines constructionism as the epistemology situated at the middle position between the extremes: objectivism and subjectivism. In constructionism, the belief is that the essence of creating knowledge is not by finding an objective truth waiting for us to be discovered. In my choice for constructionism, I followed Flick (2006): ‘It informs a lot of qualitative research programs with the approach that the realities we study are social products of the actor, of interactions and institutions’ (p. 78). Knowledge is seen as being constructed in a process of social interchange between the subject and the object. Gergen (2009) confirmed that outside the academic world, constructionist ideas have stimulated an enormous expansion in forms of practice. It was important to define this research in the constructionist tradition. In this way, an objectivist and positivist approach were rejected that would traditionally start from the theory in the body of literature to produce hypotheses to be tested. The philosophy of objectivism claims that research should be value free and objective. In my research, theory building interacted with my own understanding and interpretation. In dealing with my research sample, reality was constructed and not value free. In terms of theoretical perspective in Crotty’s (2003) model this research is situated in the tradition of symbolic interactionism, which takes the perspective that reality is ‘negotiated between people’, ‘always changing’, and ‘constantly evolving’ (Richards & Morse 2007).

377

Sjoerd Gehrels

2.2 Methodology and methods Different methodologies appeared to be appropriate to study my sample of entrepreneurs’ characteristics but soon the notion of using grounded theory presented itself as appropriate. Grounded theory methodology was first presented by Glaser and Strauss (1967) and taken further by Glaser, Strauss, Strauss and Corbin, Charmaz, Clarke and many others. The essence of grounded theory is to derive theory from data systematically gathered and analysed in the research process (Strauss and Corbin, 1998). In this methodology, data collection, analysis, and eventually theory stand in close relationship to one another. There is not one universal way in which grounded theory can be applied. Soon after Glaser and Strauss described their research approach in the sixties, they both started diverting in different directions. Strauss and Corbin would allow the research problem to be defined at the start of a project, which was countered by Glaser as this would pre‐empt the emergence of theory (Fisher, 2004).

Charmaz (2006) and Clarke (2005) talk about “discovering” the theory in which process the researcher has to use reflexivity and personal interpretation to arrive at the theoretical implications. A researcher applying grounded theory does not generally begin a research project with preconceived theory in mind but rather begins with an area of study. In this research, the way my entrepreneurs deal with their social context and how that can be used in education defined the subject area. Grounded theory is a rather open approach to data analysis, which is particularly good for dealing with transcripts of depth interviews (Easterby‐Smith et al. 2008).

Rubin & Rubin (2005) support this approach, and assert that it requires openness to new ideas not anticipated at the beginning. There are critical observations about the application of grounded theory. The original epistemological grounded theory starting point was in positivism, which would not consider the reflexivity of the researcher involved, instead letting the data produce the theoretical implications through strict application of procedures. Constructivist grounded theory, developed by Charmaz (2006) positions the interplay between data and researcher to constructs the outcomes of the study. In table 2, the considerations for applying a constructivist grounded theory methodology are summarized.

Table 2: Motivation for applying constructivist grounded theory Consideratio n Kind of knowledge?

Assumptions about the world

Conceptualiz ation role researcher Mode of working in this research

Type of analysis Challenges

Application to the entrepreneur s research

Constructivist grounded theory Identifying and explaining contextualized social processes Concepts and categories emerge from the data and are identified by the researcher Produce theory grounded in the data Looks at how humans negotiate and manage social situations and processes Studies the changing world as a product of human participation and negotiation Focuses on process and change Subscribes to the symbolic interactionist perspective and relativist ontology (realities that are locally and specifically constructed or co‐constructed) Researcher actively constructs understanding about the phenomenon under investigation Questions participants to understand what they are doing and why Researcher tries to clarify own assumptions and expectations while analysing Research problem + opening research questions Data collection + initial coding Initial memo writing + raising codes to tentative categories Further data collection + focused coding Advanced memos + refining conceptual categories Theoretical sampling + seeking specific new data Adopting certain categories as theoretical concepts + theoretical memo writing and sorting + further refining of concepts Writing the first draft + further theoretical sampling if needed Writing the final grounded theory Iteratively moving between data, codes, concepts and categories to result in a theory.

Assumptions and background of researcher can be too dominant in the analysis if not properly acknowledged Study is very time consuming Constructivist GT seemed to be very appropriate to apply to the research questions raised about the entrepreneurs and the way that their characteristics could influence hospitality management education. Particularly the social processes that the entrepreneurs are involved in could be described and explained by adopting a CGT approach.

378

Sjoerd Gehrels I agree with Hennink et al. (2011) that grounded theory is an appealing approach as it provides a rigorous and scientific approach, yet it also remains faithful to the interpretive nature of qualitative analysis. Charmaz (2006) confirms qualitative interviewing to be an appropriate instrument for grounded theory. Interviewing is interactional because of the exchange or sharing, of roles, responsibilities, feelings, beliefs, motives and information (Stewart & Cash 2006), and it can take account of many characteristics of the respondent (Keats 2000). Rubin & Rubin (2005) emphasize the complex nature of interviewing as used in this research used as ‘responsive interviewing’. Critical comments about interviewing are made by Burton & Bartlett (2005). They warn the interviewer about possible influencing or leading the respondent, the time intensive nature of the method and the possible variation in interviews, which makes collating difficult.

Hammersley (2002), interestingly, defines the outcomes and importance of this type of highly context specific research as providing “moderate enlightenment”. Policymakers and practitioners are per definition “in the dark” and research is needed for them to shed light on the reality of what they are doing and how they should be doing it. Moderate enlightenment takes away the strong generalization claims that research makes in the natural sciences approach and their strong adoption of positivist viewpoints.

2.3 Sample and data collection The number of interviews followed saturation of the data, which occurs when each new act of data collection does not add anything new (Dyson & Brown 2006). Very important for me in this process of applying grounded theory was to get to the point where I achieved “theoretical saturation”. Theoretical saturation was originally introduced by Glaser & Strauss (1967), to identify the moment to stop sampling when the data led to no new findings being generated that would complete a certain category.

My sampling process confirmed findings from Guest et al. (2006) that the big concepts emerged after 6 interviews, while saturation occurred after 11 interviews. The questions for the interviews were formulated to encourage the interviewees to talk about their lives and profession in a flexible manner but without incurring the risk for me to forget to address certain topics connected to the research questions. I followed Rubin & Rubin’s (2005) suggestion to formulate main questions and then to follow up based on the answers given.

Sometimes interviewees, as predicted by Rubin & Rubin, could offer new ideas and relevant stories but in some cases they would omitt certain information.

3. Data analysis Essentially the data analysis in this grounded theory approach involves dealing with vast amounts of written findings (transcripts). The analysis of the first few transcripts started by coding the important and interesting issues, mentioned by the respondents. For this purpose, I created a three‐column table in which the first column held the transcript divided per question and answer, the second column held the interpretation that I took from the coded text and the third column contained the code label, which was used to identify patterns in the transcripts. From the ‘three column’ table in which I did the coding of the transcript in the first column, I then took the interpretations in the second column ( the ‘coded text column’) and collected them in a new table under the labels that were used in the third column of the original table (‘code label column’). Following this procedure, a new table was created that contained the interpretations of each interview and the labels I attached to it. The code labels were described with the meaning I gave them and with some of them a description was added as to how the literature defined the particular code label.

In my analysis of the transcripts, I simultaneously generated memos in which the contemplation, triggered by the interviewees’ story lines, materialized. Charmaz (2006) advises to start writing advanced memos after initial coding. I followed Saldana’s (2009) suggestion to see the writing of analytic memos as: ‘whenever anything related to, and significant about the coding or analysis of the data comes to mind, stop whatever you’re doing and write the memo’ (p. 33). In figure 2, the processs of the research is visualized.

While coding the interview transcripts, I continued to search the literature in order to validate my code label definitions, and to see if they would fit the body of knowledge. While defining the codes, I wrote memos that materialized my understanding and analytic threads induced throughout the working on the data. The whole defining the codes and writing memos, while my understanding and interpretation developed. Certain primary interpretations were modified by comparing to the content of later interviews and by reading the literature connected to the codes generated. This process was finalized after theoretical sampling, and collecting specific

379

Sjoerd Gehrels new data by integrating the memos and concepts. I progressed gradually towards intermediate coding which had the key task to link together and integrate categories of codes. After finishing the first interviews’ analyses, categories emerged that appeared to hold the codes and their meanings. I continued this process, of coding, and reflecting in the next interviews and it allowed me to further develop the codes, their meanings and the categories. Some codes were merged, while others were added and this process was taken further until the eight’s interview.

Figure 2: Constructivist grounded theory process in this thesis research It became clear that the entrepreneurs in my research were not only definable individual personalities but were people embedded in their social context. Their lives were the result of a complex set of person related factors, people around them (partner, family, staff and others), activities and circumstances. The choices they made were in many cases influenced by certain happenings or people. In the process of simultaneous data gathering and analysis, slowly but steadily a social construct of entrepreneurship in this research became clear.

Also the influence that this phenomenon, defined as social construct, could have on hospitality management education started to emerge.

380

Sjoerd Gehrels The entrepreneurs’ characteristics, as a person, were of importance. Analysing their personalities and to see what made them to who they were, was an important element in the coding. Furthermore, it was important for me to find out how their socialisation processes had taken place. There had to be an influence of the family they came from, the education they had followed and the experiences both before and throughout their entrepreneurship. In order to find out what had made them into the people they were, I asked my interviewees to describe their lives in restaurant entrepreneurship and to indicate where they perceived had been the significant moments, interventions and decisions. While they described their lives, I could insert follow‐up questions in order to get more in‐depth and to find out if their descriptions remained at the surface or if more content rich meanings could be uncovered. I was alert to the entrepreneurs perception of what they considered ‘to be successful’. With this exploration of the term ‘successful’, I looked for indicators and features in both their professional and private lives. In order to summarize and understand the process of analysis, the schematic presentation from codes to theory by Saldana (2009) provided clarity.

Figure 3: Codes‐to‐theory model qualitative inquiry (Saldana, 2009, p. 12) It needs to be noted here that the model presentation as offered by Saldana shows the streamlined, ideal, and maybe somewhat linear flow from codes to the eventual theory. The actual process was a lot messier than suggested. Categories did not emerge nicely and neatly from the codes as might be suggested in figure 3. The iterative process took time and involved going back and forth between the data, initial thoughts and writing of memos.

4. Conclusion My approach and design are relatively underexplored in the field of hospitality management research, in which I am a practitioner. Authors situated in the objectivist, positivist, and often in their own perception scientific lines of thought, are easily tempted to discard my research methodology. Hair jr, Money, Samouel, & Page (2007) talk about the characteristics of the “good” scientific method being objective and logical. In my research design, however, I start from a reflexive personal and therefore subjective narrative based on my experience and observations. My opinion as a researcher had an influence on the process of interpreting the findings although I acknowledge when and where it impacted. Hair jr et al. (2007) assert that research should be theory driven, and built on the literature. Although I confirm the importance of adding to the body of knowledge, I took a more iterative approach in which the literature and the ‘voices of the respondents’ went “hand in hand”. The result was a grounded theory that explained an important entrepreneurs’ social construct in‐depth.

The power and impact of this constructivist grounded theory design lie in it’s firm grounding in the lived social worlds of the actors involved. The rigorous design with a fully traceable journey through the data and the analysis process garantuees the academic quality of the research needed to operate at doctorate level. My experience in applying this constructivist grounded theory methodology for doctorate study brings me to suggest to other researchers in similar settings to consider adopting the same approach. I conclude by agreeing with Urquhart (2013) that grounded theory has a unique power when correctly applied. It was revolutionary in its time and still tremendously relevant today as I found out in my doctorate research. A next direction I will explore is about how to apply this grounded theory approach in more concise studies such as at master level.

381

Sjoerd Gehrels

References Charmaz, C. (2006) Constructing Grounded Theory, A Practical Guide Through Qualitative Analysis, Sage Publications Ltd, Thousand Oaks.

Clarke, A. (2005) Situational Analysis: Grounded Theory After The Postmodern Turn, Sage Publications Ltd, Thousand Oaks.

Creswell & J. (2009) Research Design: Qualitative, Quantitative and Mixed Methods Approaches, 3rd ed, Sage Publications Ltd, Thousand Oaks.

Crotty, M. (2003) The Foundations of Social Research Process: Meaning and Perspective in the Research Process, Sage Publications Ltd, Thousand Oaks.

DePoy, E. & Gitlin, L. (1998) Introduction to Research: Understanding and Applying Multiple Strategies, 2nd ed, Mosby, Inc, St.Louis.

Dyson, S. & Brown, B. (2006) Social Theory and Applied Health Research, Open University Press, Maidenhead.

Easterby‐Smith, M., Thorpe, R. & Jackson, P. R. (2008) Management Research, 3rd ed, Sage Publications Ltd, London.

Fisher, C. (2004) Researching and Writing a Dissertation for Business Students, Pearson Education Limited, Harlow.

Flick, U. (2006) An Introduction to Qualitative Research, 3rd ed, Sage Publications Ltd, London.

Gehrels, S.A. (2012) The Contextual Characteristics of Successful Small Upper Segment Culinary Restaurant Owners and their Potential Influence on Hospitality Management Education, Unpublished EdD Thesis, University of Stirling, Stirling.

Gergen, K. (2009) An Invitation To Social Construction, 2nd ed, Sage Publications Ltd, London.

Glaser, B. G. & Strauss, A. L. (1967) The Discovery of Grounded Theory, Aldlin, Chicago.

Guest, G., Bunce, A. & Johnson, L. (2006) "How Many Interviews Are Enough?: An Experiment with Data Saturation and Variability", Field Methods, 18(1), pp. 59‐82.

Hair jr, J., Money, A., Samouel, P. & Page, M. (2007) Research Methods for Business, John Wiley & Sons Ltd, Chichester.

Hammersley, M. (2002) Educational Research, Policy Making and Practice, Paul Chapman Publishing, London.

Hennink, M., Hutter, I. & Bailey, A. (2011) Qualitative Research Methods, Sage Publications Ltd, London.

Keats, D. (2000) Interviewing: Practical Guide for Students and Professionals, Open University Press, Buckingham.

Miles, M. & Huberman, A. (1994) Qualitative Data Analysis: An Expanded Sourcebook, 2nd ed, Sage Publications Ltd, Thousand Oaks.

Phillips, E. M. & Pugh, D. S. (2010) How To Get a PhD: A Handbook for Students and their Supervisors, 5th ed, Open University Press, Buckingham.

Richards, L. & Morse, J. (2007) Readme First for a User's Guide to Qualitative Methods, Sage Publications Ltd, Thousand Oaks.

Rosenthal, R. & Rosnow, R. (2008) Essentials of Behavioral Research: Methods and Data Analysis, 3rd ed, McGraw‐Hill, New York.

Rubin, H. & Rubin, I. (2005) Qualitative Interviewing: The Art of Hearing Data. 2nd ed, Sage Publications Ltd, Thousand Oaks.

Saldana, J. (2009) The Coding Manual for Qualitative Researchers, Sage Publications Ltd, London.

Stewart, C. & Cash, W. (2006) Interviewing: Principles and Practices, Mc Graw‐Hill, New York.

Strauss, A. & Corbin, J. (1998) Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, Sage Publications Ltd, Thousand Oaks.

Urquhart, C. (2013) Grounded Theory for Qualitative Research: A Practical Guide, Sage Publications Ltd, London.

382

Crowdsourcing: Personal Values and Self‐Efficacy as Antecedents to Creative Behavior Susana Morais and Isabel Ramos Algoritmi Center, Universidade do Minho, Guimarães, Portugal andreamorais_@hotmail.com iramos@dsi.uminho.pt

Abstract: Creative crowdsourcing and crowdsourcing for innovation initiatives (hereinafter called CCI) are a business model that allows companies to leverage the crowd’s skills and know‐how through online platforms where the individuals (solvers) voluntarily supply them with ideas and solutions. CCI initiatives are similar to open contests and depend heavily on submissions that actually solve the posted challenges. Despite the frequent use of incentives (financial and others) CCI dynamics relies above all on the individuals’ predisposition and motivations. Understanding the impact of personal values and self‐efficacy on this kind of initiatives will decrease the level of uncertainty caused by the lack of any previously relation, link or contact between the companies and the solvers (which are anonymous). Certain studies point to a correlation among some values and self‐efficacy with creativity. While it is difficult to change personal values, self‐efficacy is a well‐documented approach to change behaviors. In order to answer the research question “How do the motivational antecedents affect the participants’ creative behavior in CCI initiatives?” the research will pursue the following topics: ‐ assess solvers’ values; ‐ evaluate their self‐efficacy levels; ‐ establish connections between the solvers’ values and self‐ efficacy. Given the need to capture the phenomena as a whole this investigation will adopt a qualitative approach based on netnography methodology. Considering the research question and the theoretical background based on values and self‐ efficacy, it will adopt a preliminary step comprising an anonymous online questionnaire issued to creative solvers. The research will follow with online interviews hoping to seize the interviewees’ experiences, in order to draw all possible insights related with their CCI initiatives engagement.

Keywords: crowdsourcing, innovation, creativity, self‐efficacy, values

1. Introduction CCI innovation initiatives are a specific kind of crowdsourcing whose activities or tasks – usually complex – seek unique solutions or ideas. Companies, instead of allocate internal resources to accomplish some of their creative or innovative needs, may choose a CCI approach which is a business model that allows them to fulfill those purposes with a considerably lower effort and investment (cost, time and personal).

Usually through an open call, companies (also known as seekers) publish their problems and appeal for the crowd’s skills and know‐how: the individuals of the crowd (the solvers) – who are anonymous – voluntarily supply them with ideas and solutions.

CCI frequently uses financial and other material incentives. Nevertheless, CCI dynamics depends mostly on the individuals’ predisposition and motivations (Bayus 2010; Cheshire 2008; Leimeister 2009; Lakhani 2007; Malone 2009; Preece et al 2009).

Creativity is one of the core issues in CCI domain since solvers are supposed to supply seekers with novel and useful ideas or solutions. Creativity is a well‐studied field and it focuses mainly on tasks’ performance.

Therefore, creativity is very much related to the individuals’ motivations. (Amabile 1996; Miron et al 2003; Shalley et al 2004)

Some of the research on CCI domain follows the usual background on open source research thus focusing on motivations. However, according to the literature, motivations fail to clarify CCI cases since they happen to be very similar across so different initiatives (Bayus 2010; Brabham 2008; Lakhani 2007; Leimeister 2009; Oreg 2008) Furthermore, some of the CCI motivations’ research found counterintuitive effects between rewards and creativity, a phenomenon also perceived and thoroughly discussed on psychology and behavioral economics (Amabile 1996; Ariely 2005; Frey 2001).

Personal values also comprise motivational characteristics however in a more lasting manner. Moreover, drawing on Schwartz Human Values Theory (1994), they sustained widely no matter the culture, and they also give a great understanding on motivational behaviors.

383

Susana Morais and Isabel Ramos Whereas personal values are long‐lasting and hard to change, self‐efficacy is a well‐documented approach to change behaviors and a very reliable measure of one’s effort and engagement in an activity. Self‐efficacy (Bandura 1997) is the self‐belief of being able to perform something therefore deeply related to concrete tasks.

Finding the interplay between values and self‐efficacy will provide an effective way to improve CCI online environments since self‐efficacy beliefs can be triggered from external sources (incentive, feedback and reputation mechanisms) and used to change solvers’ behaviors aligning them with the CCI initiatives’ goals.

2. Defining crowdsourcing Crowdsourcing is a growing business strategy that makes use of the global connectedness network of human resources available through the web. The underlying idea beneath this knew way to get work done is that anywhere, at any time, someone unknown with some kind of skills and some extra time is willing to do some tasks and activities, not just for the rewards but also for other diverse kind of motives.

Jeff Howe (2008), who popularized the term, divided crowdsourcing into four main categories based on the crowd’s type of contribution:

Crowd wisdom comprises initiatives carried out to obtain knowledge outside the organizations’ boundaries.

Crowd creation is related to user‐generated content which can be used as a business strategy to find new ideas and solutions.

Crowd voting covers all the initiatives where users rate or vote for something.

Crowdfunding: is a fast‐growing trend which relies on monetary crowd contributions to specific projects intending to match potential lenders to entrepreneurs with financial needs.

Tasks that can be crowdsourced range from simple to complex ones. Simple tasks are often related with actions like label, classify, identify. They are usually timeless and reward with very small payments. Companies use this strategy to achieve quickly a large amount of information. On the contrary, complex tasks demand for more cognitive effort, skilled behaviors and are typically related with higher rewards. Between simple and complex tasks there are a lot of activities’ possible combinations with moderate rewards and that’s where creative and innovative tasks commonly remain (Schenk et al 2011).

Material or financial rewards are not an absolute requirement for any of those different kinds of tasks.

Crowdsourcing cases as Wikipedia illustrate non‐rewarding initiatives covering non‐routine tasks. Other kind of non‐rewarding and non‐routine tasks are scientific one’s – a.k.a. citizen science – where users (amateurs and non‐professional scientists) are supposed to help scientific research (e.g. gather or analyze data). Dictionaries, translators and other similar tools combine machine‐learning algorithms with users’ feedback making it possible to digitize large amounts of words in an accurate and timely manner.

CCI initiatives encompass a diverse kind of creative and innovative tasks relying mainly on submissions’ quality and on solvers’ creativity (Schenk et al 2011).

3. Using netnography to study crowdsourcing phenomena Netnography was coined by Kozinets as the online version of ethnography and it is a method especially used on marketing domain to gain insights on consumer behaviors thus suitable for a cultural‐oriented study of online interactions (Kozinets 2001).

This methodology is appropriate for the present research as it allows collecting and analyzing data from crowdsourcing initiatives in a flexible, multi‐method and iterative manner requiring constant reflection and modifications when necessary. (Kozinets 2001; Loureiro et al 2012)

Netnography offers means to witness how things naturally occur, allowing different levels of observations according to the researchers’ degree of immersion. This is a method which entails researchers to live or work

384

Susana Morais and Isabel Ramo like those they’re studying (Kozinets 2001) also with the purpose of assessing the veracity of the participants’ testimonials (Loureiro et al 2012).

Considering that crowdsourcing comprises not only a new online cultural phenomenon but also an organizational one, netnography is perfectly suited to study this particular crowd behavior and ultimately answer the research questions.

According to Kozinets’s guidelines, netnography can be planned on the following phases:

learn about the research phenomenon, formulate the research questions and define the source

observe and interact with individuals

analyze and interpret, in an iterative way, through coding practices; synthetize and contextualize

research ethics: disclose research identity, ensure individuals anonymity, give research feedback and obtain permission before publishing results

give feedback to individuals who participated in the research

write the findings, reporting process, difficulties and limitations

Netnography methodology put emphasis on immersion and ethical issues on using online data. Since the present research doesn’t have access to the solvers’ historic data, their anonymity is granted by default. The first data collection (a questionnaire) will openly address this issue.

The subsequent data collections will be based on interviews which are also intended to adopt the same overt approach. Online interviews, without a face‐to‐face interaction, may fail some non‐verbal cues but, on the other hand, interviewees may feel less uncomfortable since they don’t see the interviewer.

During the online interviews, it is expected to establish some sort of connections with the interviewees: one of the researchers has also some solver’s experience which can facilitate some understandings.

4. Research design This research will adopt the Netnography methodology, a qualitative approach which allows the construction of “grounded knowledge” inductively derived from the reality. This methodological approach is suitable because CCI is a new study domain with unique characteristics through which it is aspired to develop a theoretical explanation based on empiric observations.

On CCI context, creative tasks are a critical issue. Creativity is the ability to generate new and potentially useful ideas (Amabile 1996) resulting from a combination of personal and contextual factors (Amabile 1996; Miron et al 2003; Shalley et al 2004). As a personal characteristic, creativity is deeply related to one’s motivations (Amabile 1996; Miron et al 2003; Shalley et al 2004); besides, contextual factors such as the way tasks are assigned has great impact on individuals’ engagement and their creative performance.

As already mentioned, motivations do not enlighten CCI domain since they posit counterintuitive effects when creativity and rewards (material or not) are at stake (Amabile, 1996; Ariely, 2005; Deci et al l 2008, 2000; Frey 2001).

Looking for more stable motivational constructs, personal values (Schwartz 1994) stretch a long‐lasting and comprehensive view of human behavior. According to Schwartz’s theory, values occur in a continuum and dichotomy way: one cannot pursue some value and the opposite one at the same time but he is instead very likely to pursue one more similar value.

As proposed by Kasof et al (2007) “creative behavior is related holistically to the entire integrative dynamic pattern of value types identified by Schwartz (1994)”. Following their findings creative behavior is actually promoted by the self‐direction, stimulation and universalism value types and additionally the creative behavior is inhibited by the tradition, conformity and security value types (Kasof et al, 2007). On another empirical

385

Susana Morais and Isabel Ramos research (Dollinger et al, 2007) creativity appears once more as a core value significantly correlated with self‐ direction, universalism and stimulation Schwartz’s values.

Self‐efficacy is a motivational construct able to assess the way solvers interact with tasks. Self‐efficacy “influences individual choices, goals, emotional reactions, effort, coping, and persistence.” (Gist et al 1992).

Moreover, following Tierney et al (2002), self‐efficacy beliefs also influence employee creativity, since the results of their empirical study point to a correlation between self‐efficacy and creativity.

Reasoning from the integration of the reviewed literature on creativity, values and self‐efficacy and considering the research problem the empirical work will be implemented as follows.

This research is still at an early stage. In order to answer the research problem, the empirical main goal is to examine whether certain motivational characteristics do make people more creative, how and why through a combination of a survey and netnography techniques.

The first stage is based on a preliminary case study. Following a netnography plan, since research question is already defined, next step consists on selecting the CCI initiative to be studied. This is an ongoing phase: this preliminary case study will be undertaken on a Portuguese CCI initiative chosen due to the CEO readiness and for its representativeness of this kind of platforms. This choice was made after several CCI initiatives’ observation and after one of the researcher’s online participation as a solver (this engagement took place already with research purposes since it was undertaken to try out all the process flow and related events). The researchers’ degree of involvement will evolve from a combination between observation and participation, synchronous and asynchronous.

The questionnaire (issued only to creative solvers) will have a twofold role: it will be an overt approach to anonymous respondents with a view to asking for their additional availability for a future interview; at the same time, it will be used to gather data in order to triangulate later with those interviews’ content.

The questionnaire includes: the full version of the Schwartz’s Portrait Values Questionnaire which comprises 21 questions about personal values; 15 self‐efficacy questions built following Bandura’s orientations (1997); finally, 8 socio‐demographic questions and related matters, such us some about the use of computer and CCI initiatives.

Due to policy limitations (access to solvers’ personal information is denied for legal issues), the mechanism to elicit questionnaire responses will be an e‐mail for all the solvers who submit a solution to the selected CCI initiative, inviting them to answer the questionnaire, participate on interviews or to do both of them ‐ questionnaire respondents and interviewees will self‐select themselves for the purpose.

The questions of the interview will be outlined according to the research questions. The first interview will be guided by open questions which will allow collecting a testimony of the participants’ own experience (Grams 2001). The analysis and open coding of each identified idea in this first collection will enable to narrow the questions according to it.

This iterative process will be repeated until the emerged concepts stand an overall explanation of the phenomenon. The systematic collect and analysis of data will occur simultaneously. The interviews should be as many as necessary to clarify all the emerged ideas and until the end of the appearance of new relevant concepts (Kozinets 2001).

One of the netnography’s key issues is about the researcher’s identity since it’s possible to obtain a lot of online data without any permission. In the present research all the gathered data will result from an overt approach.

Although the respondents have non‐verifiable identities the reliability of their responses can be considered trustworthiness: they engage on questionnaires and interviews after participating on a CCI initiative and all of them in a voluntary manner. Since any of those actions assume the expense of some time and effort it is reasonable to consider them reliable.

386

Susana Morais and Isabel Ramo Since the selected CCI initiative faithfully reproduces that category of crowdsourcing it is also expectable to obtain accurate and consistent data. Recognizing the limitations of just one source of data, it is intended to replicate this design case study a few more times. Additionally it is being developed a website specifically for the present research purposes hoping to reach solvers from other diverse sources (and not just from specific case studies) with the purpose to cover a broad range of respondents for a later results’ comparison

5. Conclusions Companies and organizations may undertake a CCI approach by two means: broadcast their challenges on intermediation platforms or, alternatively, on platforms of their own. Any of these strategic approaches cope and may benefit from the insights on solvers’ behavior since they will allow aligning feedback and reward mechanisms to their predispositions.

Connecting this work to research focused on other important actors on this domain (such as the crowdsourcing intermediaries) will provide practical utility and an added value to the companies that are directed towards adopting CCI as a business strategy.

From this research, it is expected to achieve an in‐depth understanding on CCI initiatives and to obtain empiric evidences on the existence of psychological background of the individuals with influence on their creative performance.

This conviction is reasoned from studies that point to a correlation among certain types of values, self‐efficacy and creativity. The practical implications of these results are the opportunity of employing incentive mechanisms fitter to the predominant crowd’s profile and promoters of creative, effort and persistence behaviors.

Acknowledgements This study is carried out under the scope of the Algoritmi Center research where other crowdsourcing investigations are being done and to whom it is deserved to thank.

This work is funded by FEDER funds through the Operational Competitiveness Programme ‐ COMPETE and also by national funds through FCT – Fundação para a Ciência e Tecnologia under the Project: FCOMP‐01‐0124‐ FEDER‐022674.

References Amabile, T. (1996) “Creativity and innovation in organizations”, Harvard Business School, 9‐396‐239 Ariely, D., Gneezy, U., Loewenstein, G. and Mazar, N. (2005) “Large Stakes and Big Mistakes”, Research Center for Behavioral Economics and Decision‐Making, Working Paper 05‐11, Federal Reserve Bank of Boston Bandura, A. (1997) Self‐efficacy: the exercise of control, W.H. Freeman and Company, New York Bayus, B. (2010) “Crowdsourcing and individual creativity over time: The detrimental effects of past success” Brabham, D. (2008) “Moving the crowd at iStockphoto: The composition of the crowd and motivations for participation in a crowdsourcing application” Cheshire, C. and Antin, J. (2008) “The Social Psychological Effects of Feedback on the Production of Internet Information Pools”, Journal of Computer‐Mediated Communication, 13, p.705‐727 Deci, E. and Ryan, R. (2008) “Self‐Determination Theory: A Macrotheory of Human Motivation”, Development, and Health.

Canadian Psychology, Vol. 49, No. 3, p.182–185 Dellarocas, C. (2010) “Online Reputation Systems: How to design one that does what you need”, MIT Sloan Management Review, Vol.51, No. 3.

Dollinger, S., Burke, P. and Gump, N. (2007) “Creativity and values”, Creativity Research Journal, 19 (2), p. 91‐103 Ford, C. (1996) “A theory of individual creative action in multiple social domains”, Academy of Management Review, Vol. 21. No. 4, 1112‐1142 Frey, B. and Jegen, R. (2001) “Motivation Crowding Theory”, Journal of Economic Surveys, 15(5), p.589‐611 Grams, D.G. (2001) “Rudiments in the use of “Grounded Theory”: a working guide”, Working draft unpublished manuscript Gist, M. and Mitchell T. (1992) “Self‐Efficacy: A Theoretical Analysis of Its Determinants and Malleability” The Academy of Management Review Vol. 17, No. 2 (Apr., 1992), pp. 183‐211 Hars, A. and Ou, S. (2002) “Working for free? Motivations for participating in open‐source Projects”, International Journal of Electronic Commerce, 6, 25‐39 Howe, J. (2008) Crouwdsourcing, Edição Actual Editora, Lisboa Kasof, J., Chen, C., Himsel, A. and Greenberger, E. (2007) “Values and creativity”, Creativity Research Journal, 19 (2), p. 105‐ 122

387

Susana Morais and Isabel Ramos Kozinets, R. (2001) “The Field Behind the Screen: Using Netnography For Marketing Research in Online Communities” Lakhani, K., Jeppesen, L., Lohse, P. and Panetta, J. (2007) “The Value of Openness in Scientific Problem Solving” Harvard Business School Working Paper 07‐050 Leimeister, J., Huber, M., Bretschneider, U. and Krcmar, H. (2009) “Leveraging Crowdsourcing ‐ Theory‐driven Design, Implementation and Evaluation of Activation‐Supporting Components for IT‐based Idea Competitions”, Journal of Management Information, Vol. 26, No. 1, p. 197–224 Loureiro‐Koechlin, C. and Córdoba‐Pachón, J. (2012) “Assessing online ethnography as a form of interpretive research in information systems”, School of Management, Royal Holloway, University of London, Working Paper Series SoMWP– 1201 Malone, T., Laubacher, R. and Dellarocas, C. (2009) “Harnessing Crowds: Mapping the Genome of Collective Intelligence”, MIT Center for Collective Intelligence, Working Paper No. 2009‐001 Oreg, S. and Nov, O. (2008), “Exploring motivations for contributing to open source initiatives: the roles of contribution context and personal values”, Computers in Human Behavior 24, 2055–2073 Preece, J. and Shneiderman, B. (2009) “The reader to leader framework: Motivating technology‐mediated social Participation”, AIS Transactions on Human‐Computer Interaction, 1,1 (2009), 13‐32 Ryan, R. M. and Deci, E. L. (2000) “Intrinsic and extrinsic motivations: Classic definitions and new directions”, Contemporary Educational Psychology, 25 (2000) 54–67 Schenk, E. and Guittard, C. (2011) “Towards a characterization of crowdsourcing practices”, Journal of Innovation Economics, 2011/1 n°7, p. 93‐107 Schwartz, S. (1994) “Are There Universal Aspects in the Structure and Contents of Human Values?”, Journal of Social Issues, Vol. 50, No. 4, pp. 19‐45 Shalley, C., Zhou, J. and Oldham, G. (2004) “The effects of personal and contextual characteristics on creativity: where should we go from here?”, Journal of Management 2004 30(6) 933–958.

Tierney, P., and Farmer, S. M. (2002) “Creative self‐efficacy: Potential antecedents and relationship to creative performance”, Academy of Management Journal, 45: 1137–1

388

Work in Progress Papers

389

390

Influenza Pandemic Outbreaks: Conducting Focus Groups with Rural Communities in Baram, Sarawak Constance Rinen Justin Wah and Rohaya Mohd‐Nor Department of Business Management, Faculty of Economics and Business, Universiti Malaysia Sarawak, Kota Samarahan, Malaysia rjwconstance@feb.unimas.my mnrohaya@feb.unimas.my

Abstract: Our research aims to explore rural communities’ understanding, awareness and attitude towards H1N1 Influenza pandemic outbreaks and prevention measures. We attempt to ascertain their views with regard to the effective use of various media platforms in educating them about the pandemic influenza outbreaks. In 2012, a series of focus group sessions was conducted in Baram, Sarawak. We assess our choice of methodology and approaches in setting‐up and conducting these focus groups. Based on our experience, we found that cultural factor, language barrier and level of education (among others) are some issues that a researcher has to consider prior to conduct a focus group to obtain good quality data from the rural participants.

Keywords: H1N1, focus groups, rural communities, Baram and Sarawak

1. Introduction In Malaysia, the first pandemic H1N1 outbreak was identified between April and September 2009 (Wong and Sam, 2011). Since then, the Ministry of Health in Malaysia has been proactive in setting up communication platforms to disseminate up‐to‐date information relevant to H1N1 risks and prevention measures. In Sarawak, the first incidence of the influenza outbreak through local transmission was recorded on July 5th 2009 in Bintulu, followed by the first H1N1 related death reported on 4th of August 2009 (Then, 2009). While information about influenza outbreaks and its preventive measures continue to be disseminated via various communication media, but it is not known which of these media are effective in enhancing the rural communities’ understanding and awareness about the disease. In addition, there has been a serious lack of empirical research to assess how their understanding with regard to influenza risks and the prevention measures is constructed and contributing to the positive health practices.

The objectives of this study are twofold. First, we attempt to explore the rural communities’ attitude towards H1N1 pandemic outbreaks and prevention measures. Second, we attempt to find their views on the effective use of various media platforms to educate them about the outbreak. To undertake the study, we deployed qualitative research methods, namely focus groups and survey interview. To date, we have conducted a series of focus groups with the rural participants from Serian and Baram in Sarawak (Figure 1 depicts the location of Baram in Sarawak). In this paper, we present our reflections on conducting the focus groups with the rural participants in Baram. We assess our choice of methodology and approaches in setting‐up and conducting these focus groups.

2. Focus group as our methodological choice Focus group is an appropriate research method if one attempts to explore in what ways some ideas, knowledge as well as experience develop and function within a given cultural setting (Kitzinger, 1994, 1995). It is also a method to obtain qualitative data particularly about participants’ perceptions on a specific topic of interest in their natural setting (Krueger and Casey, 2009). This is an extremely useful method in obtaining data as it does not exclude illiterate people and provides a comfortable setting to encourage their participation among the group under study. Robinson (1999) defined a focus group as ‘an in‐depth, open‐ended group discussion of 1‐2 hours’ duration that explores a specific set of issues on a predefined and limited topic. It can be viewed as a form of group interview where the focus is on the research participants’ interactions (Ktizinger, 1995). In a focus group session, the participants are encouraged to interact with each other by exchanging opinions and giving comments. (Kitzinger, 1995). Such interaction can clarify the participants’ frameworks of understanding, their attitude as well as encouraging them to discuss embarrassing issues or subjects openly (Kitzinger, 1995). Robinson (1999) proposes the number of participants within a group should be between five to eight participants and a facilitator or a moderator will conduct the session. Others suggest that the number of participants could be between 6 and 12 participants (Morgan, 1997; Stewart and Shamdasani, 1990).

391

Constance Rinen Justin Wah and Rohaya Mohd‐Nor In the context of our study, using focus group method is a preferred choice because:

It permits us to obtain useful data from the rural participants where the quantitative method such as survey is not feasible.

We have anticipated that there will be rural participants who are illiterate, and can only converse in their local language. As our research focuses on obtaining their views as well as understanding about their attitude in dealing with the influenza risks and health practices, focus group inarguably is a feasible choice.

We found that the rural participants were more relaxed and did not feel embarrass to put forward their views about H1N1. Most of the participants also relied on the comment from other participants to enable them to understand better on the issue that were discussed in the session. Several participants found it difficult to express their understanding about “What is H1N1?” However, after listening to the group conversations (i.e., also their friends), then they could respond to question. It was an interesting experience to observe the group interactions where their responses reflected the communal lifestyle and thinking. We also obtained rich data about the extent of their understanding and what contributed most to influence their approaches towards positive healthcare practices.

It enables us to understand the interaction and communication styles of these communities, and obtains a deeper understanding about the impact of individual role in the family unit as well as in the community context.

In Sarawak, the rural communities of Baram are known locally as a quiet community. We observed that most of the participants particularly female were very shy and quiet. Our observation on the group interactions really enhances our understanding about the community interaction and communication styles. For example, through their responses on the question “If there is an outbreak, how do you prevent from getting the flu?,” we managed to obtain data about what channels can be considered effective to enhance the rural communities’ understanding and awareness about the disease outbreaks.

It allows us to gather the rural participants who come from different ethnic group, age, gender and social class in the community within just a short period.

Through the assistance of the community leader of Long Bedian and Long San, we managed to recruit the participants. These participants come from different tribal group, age, gender as well as social class.

3. Conducting focus groups in rural Baram An Overview of the Research Setting

Baram

Figure 1: Location of Serian and Baram in Sarawak, Malaysia (Google Maps, 2013) Sarawak is located on the island of Borneo and famously known as “Land of the Hornbills”. Based on the published statistics (Malaysia Department of Statistics, 2010), Sarawak population in 2010 is 2,399,839 million.

Sarawak is also the largest state in Malaysia with a recorded land area about 124, 449.51km2 (Sarawak Government, 2010). The three largest ethnics of Sarawak are Iban, Chinese, and Malay; then followed by Bidayuh, Orang Ulu – Kayan, Lun Bawang, Kelabit, Kenyah, Penan, Bisaya ‐, Melanau and others. Currently, Sarawak consists of 11 divisions. Baram is under Marudi District.

We conducted focus groups at two locations in rural Baram namely Long Bedian and Long San. The journey to reach these places was very challenging due to the road condition. The road caters largely for heavy vehicles suited for logging activities. Long Bedian has a population of a slightly more than 1500 people. The Long Bedian

392

Constance Rinen Justin Wah and Rohaya Mohd‐Nor community is largely from Kayan ethnic. Other ethnic groups of Long Bedian include Kelabit, Kenyah, Morek, Penan and others. At Long Bedian, we conducted three focus group sessions concurrently. At Long San, we managed to conduct only one focus group session.

How Did We Set‐up the Focus Groups?

Getting the consent from the community leader is the first step in setting up the focus group and seek their support. Prior to our visit, we inform the community leaders and district officer. We engaged the community leader directly to help us to recruit the participants. At Long Bedian, the community leader permitted us to use the community hall to conduct the focus groups. However, in Long San, we conducted a focus group at the community leader’s house since there was no appropriate place for us to use such as a community hall.

Based on our experience, we found that the following is useful to do:

Appropriate time: We recommend after 7pm onwards is the suitable time to gather the participants for the focus groups. During daytime, the communities will be busy with their daily activities such as farming, hunting or finding forest products.

Suitable location: It was highlighted in the literature that the success of any focus group is really depending on how the researcher manages the logistic issues and location (Health Community Unit, 2002). The chosen location should be accessible and familiar to the participants in so that they can feel comfortable and relaxing.

How Did We Conduct the Sessions?

In the literature, it was highlighted that the role of facilitator and transcriber is important in obtaining quality data from focus groups. It was further suggested that the facilitator has a similar background with the participants of a focus group (Fern, 2001). In the context of our study, the facilitators that we have in our study were from the Department of Nursing who are also the Sarawak locals and very familiar with local language.

The focus group sessions went smoothly because these facilitators are very knowledgeable in the area of curbing H1N1 influenza infection and promoting positive healthcare practices.

The following steps were undertaken when conducting the focus groups in Baram:

The group members prepared the seating allocation and identified the suitable facilitator based on the social‐demographic of the participants. Since most of the participants were female, we also probed deeper on their roles in the communities in preventing the spread of the disease within the family unit.

Based on our experience, we found that female participants were more comfortable to discuss sensitive issues with the female facilitator. We replicated few good approaches of our previous experience to ensure our success in obtaining good data. We also prepared in advance the probing questions and these questions were asked in a very structured way.

At the beginning of the session, the moderator personally greeted each of the participants and thanked them for their participation. The moderator then introduced herself/himself and followed by the transcriber. All the discussion was recorded in audio tape and consent were obtained from the participants before this could be done. The audio device was put in the center of the table to ensure that the participants’ voices are clear in the recording. The moderator then asked each of the participant to write down their nickname in a piece of paper. As several of our participants were illiterate, we assisted them resulting on more conducive environment for discussion too. The moderator then further explained the purpose of our study by using simple language.

During the focus group discussion, the moderator has to play crucial role in establishing trust with the participants. Gaining trust is important in motivating the participants to talk. The moderator also has an important role to observe how participants felt about certain ideas and opinions expressed by other participants in the group (Health Community Unit, 2002). Moreover, the moderator must be skillful and tactful in questioning. At the end of the session, we presented each of the participants with a token of honorarium to express our appreciation for their time and to compensate for any cost incurred in attending the event. The small amount of money can cover the transportation cost as some of the participants live at different long houses that were quite far away from the venue. The duration of the focus group sessions were between 50 minutes and 90 minutes.

393

Constance Rinen Justin Wah and Rohaya Mohd‐Nor

After the session completed, we also took the opportunity to fulfill the request by the community leader to conduct a health talk and respond to several inquiries from the community.

4. Conclusion Conducting focus groups that involved rural participants can enrich a researcher understanding about the importance of engaging the community leaders and the communities in order to successfully conduct the focus groups. The rich data that were obtained from focus groups are the outcome from an established trust between the researchers and the rural participants. In the context of our study, no doubt the focus group method is the best option to obtain rich data about rural communities’ understanding, awareness and attitude towards H1N1 Influenza pandemic outbreaks and prevention measures.

Acknowledgments This research is funded under the Fundamental Research Grant Scheme FRGS/05 (32)/804/2011 (73) and is supported by the University Malaysia Sarawak.

References Bammer, F. D.L., Naoom, S. F., Blasé, K. A., Friedman, R. M., and Wallace, F. (2005) Implementation Research: A Synthesis of the Literature (FMHI pub. No. 231). Tampa, FL.: University of South Florida, Louis de la Parte Florida Mental Health Institute, National Implementation Research Network.

Berg, B.L. (1998) Qualitative Research Methods for the Social Sciences (Third Edition) (Boston: Allyn and Bacon).

Calder, B.J. (1977) “Focus Groups and the Nature of Qualitative Marketing Research.” Journal of Marketing Research. Vol 14, No 3. pp 353‐364.

Freitas, H., Oliveira, M., Jenkins, M., and Popjoy O. (1998) The Focus Group, A Qualitative Research Method. ISRC, Merrick School of Business, University of Baltimore (MD, EUA), WP ISRC No. 010298, p. 22.

Google Maps (2013) [Baram, Sarawak, Malaysia] [Street Map] Retrieved from [online], http://maps.google.com.my.

Kitzinger, J. (1994) The Methodology of Focus Groups: The Importance of Interaction Between Research Participants.

Sociology of Health & Illness, Vol 16, No. 1, pp 103‐121.

Kitzinger, J. (1995) Qualitative Research: Introducing Focus Groups. BMJ, 311: p 299.

Krueger, R.A., and Casey, M.A. (2009) Focus Groups: A Practical Guide for Applied Research (4th Ed.).Thousand Oaks, CA: Sage Publications.

Malaysia Department of Statistics. (2010) Basic Population Characteristics by Administrative State. [online], Government Statistics, http://www.statistics.gov.my.

Malaysia Department of Statistics. (2010). Vital Statistics. Retrieved from [online], http://www.statistics.gov.my.

Malaysia Department of Statistics. (2010). Population Distribution and Basic Demographic Characteristic Report. [online], Government Statistics, http://www.statistics.gov.my.

Malaysia Ministry of Health (2013) Health Advisory. Influenza H1N1 in the Inter‐Pandemic Period. Retrieved from [online], http://h1n1.moh.gov.my.

Morgan, D.L. (1997) Focus Groups as Qualitative Research. Thousand Oaks, Ca.: Sage.

The Borneo Post Online. (2011) Keep a Good Eye on H1N1. Retrieved from [online], http://www.theborneopost.com/2011/01/30/keep‐a‐good‐eye‐on‐h1n1/.

The Star. (2009) Cases of a H1N1 in Sarawak. Retrieved from [online], http://www.thestar.com.my/.

Sarawak Government (2010) The Official Portal of the Sarawak Government, the Geography of Sarawak, Retrieved from http://www.sarawak.gov.my/en/about‐sarawak/geography.

Stewart, D.W. and Shamdasani, P.N. (1990) Focus Groups: Theory and Practice. Newbury Park, Ca.: Sage.

Workbook of Health Communication Unit Using Focus Groups, (2002) Centre for Health Promotion University of Toronto Version 2.0.

Wong, L.P., and Sam, I‐Ching. (2011) Behavioral responses to the Influenza A(H1N1) outbreak in Malaysia. J Behav Med, Vol 34, pp 23‐31.

394

Use of a Learning Management System in Undergraduate Business Communications Courses Arlene Nicholas Salve Regina University, Newport, USA arlene.nicholas@salve.edu

Abstract: Does the current college population embrace the use of technology for classwork? They may be devoted users of text messaging, Facebook and even Google, but do they take advantage of the learning tools created for their specific coursework? Beyond the cost for the university to have licensing for the software, there is considerable effort of faculty and technology staff to create, upload, troubleshoot and maintain these educational support areas. Does it advance the students’ knowledge? Is it a useful tool to understand course objectives? A small case study of students in a northeast liberal arts university, business majors or minors, who used a learning management system (LMS) in their Business Communications course was conducted. Analysis of this study and other recent research in this rapidly growing pedagogical method will be reported.

Keywords: learning management systems (LMS), hybrid, blackboard, online, business communications

1. Background The use of LMS in higher education has expanded to support online teaching which, for some institutions, includes distance learning. Faculty usage in colleges that offer bachelor’s degrees has increased from 25% in 2005 to almost 40% in 2009 (Arroway, Davenport, Guangning, & Updegrove 2010). There are many versions of LMS software including Blackboard, Canvas, Moodle, Angel, Desire2Learn, Tyco and Efrontlearning. Faculty are encouraged to enhance their courses with LMS and many schools have added support technology departments, such as CTLs (Center for Teaching and Learning) for faculty and student assistance.

Attracting/retaining students with technology, having multiple methods of interaction, an area for notes for students’ review and for addressing any universal design needs for various types of learners are driving goals of utilizing LMS. It also allows possible online/distant education methods. LMS could also be used simply for syllabus and study guides, or discussions, self‐assessment tests, quizzes, grade posting and group communications (Clark 2002). The students in this study were able to access PowerPoint chapter slides, instructional articles and sites, the class syllabus, required discussions and wiki group work.

1.1 Recent studies Student satisfaction with an LMS has been correlated with course content (Malikowski 2008; Naveh, Tubin, & Pilskin 2010; Sun, Tsai, Finger, Chen, & Yeh 2008) and with course discipline (Naveh, Tubin, & Pilskin 2010).

Bikrimirov and Klassen (2008) found that student performance in an introductory hybrid finance course was significantly related to access to homework solutions on WebCT. In a study of diverse international accounting students, high levels of satisfaction using an LMS was associated with five variables: lecture notes availability, online assessment, announcements, model answers, and online chat (Basioudis, De Lange, Suwardy, & Wells 2007). In another study of 111 business students in the southeastern United States, the results were neutral in general satisfaction with the hybrid learning of 60% class time and 40% online, however, high levels of satisfaction with prompt feedback, active learning and faculty contact were reported (Young & Duhaney 2008).

1.2 This study The hybrid business communication courses in this study were comprised of 80% in class work and 20% usage of the online LMS. The face‐to‐face component and use of the Internet accessible Blackboard LMS course companion had graded elements. For example, out of 100%, in‐course attendance and participation in class had a 15% possible grade and three online discussions and initial wiki group interaction were a possible 6% and 2% respectively. Therefore, 8 points of 100 was possible for graded work through LMS online asynchronous interaction. Many resources, such as chapter slides from the publisher, links for websites of project resources such as http://businessculture.com and http://www.travlang.com, examples of cover letters, resumes, executive summaries, mission statements, logo creation and other course material that aided team or individual assignments, were on the LMS.

395

Arlene Nicholas The Business Communications for Success book was accessible for free online from the publisher, Flatworldknowledge, and for $40 hardcopy from the bookstore or the publisher. The access to the online book was a separate web site from the Blackboard LMS.

Students were asked how many times they logged in to the LMS, other than for graded discussion or the wiki.

The faculty dashboard of recent student activity was also assessed for general usage. Other questions investigated the usefulness of posted material including, slides, format outlines, discussion articles, links to research sites and other instructional material for the course project and the wiki group work area.

2. Methodology Students in two Business Communications courses were surveyed online through items created on SurveyMonkey. The respondents included one freshman, twelve sophomores, thirteen juniors and two seniors; seventeen had used a LMS before the course and twenty were also using it for another course. A section of the survey included the validated student engagement instrument by Handlesman, Briggs, Sullivan and Towler (2005). Students were e‐mailed the survey link through the LMS course mailing list which sent it to their school e‐mail account. Of the 50 students in the two classes, 28 (56%) responded. The survey allowed qualitative data to be collected in comment sections provided.

3. Results The majority of the respondents, 96.4%, used the LMS for their Business Communication course, 71.4% also used it for another course and 60.7% had used an LMS before this current course. This indicates at least familiarity with login and basic features, but did not ascertain experience with discussion and the group wiki features. The majority of respondents noted accessing the LMS several times a week (see Table 1 below).

Table 1: Access of LMS More than once a day Once a day Once a week Several times a week Every other week Once a month Less than once a month

10.7% 10.7% 25.0% 35.7% 21.4% 7.1% 0%

Specific usage of the LMS had mostly positive experiences; the exception to this was wikis (see Table 2 below).

Table 2: Using LMS features Please check any that apply to using an LMS has increased my understanding of the course material found sample papers helpful powerpoints were good guide for chapter topics mandatory use of wiki was a good learning experience should not have to use group wiki wiki was difficult used another group tool (please note in other) online discussions for assignments were easy to access and complete online discussions enhance subject material

Strongly agree

Agree

Disagree

Strongly disagree

21.4%

60.7%

14.3%

3.6%

28.6%

60.7%

10.7%

0.0%

28.6%

46.4%

21.4%

3.6%

14.3%

35.7%

42.9%

7.1%

14.3%

28.6%

46.4%

10.7%

0.0%

35.7%

50.0%

14.3%

14.8%

40.7%

29.6%

14.8%

21.4%

67.9%

10.7%

0.0%

17.9%

46.4%

32.1%

3.6%

396

Arlene Nicholas

3.1 Wikis The use of wiki was the most problematic. Students were evenly split that usage was valuable. Yet, students understood that it was important to use a wiki for group work. Qualitative responses from five subjects stated that Facebook group message and one noted google docs group tool was the main method of online collaboration for their group project. This corroborated the study results of Gillingham and Molinari (2012) who found that students regarded wikis as less useful than general courseware, discussions and videos, and attributed this attitude to possible unfamiliarity with the collaborative project tool.

3.2 Performance dashboard The performance dashboard is available only to the instructor and depicts recent log‐in of students in the LMS. The dashboard was shown to the classes so students were aware that their interaction was monitored. At the end of the semester, a range of 0 – 48 days was recorded for 50 students; the most common days since activity were 4‐5. Fourteen students had not accessed it in over 20 days and five had not accessed it in over 10 days.

3.3 Discussion questions Access for the purpose of reviewing material, could have been possible with students viewing the LMS together on a one name log‐in; however, the three discussion questions, spaced throughout the semester, required the individual student to log in to earn credit. Of the 50 students, there were two students who did not do any discussions nor get any grade, a possible 6 points, for the assignment. Eleven students only answered one of the three questions; seventeen answered two questions and twenty participated in all three discussions. It is not surprising that the overview of student usage results in the performance ‘dashboard’ correlated with higher performance. Regular usage of the LMS resulted in higher grades in most cases.

Students strongly agreed/agreed (64.3%) that the discussions enhanced course material and were easy to use (89.3%).

3.4 Student engagement To assess the students’ perceptions of their own attitudes toward coursework, the “Student Engagement Survey” instrument of Handlesman et al. (2005) was incorporated. From the results of this survey, students generally think their effort and keeping up with the readings is, at the least, moderately characteristic of them with the majority ranking both these items as characteristic/very characteristic. Only a small percentage, 17.9%, did not seem comfortable with asking questions when they did not understand the instructor. An even smaller percentage, 10.7%, admitted to not looking over class notes and another 10.7% stating it was not really characteristic of them.

4. Future studies and conclusion Courses will continue to incorporate technology to captivate and educate students. Group tools and other online interactions are critical for business communications. Class lectures to the Business Communication students emphasized the need to understand virtual team work and online media as it could be a requirement of future employment. Do students resist online work? For example, the book for this course was available free online but only 39% used it exclusively online, 53.6% used only hard copy versions and 7% used both the hardcopy and online versions. Undergraduate students may be considered part of the Net Generation, but motivating them to study online may take skills from the faculty regarding the design of the course.

Perhaps more engagement could be promoted through synchronous (real time) communication. The asynchronous option in this study gave students flexible choices for times to perform activities and ‘real time’ work was available in face to face class time. However, from a study by Cao, Griffin and Bai (2009), it was suggested that synchronous interaction should be incorporated in face‐to‐face courses as it was a preferred option by the 102 students in their study. Future research could include integration of synchronous tools, such as graded quizzes and instant messaging, to measure student engagement.

Hybrid and online learning offer important accommodating options for future coursework. Understanding what features are required for both faculty and student success with web‐based LMS will be critical.

397

Arlene Nicholas

References Arroway, P., Davenport, E., Guangning, X., & Updegrove, D. (2010) EDUCAUSE Core Data Service Fiscal Year 2009 Summary Report. http://net.educause.edu/ir/library/pdf/PUB8007.pdf Basioudis, I., De Lange, P., Suwardy, T., & Wells, P. (2007) Accounting Students’ Perceptions of a Learning Management System: An International Comparison. Proceedings AAFAANZ Conference, Gold Coast, Australia.

Biktimirov, E.N. & Klassen, K.J. (2008) Relationship Between Use of Online Support Materials and Student Performance in an Introductory Finance Course. Journal of Education for Business, Vol 83 No. 3, pp 153‐158.

Cao, Q., Griffin, T.E., & Bai, X. (2009) The Importance of Synchronous Interaction for Student Satisfaction with Course Web Sites. Journal of Information Systems Education, Vol 20 No. 3, pp 331‐ 338.

Clark, J. (2002) A Product Review of WebCT. The Internet and Higher Education, No. 5, pp 79‐82.

Gillingham, M. & Molinari C. (2012) Online Courses: Student Preferences Survey. Internet Learning, Vol 1 No. 1, pp 36‐45.

Handlesman, M.M, Briggs, W.L., Sullivan, M. & Towler, A. (2005) A Measure of College Student Course Engagement. The Journal of Educational Research, Vol 98 No. 3, pp 184 ‐191.

Malikowski, S. (2008) Factors Related to Breadth of Use in Course Management Systems. Internet and Higher Education, Vol 11 No. 1 pp 81‐86.

Naveh, G., Tubin, D. & Pliskin, N. (2010) Student LMS Use and Satisfaction in Academic Institutions: The Organizational Perspective. The Internet and Higher Education, Vol 13 No. 3 pp 127‐133.

Sun, P., Tsai, R. J., Finger, G., Chen, Y. & Yeh, D. (2008). What Drives a Successful E‐learning? An Empirical Investigation of the Critical Factors Influencing Learner Satisfaction. Computers & Education, Vol 50, pp 1183‐1202.

Young, A. E. & Duhaney, D. C. (2008). Hybrid Learning and the Principles of Good Practice in Undergraduate Education.

International Journal of Instructional Technology and Distance Learning, Vol 5 No. 5 http://www.itdl.org/Journal/Nov_08/article04.htm

398