Volume XI - Numer I by Laboratório de Tecnologia e Sistemas de Informação

JISTEM JOURNAL OF INFORMATION SYSTEMS AND TECHNOLOGY MANAGEMENT REVISTA DE GESTÃO DA TECNOLOGIA E SISTEMAS DE INFORMAÇÃO

www.jistem.fea.usp.br

ISSN: 1807-1775

Volume 11 : Number 1 : 2014

Available Online Disponível Online Support:

JISTEM Journal of Information Systems and Technology Management Revista da Gestão da Tecnologia e Sistemas de Informação ISSN online: 1807–1775

Every four months/Quadrimestral

Universidade de São Paulo – FEA USP Prof. Dr. Marco Antonio Zago – USP Reitor/Rector Prof. Dr. Vahan Agopyan – USP Vice-Reitor/Vice-Rector Prof. Dr. Reinaldo Guerreiro - Diretor da FEA/Dean of FEA Editor Prof Edson Luiz Riccio, PhD. University of São Paulo – FEA, São Paulo Brazil Assistant Editor Marici Gramacho Sakata, PhD. TECSI University of São Paulo – FEA, São Paulo Brazil Editorial Board – Comitê de Política Editorial Armando Malheiro da Silva, University of Porto, Porto, Portugal Christophe Benavent, Université Paris Ouest Nanterre La Defense, Paris, France Henrique Freitas, Federal University of Rio Grande do Sul, Rio Grande do Sul, Brazil JaeJon Kim, Chonnam National University, Gwangju, Korea Luc Marie Quoniam, University Paris 8, Paris, France Michael D. Myers, University of Auckland, Auckland, New Zealand Miklos Vasarhelyi, Rutgers Business School, New Jersey, USA Rejane Maria da Costa, University of Brasilia, DF, Brazil Robert D. Galliers, Bentley College, Massachusetts, USA Editorial Review Board – Comitê Científico Editorial Adam Mazurkiewicz, Instytut Technologii Eksploatacji, Poland Adalberto A. Fischmann, University of São Paulo, São Paulo, Brazil Antonio Carlos dos Santos, Federal University of Sao Carlos, São Carlos, Brazil Birger Hjorland, Royal School of Lis, Copenhagen, Denmark Burak Arzova, Marmara University, Istanbul, Turquia Dennis F. Galletta, University of Pittsburgh, Pittsburgh, USA Emerson Maccari, Uninove, Sao Paulo, Brazil Fabio Frezatti, University of São Paulo, São Paulo, Brazil Fernando Colmenero Ferreira, University of Madeira, Madeira, Portugal Geraldo Lino de Campos, University of Sao Paulo, Sao Paulo, Brazil Gilson Schwartz, University of Sao Paulo, Sao Paulo, Brazil Guilherme Ari Plonski, University of São Paulo, São Paulo, Brazil Jan Capek, Univerzita Pardubice, Pardubice, Czech Republic Jose Dutra de Oliveira Neto, University of Sao Paulo, Sao Paulo, Brazil José Rodrigues Filho, Universidade Federal da Paraíba, Paraíba, Brazil Miguel Juan Bacic, University of Campinas, Campinas, Brazil Napoleão Verardi Galegale, Centro Paula Souza and Galegale Associados, Sao Paulo, Brazil Rosana Grillo Gonçalves, University of Sao Paulo, São Paulo Brazil Salvador Ruiz-de-Chavez, APCAM, Ciudad de Mexico, Mexico Published by TECSI - Laboratório de Tecnologia e Sistemas de Informação - Revista de Gestão da Tecnologia e Sistemas de Informação - EAC FEA USP Av. Prof. Luciano Gualberto, 908 FEA 3, Cidade Universitária - São Paulo/SP 05508-900 Brasil Fone: 55-11-3091 5820 r.190 Fax: 55-11-3091 5820 jistem@usp.br Indexation/Directories SciELO, Latindex, Proquest, Ulrich's Periodical Directory, DOAJ, The Index of Information Systems Journals, ACPHIS, Dialnet, Ebsco, Gale Infotrac, CLASE, Portal de Periódicos USP, Qualis CAPES, Cabell's Directory Webmaster jistem@usp.br Technical Support Equipe TECSI pesquisatecsi@usp.br Terms and Conditions The license lets others distribute, remix, tweak, and build upon your work, even commercially, as long as they credit you for the original creation. This is the most accommodating of licenses offered. Recommended for maximum dissemination and use of licensed materials. Direitos e Permissão Os artigos são de total responsabilidade dos autores e todos os direitos reservados ao TECSI. Esta licença permite que outros distribuam remixem e construam sobre a sua obra, mesmo comercialmente, desde que lhe deem crédito pela criação original.

Support:

JISTEM Journal of Information Systems and Technology Management Revista de Gestão da Tecnologia e Sistemas de Informação Vol. 11, No.1, Jan/Apr. 2014, pp. 1-235 ISSN online: 1807-1775

Volume 11: Number 1 / Volume 11: Número 1 Content / Índice

2014 1-2 3-4

Editorial 1

Discovering Citizens Reaction Toward E-Government: Factors in EGovernment Adoption Mohammad Kamel Alomari, College of Business and Economics, Qatar University, Doha, Qatar

The Future of Audit Danielle Lombardi, Villanova University, Pennsylvania, United States, USA Rebecca Bloch, Fairfield University, Connecticut, United States, USA Miklos Vasarhelyi, Rutgers University, New Jersey, Unites States, USA

21-32

Factores Críticos de Éxitos en Busca de la Ventaja Competitiva dentro del Comercio Electronico: un Estudio Empírico en las Empresas Paraguayas

33-52

5-20

Critical Success Factors for a Competitive Advantage in Electronic Commerce: Empirical Study in Paraguayan Companies Chap Kau Kwan Chung, Universidad Americana, Asunción, Paraguay Rosario García Cruz, Universidad de Sevilla, Sevilla, España 4

Convergence in Information and Communication Technology (ICT) using Patent Analysis Eunhee Kim, Chonnam National University, Buk-gu, Gwangju, Korea Jaejon Kim, Chonnam National University, Buk-gu, Gwangju, Korea Joon Koh, Chonnam National University, Buk-gu, Gwangju, Korea

53-64

Use of RFID Technology to Overcome Inefficiencies in the Production Process: an analysis of a Microcomputer Company in Ilhéus – Bahia, Brazil Frederico Wergne de Castro Araújo Filho, Integrated Center of Manufacture and Technology – National Service of Industrial Learning – DR, Salvador, Bahia, Brazil X. L. Travassos, Integrated Center of Manufacture and Technology – National Service of Industrial Learning – DR, Salvador, Bahia, Brazil Paulo S. Figueiredo, Integrated Center of Manufacture and Technology – National Service of Industrial Learning – DR, Salvador, Bahia, Brazil

65-84

Medición de Usabilidad de Trámites Públicos en Línea en Chile: Un Caso de Estudio en Gobierno Electrónico Usability Measure of Chilean Public Online Services: an E-Government Case Study Marcelo Garrido Palma, AMABLE - Laboratorio UX, Santiago, Chile Claudio Lavín Tapia, CEAR, Facultad de Psicología, Universidad Diego Portales, Santiago, Chile Nelson Rodríguez-Peña, AMABLE – E-gob., Santiago, Chile

R. Gest. Tecn. Sist. Inf. /JISTEM Journal of Information Systems and Technology Management, Brazil

85-104

2 Content / Indice

Technological Innovation Projects: Proposal For an Integrative Model Between Project Management and Knowledge Management in a Customer-Supplier Perspective Projetos com Inovação Tecnológica: Proposta de um Modelo Integrativo entre Gerência de Projetos e Gerência do Conhecimento numa Perspectiva Cliente-Fornecedor Edval da Silva Tavares, Escola Superior Nacional de Seguros, Funenseg, São Paulo, Brasil Marcelo Schneck de Paula Pessoa, University of São Paulo, POLI/USP, São Paulo, Brazil

105-130

Method for Measuring the Alignment Between Information Technology Strategic Planning and Actions of Information Technology Governance Lúcio Melre da Silva, Catholic University of Brasilia, Brasília, Distrito Federal João Souza Neto, Catholic University of Brasilia, Brasília, Distrito Federal

131-152

Benefits of the Marketing Information System in the Clothing Retail Business Josimeire Pessoa de Queiroz, Centro Universitário da FEI, São Paulo, Brazil Braulio Oliveira, PPGA Centro Universitário da FEI, São Paulo, Brazil

153-168

Evaluation of Management Control Systems in Higher Education Institution with the Performance Management and Control Avaliação dos Sistemas de Controle Gerencial em Instituição de Ensino Superior com o Performance Management and Control Ilse Maria Beuren, Federal of Parana, Parana, Brazil Silvio Aparecido Teixeira, Regional University of Blumenau, Santa Catarina, Brazil

169-192

Eliciting and Defining Requirements Based on Metaevaluation: the Case of the CRAS 2008 Census Elicitação e Definição de Requisitos Baseada em Meta-Avaliação: O Caso do Censo CRAS 2008 Edilson Ferneda, Catholic University of Brasilia, BSB, Distrito Federal, Brasil Hércules Antonio do Prado, Catholic University of Brasilia, BSB, Distrito Federal and Empresa Brasileira de Pesquisa Agropecuária, Brasil Marcelo Gomes Gadelha, Ministério do Desenvolvimento Social e Combate à Fome, BSB, Distrito Federal, Brasil Helga Cristina Hedler, Catholic University of Brasilia, Distrito Federal, Brasil

193-214

Repertory Grid, Laddering e Análise de Conteúdo: Uma abordagem Psicométrica para Entender Profissionais de Tecnologia da Informação Repertory Grid, Laddering, and Content Analysis:a Psychometric Approach to Understand the Information Technology Workforce Pedro Jácome De Moura Junior, Universidade Federal da Paraíba, PPGA/UFPB, MPGOA/UFPB, PPGCI/UFPB, Brasil Carlo Gabriel Porto Bellini, Universidade Federal da Paraíba, PPGA/UFPB, MPGOA/UFPB, PPGCI/UFPB, Brasil

215-232

Events / Eventos Contributions / Submissão de Artigos

233 234-235

JISTEM, Brazil Vol.11, No.1, Jan/Apr. 2014, pp. 01-02

www.jistem.fea.usp.br

JISTEM Journal of Information Systems and Technology Management Revista de Gestão da Tecnologia e Sistemas de Informação Vol. 11, No.1, Jan/Apr. 2014, pp. 03-04 ISSN online: 1807-1775 DOI: 10.1590/S1807-17752014000100000

Editorial Edson Luiz Riccio, Editor

Dear Readers, In this first issue, January – April of 2014, we are pleased to present a selection of 12 high-quality articles. We take this opportunity to welcome authors from Qatar, the USA, Paraguay, Spain, Korea, Chile and Brazil. We are continuously working to make JISTEM a multidisciplinary and international journal. We focus on presenting key and innovative articles originated from grounded and theoretical research, developed in different countries and regions with a diversity of perspectives. With this issue we believe to achieve this objective again. I believe that commenting each article is unnecessary because each title is selfexplanatory and it has also the purpose of awakening the curiosity of the readers in exploring each one of them. We thank all authors for their confidence, time, and dedication to the review process and for attending to all our requests. Our sincere wishes for a pleasant reading of the articles.

R. Gest. Tecn. Sist. Inf. /JISTEM Journal of Information Systems and Technology Management, Brazil

4 Content / Indice

Editorial Edson Luiz Riccio, Editor

Caros leitores

Nesta primeira edição, Janeiro-Abril de 2014 temos o prazer de apresentar uma seleção de 12 artigos da mais alta qualidade. Nesta edição, damos as boas-vindas aos autores do Catar, dos Estados Unidos, do Paraguai, da Espanha, da Coréia, do Chile e do Brasil. Estamos trabalhando continuamente para fazer da JISTEM uma revista multidisciplinar e internacional. Concentramo-nos em apresentar artigos importantes e inovadores originados de pesquisas de campo e/ou teóricas, desenvolvidas em diferentes países e regiões e com diversidade de perspectivas. Com esta edição acreditamos alcançar novamente este objetivo. Acredito que é desnecessário comentar cada artigo porque cada título é autoexplicativo e tem também o objetivo de despertar a curiosidade dos leitores em explorar cada um. Agradecemos a todos os autores por sua confiança, tempo, dedicação ao processo de revisão e por atender a todos os nossos pedidos. Nossos sinceros votos de uma boa leitura dos artigos. Edson Luiz Riccio - Editor Chefe

JISTEM, Brazil Vol.11, No.1, Jan/Apr. 2014, pp. 03-04

www.jistem.fea.usp.br

JISTEM - Journal of Information Systems and Technology Management Revista de Gestão da Tecnologia e Sistemas de Informação Vol. 11, No. 1, Jan/Apr., 2014 pp. 05-20 ISSN online: 1807-1775 DOI: 10.1590/S1807-17752014000100001

DISCOVERING CITIZENS REACTION TOWARD EGOVERNMENT: FACTORS IN E-GOVERNMENT ADOPTION Mohammad Kamel Alomari College of Business and Economics, Qatar University, Doha, Qatar _____________________________________________________________________ ABSTRACT E-government has been considered as one approach for changing the face of government in the eyes of the citizenry. Therefore, citizens’ socialization in relation to their engagement with e-government should be explored. This study argues that citizens played a significant role in determining the success of an e-government project in the Middle Eastern country of Jordan. This paper aims to provide insight and evaluation into the factors that could influence e-government’s effective functioning in the Jordanian social community through its interaction with citizens. The study collected data from 356 Jordanian citizens via a survey, to ascertain their understanding of 10 factors that may influence their intention to use e-government services. To investigate the adoption of e-government services in depth, two departments in Jordan were selected: the Jordanian Government’s Income and Sales Tax Department (ISTD) and its Driver and Vehicle Licenses Department (DVDL). The factor analysis technique was used to identify the main factors related to e-government services’ adoption. The results indicated that trust in the internet, website design, religious beliefs, internet and computer skill confidence, word of mouth, resistance to change, perceived usefulness, relative advantage and complexity are the main factors that should be considered when addressing the topic of e-government services’ adoption in Jordan. This study is different from most existing studies on e-government adoption as it empirically investigated the impact of word of mouth (WOM), wasta (favoritism), and resistance to change on e-government adoption. This study highlights the importance of considering the social cohesion of the Jordanian community when exploring factors related to egovernment adoption. Keywords: Technological innovation, government, citizens, social cohesion

e-government,

adoption,

services,

survey,

_____________________________________________________________________________________________ Manuscript first received/Recebido em: 13/05/2013 Manuscript accepted/Aprovado em: 13/02/2014 Address for correspondence / Endereço para correspondência Mohammad Kamel Alomari. College of Business and Economics, Qatar University, Doha, Qatar. Dr. Mohammad Alomari is an assistant professor at Qatar University. He received his PhD in Information Systems from Griffith University researching in the area of E-government adoption. E-mail: m.alomari@qu.edu.qa Published by/ Publicado por: TECSI FEA USP – 2014 All rights reserved.

6 Alomari, M. K.

INTRODUCTION

Governments worldwide are implementing and supporting a variety of electronic initiatives such as e-government, e-commerce and e-business. E-government has been developed by governments to improve the efficiency and effectiveness of internal operations and processes, public communication with the citizenry and engagement in transactional processes with individual and organizational elements (Warkentin et al., 2002). For the purpose of the current research paper, e-government was identified as a Mechanism through which government services are produced and delivered to citizens utilizing Web-based internet applications. The government in Jordan is one of the governments that decided to implement e-government. It has introduced e-government as one of the initiatives for creating a knowledge-based society (MoICT, 2006a). In 2006, the Jordanian e-Government Strategy ensured e-government’s commitment to a customer-centric approach and considered citizens as a core concern when introducing its public services. Delivering enhanced and faster public service through e-government became one of the Jordanian government’s obligations. Yet, it has been demonstrated that the lack of citizencentricity in the implementation of e-government was one of the main challenges of egovernment implementation in Jordan (MoICT, 2006b). Therefore, it is necessary to question whether or not citizens intend to use e-government services and to accept this new form of interaction with the government. Previous research has found that the success of e-government implementation was dependent not only on government support, but also on citizens’ willingness to accept and adopt e-government services (Alomari et al., 2012). Therefore, it was necessary to obtain and evaluate Jordanian citizens’ perceptions about e-government as a change in their lives and therefore to clarify their response to government in relation to this new technological initiative. This study has shown that the response consisted of a combination of the following factors that need to be scrutinized when highlighting the topic of e-government services’ adoption, particularly in a Middle Eastern country like Jordan: trust in the internet, website design, religious beliefs, internet and computer skill confidence, word of mouth (WOM), resistance to change, Wasta (favoritism), perceived usefulness, relative advantage, complexity. The paper is organized as follows. First, the paper presents previous research work about e-government adoption. After the research’s theoretical framework was delineated, data for this study were collected using a questionnaire and were analyzed using factor analysis: significant ﬁndings were then discussed and the study’s conclusion was presented. Finally, this paper by collecting new data sets, based on a new sample and new research questions, follows a new approach extending previous research work (Alomari, 2010; Alomari et al., 2010, 2012).

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 05-20

www.jistem.fea.usp.br

Discovering citizens reaction toward E-government: factors in E-government adoption

BACKGROUND

E-government like other technological innovations such as e-business, ecommerce and internet banking is an internet-driven activity. Moon (2002) declared that “… the idea of e-government followed the private-sector adoption of so-called ebusiness and e-commerce” (p.425). Fang (2002) stated that: “e-business and ecommerce are subsets of e-government” (p.2) Previous studies on e-government were administered based on analytical and evaluation research conducted in the areas of ecommerce and e-business (Alomari et al., 2012; Carter and Bélanger, 2005; Ebrahim and Irani, 2005; Pons, 2004). On researching e-government in Arab countries, Pons (2004) reported “we believe that the issues involving e-commerce can be applied to egovernment in order to predict the concerns and problems of the technology” (p.31). The following paragraphs report on the previous main studies which have explored factors related to the adoption of e-government with indications that the research was steered towards addressing the same factors related to the adoption of other technological innovations including internet banking. Trust in the internet is one of the main factors highlighted as significant in investigating e-government adoption in both developed and developing countries (AlShafi and Weerakkody 2008; Carter and Bélanger 2005; Carter et al., 2011; Chang et al. , 2005). Trust in the internet is often identiﬁed as institution-based trust which is “the belief that needed structural conditions are present (e.g. in the internet) to enhance the probability of achieving a successful outcome in an endeavour like e-commerce” (McKnight et al., 2002; p. 339). Warkentin et al. (2002) clarified the prominent relationship between institution-based trust and e-government adoption. Website design has been endorsed as one of the main factors which assist in explaining e-government adoption (Alomari et al., 2012; Gilbert and Balestrini, 2004). Websites were recognized as the main gateway through which the Jordanian Government could deliver its services and interact online with its citizens (MoICT, 2006a, 2006b). Therefore, well-presented content on government websites was very important. Previous evaluation studies of government websites introduced features and criteria that should be included in their design, such as the availability of e-government websites with clear and organized content (Smith, 2001; Zhang and von Dran, 2000). Gilbert and Balestrini (2004) emphasized the importance of the visual appeal of website design on people’s willingness to use e-government services. Religious beliefs were another factor discussed in the literature with respect to information technology especially after the rapid growth of the internet. This has led to a change in the way that people interact in social communities, from face to face to online interaction. Researchers have therefore been prompted to investigate the internet’s impact on societies where traditions, norms and religion play a significant role in people’s lives (Alomari et al., 2012; Al-Saggaf, 2004; Hill et al., 1998). In previous studies, researchers have explained the effect of religious views on the usage of information technology in Arabic countries (Al-Saggaf, 2004; Hill et al., 1998; Norton, 2002).

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 05-20

www.jistem.fea.usp.br

8 Alomari, M. K.

The literature has also indicated that confidence in internet and computer skills was another factor related to e-government adoption. Having the required skills to use the technology is necessary to ensure the success of any technology usage (Dugdale et al., 2005; Pons, 2004). Previous studies have indicated the importance of investigating the influence of individuals’ internet and computer skills on their intention to use egovernment services (Bélanger and Carter, 2009; Carter and Weerakkody, 2008; Vassilakis et al., 2005). Word of mouth (WOM), wasta (favoritism), and resistance to change are other factors that should be considered when exploring factors related to e-government adoption by the citizenry. There is a lack of studies in the literature investigating the role of these three factors on technology and e-government adoption. This study therefore will be a useful resource and a valuable addition to the literature about technological and e-government adoption. Word of mouth has captured the attention of many marketing researchers who have considered WOM to be a commanding medium for publicity (Arndt, 1967; Westbrook, 1987). This paper acknowledges the importance of word of mouth in the electronic context in general and in e-government in particular. Therefore, it is very important to highlight its influence on e-government adoption. In researching WOM with respect to technology adoption, Kim and Prabhakar (2004) conducted a surveybased study which revealed the intermediate influence of WOM on the adoption of internet banking. The current research paper is investigating the impact of word of mouth on another technological innovation that is e-government. Wasta is one of the underlying characteristics of social life and networks in Arabic countries (Makhoul and Harrison ,2004). “Wasta” is clarified as an Arabic term that indicates the act and person who intercede on behalf of another party or parties (Cunningham and Sarayrah, 1994). On the other hand, Feghali (1997) mentioned that the process of utilizing influence in one’s interpersonal network to receive favors is called “wasta”. In wasta, people deploy their interpersonal connections in order to obtain advantages that would not otherwise be obtainable. Several studies have been conducted to address the issues of wasta within different contexts (Al Awadhi and Morris, 2009; Makhoul and Harrison, 2004). In the technological context, Al Awadhi and Morris (2009) provided a qualitative focus group-based study that indicated that wasta was one of the significant factors related to e-government adoption in the Middle Eastern country of Kuwait. This study through a quantitative-based survey investigates the “wasta” in relation to the adoption of e-government in another Middle Eastern country, Jordan. Resistance to change is one of the terms that has been identified and clarified mainly in the managerial and organizational context. This study explains resistance to change from a social perspective as it is explores the factors related to citizens’ egovernment adoption in the social community of Jordan. Ebbers et al (2007) highlighted the importance of investigating the resistance to e-government innovations. Zander (1950) defined “resistance to change” as: “[b]ehavior which is intended to protect an individual from the effects of real or imagined change” (p. 9). Sathye (1999) proved the necessity of investigating the role of resistance to change in adopting the technological innovation, internet banking, through a survey-based study which

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 05-20

www.jistem.fea.usp.br

Discovering citizens reaction toward E-government: factors in E-government adoption

examined the factors influencing the adoption of internet banking by Australian consumers. To sustain a comprehensive understanding of e-government adoption in Jordan, it was necessary to introduce the theoretical framework used in this research paper in the following section. 3.

THEORETICAL FRAMEWORK

This study uses two theoretical frameworks to study factors related to egovernment adoption in Jordan: these are the Diffusion of Innovation Theory (DOI) (Rogers, 1983) and the Technology Acceptance Model (TAM) (Davis, 1989). Following is a brief presentation on these theories. Diffusion of Innovation Theory (DOI) “Innovation” is an “idea, practice, or object that is perceived as new by an individual or other unit of adoption” (Rogers, 1983, p. 11). Based on the characteristics of innovation, researchers studying the adoption of information technology have utilized the Diffusion of Innovation Theory (DOI) to discuss information technology innovation. According to the DOI, there are five characteristics of successful adoption (Rogers, 1983): relative advantage, complexity, compatibility, triability, observability. Three among these five characteristics have been found to be consistently significant in technology adoption (Tornatzky and Klein, 1982). These are:  Relative advantage: “the degree to which an innovation is perceived as better than the idea it supersedes” (Rogers, 1983, p. 213)  Compatibility: “the degree to which an innovation is perceived as being consistent with the existing values, past experience, and needs of potential adopters” (Rogers, 1983, p. 223)  Complexity: “the degree to which an innovation is perceived as difficult to understand and use” (Rogers, 1983, p. 230). Previous research has recognized the importance of relative advantage, compatibility, and complexity in investigating the adoption of different technological innovations including e-government (Carter and Belanger, 2005; Ojha et al., 2009; Schaupp and Carter, 2005; Van Slyke et al., 2004). Technology Acceptance Model (TAM) The Technology Acceptance Model (TAM) is an adaptation of the Theory of Reasoned Action which states that actual behavior is influenced by the person’s intention to perform such behavior with this intention influenced by one’s attitudes and subjective norms (Ajzen and Fishbein, 1972). The TAM asserts that there are two determinants for the consumer’s attitudes towards usage intention which are:  Perceived usefulness (PU): “the degree to which a person believes that using a particular system would enhance his or her job performance” (Davis, 1989)

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 05-20

www.jistem.fea.usp.br

10 Alomari, M. K.

 Perceived ease of use (PEOU): “the degree to which a person believes that using a particular system would be free of effort” (Davis, 1989). If technology is relatively easy to use and helpful, this will positively inﬂuence the person’s attitudes and intention towards using that technology (Davis, 1989). Davis’s (1989) model, the TAM, is used to evaluate a user’s acceptance of a technology. Previous studies have confirmed the applicability of using the aforementioned constructs to research e-government adoption (Carter and Blanger, 2005; Chang et al., 2005; Hung et al., 2006). In previous research (Alomari, 2010; Alomari et al., 2012), the above constructs of the DOI and the TAM were used. Based on qualitative and quantitative studies, the authors found that relative advantage, complexity, and PU played a significant role when they investigated the adoption of e-government websites in Jordan. Thus, a modified version of these theories will be examined in this paper by investigating in depth the applicability of the aforementioned constructs to study the adoption of egovernment services with the services of two government departments, ISTD and DVLD, selected for study. Based on a quantitative based survey, new data sets with a new sample were collected to conduct the current study. The following section illustrates the method used in this research paper. 4.

METHOD

This section introduces the main demographic characteristics of the sample used and also details of the survey utilized to conduct the study. Sample A questionnaire was administered to 356 Jordanian citizens who regularly accessed the internet and who were major users of ISTD and DVLD’s services, to obtain their perceptions about e-government adoption. Purposive sampling was used in the current study. This kind of sample is used when the purpose is to gain information from particular target groups (Tashakkori and Teddlie ,2003). Of the respondents, 64.9% were males and 35.1% were females. Of the sample, 3.9% were less than 20 years old, 36% were in the age group of 20–29 years old, 30.1% were in the age group of 30–39 years old, 21.6% were in the age group of 40–49 years old, and % were over 50 years old. Among respondents, 71.3% resided in urban areas in Jordan, while 27.2% of respondents resided in remote areas. The majority of respondents were generally employees: 33.4% were employees in government services and 37.4% were employed in the private sector. Most of the respondents (52.8%) held a bachelor degree level of education. Internet usage at home and work recorded the highest percentage, 45.5% and 30.9% respectively. Of the respondents, 32.8% used the internet for email and chatting purposes, 5.9% used it for shopping, 22.7% used it for homework or checking educational study results, 41% used it for reading news, and 31.2% used it for obtaining information from government websites and downloading forms. Most of the respondents (48.9%) accessed the internet one to three hours per

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 05-20

www.jistem.fea.usp.br

Discovering citizens reaction toward E-government: factors in E-government adoption

week. Figure 1 shows a graphical presentation for some demographic characteristics of the current study’s participants.

Figure 1: Demographic Characteristics of Survey’s Participants

The survey This study used a survey to determine the factors related to e-government adoption in Jordan. To develop a greater understanding of e-government adoption in Jordan, two survey instruments were designed for the two government departments: 

Income and Sales Tax Department (ISTD)



Driver and Vehicle Licenses Department (DVDL)

The same item scales were used in the two instruments. The survey consisted of 53 scale items which were designed to examine different factors. The items of some factors were adopted from previous studies (Carter and Bélanger, 2005; Wang, 2003; Alomari et al., 2012; Waddel and Sohal, 1998; Harrison-Walker, 2001; Carter and Bélanger, 2005; van Slyke et al., 2004). Some items were self-developed items with their design based on a literature review and research project conducted by Alomari (2010). These items were reworded to make them more suitable for Jordanian participants. This study’s questionnaire utilized the five-point Likert scale (interval scale from strongly agree to strongly disagree) to measure different scale items. As English is not the first language of Jordan, with most people not fluent in English, the questionnaire was translated into Arabic by an accredited translator. A panel of experienced people then reviewed the translations to ensure accuracy and integrity. A JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 05-20

www.jistem.fea.usp.br

12 Alomari, M. K.

back-translation was also utilized and reviewed to ensure the credibility of the questionnaire after its translation, firstly from English to Arabic and then from Arabic to English. A panel of experienced e-government researchers reviewed and approved the final questionnaire. 5.

RESULTS

This section reports the results of factor analyses undertaken to identify the main factors related to e-government adoption in Jordan. It describes factor analysis and its reliability. Factor analysis To first analyse the results of the survey, exploratory factor analysis was conducted for each independent variable and the dependent variable. The 53 items of the Likert scale were subjected to axial components analysis using SPSS version 17.0. Prior to performing axial component analysis, the suitability of data for factor analysis was assessed. Inspection of the correlation matrix revealed the presence of many coefficients of 0.3 and above. The Kaiser–Meyer–Oklin values were above 0.6 for the different constructs, except for wasta (favouritism), and the Bartlett’s Test of Sphericity reached statistical significance, supporting the factorability of the correlation matrix. Axial components analysis was conducted with Varimax rotation. The axial factor analysis revealed the presence of 3, 8, 3, 4, 5, 5, 4, 5, 4, and 5 components with eigenvalues exceeding 1 for trust in the internet, website design, belief, internet and computer skill confidence, word of mouth, resistance to change, perceived usefulness, relative advantage, complexity, and e-government adoption, respectively. According to the general rule of thumb, only those factors with eigenvalues greater than 1 should be considered important for analysis purposes (Hair et al., 1998). The screenplot for the aforementioned variables was inspected. Thus, the variables which showed a strong loading in this study: trust in the internet, website design, religious beliefs, internet and computer skill confidence, word of mouth, resistance to change, perceived usefulness, relative advantage, complexity, egovernment adoption. It was decided to eliminate the factor of wasta from any further analysis. Table 1 shows the loading for the different constructs.

Table 1: Factor analysis—citizen data FACTOR LOADING ITEM

TRUST_I

WEB

BELIEF

SKILL

WOM

RTC

TRUST_I3 .788

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 05-20

www.jistem.fea.usp.br

ADOP

Discovering citizens reaction toward E-government: factors in E-government adoption

TRUST_I1 .788 TRUST_I2 .712 WEB3

.828

WEB4

.808

WEB6

.759

WEB5

.750

WEB2

.713

WEB1

.705

WEB7

.683

WEB8

.550

BELIEF3

.951

BELIEF2

.930

BELIEF1

.780

SKILL1

.827

SKILL4

.821

SKILL3

.797

SKILL2

.760

WOM3

.741

WOM1

.708

WOM4

.615

WOM6

.594 .564

WOM2 RTC3

.802

RTC4

.789

RTC5

.788

RTC2

.550

RTC1

.463

PU2

.816

PU1

.709

PU5

.689

PU4

.651

RA5

0.710

RA6

0.670

RA1

0.669

RA2

0.663

RA4

0.647

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 05-20

www.jistem.fea.usp.br

14 Alomari, M. K.

CX3

.811

CX2

.762

CX4

.742

CX1

.688

ADOP1

.688

ADOP3

.685

ADOP5

.646

ADOP2

.599

ADOP4 .555 Notes: Trust_I = trust in the internet; WEB = website design; BELIEF = religious beliefs; SKILL = internet and computer skill confidence; WOM = word of mouth; RTC = resistance to change; PU = perceived usefulness; RA = relative advantage; CX = complexity; ADOP = adoption

Reliability Analysis The internal reliability of the main components of factor analysis was evaluated using Cronbach’s alpha. Table 2 introduces the major components of the exploratory factor analysis and the reliability analysis.

Table 2 : Reliability analysis of variables CONSTRUCT

NUMBER OF ITEMS

CRONBACH’S α

Trust in the internet Website design Religious beliefs Internet and computer skill confidence Word of mouth Resistance to change Perceived usefulness Relative advantage Complexity Adoption

3 8 3 4 5 5 4 5 4 5

.807 .899 .917 .879 .783 .804 .804 .806 .838 .775

DISCUSSION

This section discusses the following significant factors as presented in Table 2: trust in the internet, religious beliefs, website design, internet and computer skill

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 05-20

www.jistem.fea.usp.br

Discovering citizens reaction toward E-government: factors in E-government adoption

confidence, word of mouth, resistance to change, perceived usefulness, relative advantage and complexity. The results related to trust in the internet showed that it is essential to incorporate the concerns of citizens in the developing country of Jordan with regard to the privacy and security of their personal details and to consider their willingness to engage with e-government. This study’s findings are in the line with previous research conducted by Carter and Bélanger (2005) in the developed country of the USA as trust in the internet showed a strong loading in the factor analysis. This research paper showed the necessity of exploring this factor in relation to e-government adoption in developed and developing countries. The factor, religious beliefs, was measured using different scale items which described different religious beliefs and views toward the internet, for example, immorality issues and adult themes. Although there is a lack of research on the role of religious beliefs in e-government adoption, the substantial body of research conducted by Alomari et al. (2012) has leveraged a knowledge base in order to explain this role. This paper extends the authors’ research by providing an in-depth understanding through the usage of more scale items, such as anti-religious propaganda, to measure religious beliefs. Website design emerged as a significant component of e-government adoption in Jordan. Different evaluation studies denoted the effect of government websites including adequate features to ensure users’ satisfaction (Smith, 2001; Wang et al., 2005). These studies provided a base for the present study in its design of the main scale items for measuring website design: an example of these items is the availability of clear directions for navigating e-government websites. In terms of the factor, internet and computer skill confidence, this study presents the importance of considering the fundamental role of technical skills, that is, internet and computer skills, when researching the factors that influence people’s intentions to use e-government services. The varied purposes of using the internet, such as reading news and online shopping, mentioned by survey respondents indicated their different skill levels in interacting with the internet. This study is in line with the literature as it reports on the need to discuss the variation in citizens’ technical skill level and their enthusiasm for using egovernment services (Belanger and Carter, 2006; Vassilakis et al., 2005). The nature of the survey respondent population, who were mostly employees, explained the strong loading in the factor analysis in terms of the resistance to change factor. Most of the scale items used to measure this factor reflected the changes that would occur with the introduction of e-government services. Employees would be concerned about different kinds of changes related to e-government as a technological innovation such as losing their jobs as they might be replaced by technology. On the other hand, word of mouth recorded a strong loading in the factor analysis. The way in which this factor was measured showed how people would socialize and network about e-government. Jordan is one of the Arabic societies that are collectivist in nature (Hofstede, 2009). Individuals demonstrate their commitment to and trust in the group to which they belong in different ways, such as when they make decisions about whether or not to use the internet based on the experiences of others whom they trust.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 05-20

www.jistem.fea.usp.br

16 Alomari, M. K.

Relative advantage, perceived usefulness and complexity factors were recorded with strong loadings in the factor analysis. The participants were internet-literate and major users of the two government departments (DVLD and ISTD) on which the survey was conducted. This enabled participants to be more capable of assessing how easy it was to understand and use e-government services, how useful these services were when conducting different transactions and to what extent they considered egovernment more important than the traditional ways of interacting with the government. In previous research conducted in a developed country, the USA, relative advantage and perceived usefulness were loaded together in the factor analysis and the researchers decided to drop them from further analysis (Carter and Bélanger, 2005). Yet in this research, PU and relative advantage were loaded separately, thus showing the importance of including both constructs to investigate e-government service adoption in developing and Middle Eastern countries like Jordan. Wasta was the only factor was dropped down from further analysis since it has recorded a weak loading. This might refer to the sample used in this study who was internet literate people. Internet literate people may view the internet as a desirable channel by which to interact with government, as they are more aware of how this kind of interaction could limit wasta. The discussion and evaluation of the aforementioned factors are necessary to further examine their direct influence on citizens’ intentions to use e-government. Further research will determine the direct impact of these factors on e-government adoption. 7.

RESEARCH IMPLICATIONS

This research prompts the government in Jordan on a series of factors that could be used to stimulate a proposal for their promotional campaign about egovernment, with word of mouth (WOM) being an example of these factors. To join the two-way live conversation occurring between people about e-government, the government in Jordan needs to design the right promotional campaigns about egovernment services. These campaigns could include different themes such as why egovernment is important in people’s real lives and real success stories about using egovernment. These campaigns should clearly address and answer people’s different concerns and questions related to e-government. These concerns include the issue of security and privacy in dealing with e-government through different electronic channels and could be addressed by announcing the different strategies and applications being used to secure data. Through the promotional campaigns, different brand communications could be used to ensure access for different categories of society (people who have or who do not access to the internet): these would include traditional advertising (offline media) and technology-based advertising through internet and television. Social networking applications, such as Facebook and Twitter, are one of the internet based technologies through which the Jordanian government can increase

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 05-20

www.jistem.fea.usp.br

Discovering citizens reaction toward E-government: factors in E-government adoption

awareness of people about e-government. The government in Jordan should introduce social networking applications as future potential channels to enhance citizenry’s interaction with different governmental ministries and agencies. 8.

CONCLUSION

This paper presents a study with a multidimensional theoretical framework which combines a literature review, the Diffusion of Innovation Theory (DOI) and the Technology Acceptance Model (TAM) to identify the main factors related to egovernment adoption by the public in Jordan. The study collected data from 356 Jordanian people who had regular access to the internet. Factor analysis was the main analytical technique utilized to identify the factors related to e-government services’ adoption. This study has provided an in-depth analysis of e-government adoption by focusing on services provided by two government departments, ISTD and DVDL. The main conclusion is that trust in the internet, religious beliefs, website design, internet and computer skill confidence, word of mouth, resistance to change, perceived usefulness, relative advantage, and complexity are the main factors related to egovernment adoption. The research paper has highlighted that the government in Jordan should be sensitive to the dynamics of social and cultural life in Jordan in formulating the response needed from citizens when introducing e-government services as a new channel of interaction with government. REFERENCES Ajzen, I. & Fishbein, M. (1972). “Attitudes and Normative Beliefs as Factors Inﬂuencing Behavioral Intentions,” Journal of Personality and Social Psychology, 21(1), pp.1-9. AlAwadhi, S., & Morris, A. (2009). “Factors Influencing the Adoption of EGovernment Services,” Journal of Software, 4 (6), pp. 584-590. Alomari M.K. (2010). Predictors for Successful E-government Adoption in the Hashemite Kingdom of Jordan: The Deployment of an Empirical Evaluation Based on Citizen- Centric Perspectives, PhD Thesis, Griffith University, Brisbane, Australia. Alomari, M.K., Sandhu, K. & Woods, P. (2010). “Measuring Social Factors in Egovernment Adoption in the Hashemite Kingdom of Jordan,” International Journal of Digit Society (IJDS), 1 (2), pp.163-172. Alomari, M.K., Woods, P, & Sandhu, K. (2012). “Predictors for E-government Adoption in Jordan: Deployment of an Empirical Evaluation based on a Citizen-Centric Approach,” Information Technology & People, 25 (2), pp.207-234. Al-Saggaf, Y. (2004). “The Effect of Online Community on Offline Community in Saudi Arabia,” The electronic Journal on Information Systems in Developing Countries, 16(2), pp.1-16.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 05-20

www.jistem.fea.usp.br

18 Alomari, M. K.

Al-Shafi, S., & Weerakkody, V. 2008. “Adoption of Wireless Internet Parks: An Empirical Study in Qatar”. Proceedings of the European and Mediterranean Conference on Information Systems, ISeing Press. Arndt, J. (1967). “Role of Product-Related Conversations in the Diffusion of a New Product,” Journal of Marketing Research, 4 (3), pp.291-295. Bélanger, F., & Carter, L. (2009). “The Impact of the Digital Divide on e-Government Use,” Communications of the ACM, 52 (4), pp.132-135. Carter, L., & Bélanger, F. (2005). “The Utilization of E-Government Services: Citizen Trust, Innovation and Acceptance Factors,” Information Systems Journal, 15(1), pp.525. Carter, L., Shaupp, L. C., Hobbs, J., & Campbell, R. (2011). “The role of security and trust in the adoption of online tax filing. Transforming Government: People, Process and Policy, 5(4), 303-318. Carter, L., & Weerakkody, V. (2008). “E-Government Adoption: A Cultural Comparison,” Information Systems Frontiers, 10(4), pp.473-482. Chang, I. C., Li, Y.-C., Hung, W.-F., & Hwang, H.-G. (2005). “An Empirical Study on the Impact of Quality Antecedents on Tax Payers' Acceptance of Internet Tax-Filing Systems,” Government Information Quarterly, 22(3), pp.389-410. Cunningham, R. B., & Sarayrah, Y. K. (1994). “Taming Wasta to Achieve Development,” Arab Studies Quarterly, 16(3), pp.29-41. Davis, F.D. (1989). “Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology”. MIS Quarterly, 13(3), pp.319-40. Dugdale, A., Daly, A., Papandrea, F., & Maley, M. (2005). “Accessing E-Government: Challenges for Citizens and Organizations,” International Review of Administrative Sciences, 71(1), pp.109-118. Ebbers, W. E., & van Dijk, J. A. G. M. (2007). “Resistance and support to electronic government, building a model of innovation”. Government Information Quarterly, 24(3), 554-575. Ebrahim, Z., & Irani, Z. (2005). “E-government Adoption: Architecture and Barriers,” Business Process Management Journal, 11(5), pp.589-611. Fang, Z. (2002). “E-Government in Digital Era: Concept, Practice, and Development,” International Journal of The Computer, The Internet and Management, 10(2), pp.1-22. Feghali, E. (1997). “Arab Cultural Communication Patterns,” International Journal of Intercultural Relations, 21(3), pp.345-378. Gilbert, D., & Balestrini, P. (2004). “Barriers and Benefits in the Adoption of EGovernment,” The International Journal of Public Sector Management, 17(4), pp.286301. Hair, J.F. Jr, Anderson, R.E., Tatham, R.L. & Black, W.C. (1998). Multivariate Data Analysis with Readings. Prentice Hall International, Inc: New Jersey.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 05-20

www.jistem.fea.usp.br

Discovering citizens reaction toward E-government: factors in E-government adoption

Harrison-Walker, L. J. (2001). “The Measurement of Word-Of-Mouth Communication and an Investigation of Service Quality and Customer Commitment as Potential Antecedents,” Journal of Service Research: JSR, 4 (1), pp.60-75. Hill, C. E., Loch, K. D., Straub, D. W., & El-Sheshai, K. (1998). “A Qualitative Assessment of Arab Culture and Information Technology Transfer,” Journal of Global Information Management, 6 (3), pp.28-38. Hofstede, G. (2001). Culture's Consequences: Comparing Values, Behaviors, Institutions, and Organizations Across Nations. Sage Publications: Thousand Oaks, California. Hofstede, G. (2009). “Geert Hofstede Cultural Dimensions.The Hofstede Centre,” Retrieved from http://www.geert-hofstede.com/hofstede_arab_world.shtml. on April 23, 2010. Hung, S.-Y., Chang, C.-M. & Yu, T.-J. (2006). “Determinants Of User Acceptance Of The E-Government Services: The Case Of Online Tax ﬁling And Payment System,” Government Information Quarterly, 23(1), pp.97-122. Kim, K. & Prabhakar, B. (2000). “Initial Trust, Perceived Risk, and the Adoption of Internet Banking,” Proceedings of the 21st international conference on Information systems, AIS Electronic Library. Makhoul, J., & Harrison, L. (2004). “Intercessory Wasta and Village Development in Lebanon,” Arab Studies Quarterly, 26(3), pp.25-41. McKnight, D.H., Choudhury, V. & Kacmar, C. (2002). “Developing and Validating Trust Measures for E-Commerce: An Integrative Typology,” Information Systems Research, 13(3), pp.334-59. MoICT (2006a). “E-readiness Assessment of the Hashemite Kingdom of Jordan,” Retrieved from http://moict.gov.jo/MoICT/MoICT_Jordan_ereadiness.aspx. on April 12, 2009. MoICT (2006b). “Jordan E-government Program: E-government Strategy,” Retrieved from http://moict.gov.jo/e_gov_strategy/e-Government%20Strategy.pdf. on April 16, 2009. Moon, M. J. (2002). “The Evolution of E-Government among Municipalities: Rhetoric or Reality?,” Public Administration Review, 62(4), pp.424-433. Norton, L. (2002). “The Expanding Universe: Internet Adoption in the Arab Region,” Retrieved from http://www.worldmarketsanalysis.com/InFocus2002/articles/middleeast_internet.html on October 17, 2007. Ojha, A., Sahu, G. P., & Gupta, M. P. (2009). “Antecedents of Paperless Income Tax Filing by Young Professionals in India: An Exploratory Study,” Transforming Government: People, Process and Policy, 3(1), pp.65-90.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 05-20

www.jistem.fea.usp.br

20 Alomari, M. K.

Pons, A. (2004). “E-government for Arab countries,” Journal of Global Information Technology Management, 7(1), pp.30-46. Rogers, E.M. (1983). Diffusion of Innovations. The Free Press: New York. Sathye, M. (1999). “Adoption of Internet Banking By Australian Consumers: An Empirical Investigation,” International Journal of Bank Marketing, 17 (7), pp.324-334. Schaupp, L.C. & Carter, L. (2005). “E-voting: from apathy to adoption,” The Journal of Enterprise Information Management Science, 18(5), pp.586-601. Smith, A.G. (2001). “Applying Evaluation Criteria to New Zealand Government Websites,” International Journal of Information Management, 21(2), pp.137-49. Tashakkori, A., Teddlie , C. (2003). Handbook of Mixed Methods in Social and Behavioral Research. Sage: Thousand Oaks. Waddell, D., & Sohal, A. S. (1998). “Measuring Resistance to Change: An Instrument and Its Application” (Working Paper 62/98), Monash University, Caulfield East, Victoria. Wang, Y.-S. (2003). “The Adoption of Electronic Tax Filing Systems: An Empirical Study,” Government Information Quarterly, 20(4), pp.333-352. Warkentin, M., Gefen, D., Pavlou, P.A. & Rose, G.M. (2002). “Encouraging Citizen Adoption of E-Government by Building Trust,” Electronic Markets, 12(3), pp.157-62. Westbrook, R. A. (1987). “Product/Consumption-Based Affective Responses and Postpurchase Processes,” Journal of Marketing Research, 24(3), pp.258-270. Van Slyke, C., Belanger, F., & Comunale, C. L. (2004). “Adopting Business -ToConsumer Electronic Commerce: The Effects of Trust and Perceived Innovation Characteristics,” The Data Base for Advances in Information Systems, 35(2), pp.32-49. Vassilakis, C., Lepouras, G., Fraser, J., Haston, S., & Georgiadis, P. (2005). “Barriers to Electronic Service Development,” e-Service Journal, 4(1), pp.41-63. Zander, A. F. (1950). “Resistance to Change-Its Analysis and Prevention,” Advanced Management, 4(5), pp.9-11. Zhang, P. & von Dran, G.M. (2000). “Satisﬁers and Dissatisﬁers: A Two-Factor Model for Website Design and Evaluation,” Journal of the American Society for Information Science, 51(14), pp.1253-68.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 05-20

www.jistem.fea.usp.br

JISTEM - Journal of Information Systems and Technology Management Revista de Gestão da Tecnologia e Sistemas de Informação Vol. 11, No. 1, Jan/Apr., 2014 , pp. 21-32 ISSN online: 1807-1775 DOI: 10.1590/S1807-17752014000100002

THE FUTURE OF AUDIT Danielle Lombardi Villanova University, Pennsylvania, United States, USA Rebecca Bloch Fairfield University, Connecticut, United States, USA Miklos Vasarhelyi Rutgers University, New Jersey, Unites States, USA ______________________________________________________________________ ABSTRACT The purpose of this study is to discuss the current state and future of auditing. Expert consensus is used as a basis to examine the current state of auditing and generate modifications both needed and likely to occur in the audit profession. This study contributes to the literature by using the Delphi method to develop predictions as to the direction of the audit industry and discuss the implications associated with these predictions. If auditors can better understand where the profession stands and where it is headed, then they can better prepare for the future. Some predictions emerging from this study relative to future audit practices include increasing automation of audit procedures, more predictive financial statements, continuous auditing of financial statements and transactions, and an increasingly global perspective regarding audit activities. Keywords: Audit; Brainstorming; Delphi method; Information systems; Expert panel 1.

INTRODUCTION

The auditing field is at a critical juncture. Independent audits continue to take place annually, whereby associated analysis and reporting routines are based solely on historical data. With the resulting lack of timeliness between data generation and information assurance in this context, it seems that stakeholders would typically not view audited financial statements as being useful for decision-making in the current and ______________________________________________________________________________________________ Manuscript first received/Recebido em 10/12/2013 Manuscript accepted/Aprovado em: 05/03/2014

Address for correspondence / Endereço para correspondência Danielle Lombardi, Assistant Professor at the Villanova School of Business, Villanova University, 800 E. Lancaster Ave. Bartley Hall - 3048 Villanova, PA 19085, E-mail: danielle.lombardi@villanova.edu

Rebecca Bloch, Assistant Professor at the Dolan School of Business, Fairfield University, 1073 North Benson Road, Dolan School of Business – 2109, Fairfield, CT 06824 E-mail: rbloch@fairfield.edu

Miklos Vasarhelyi, Professor II KPMG Professor of AIS Director CarLab, RARC room 946 1, Washington Park Newark, NJ 07102 tel: 201-4544377 Rutgers University, New Jersey, Unites States, USA E-mail: miklosv@andromeda.rutgers.edu Published by/ Publicado por: TECSI FEA USP – 2014 All rights reserved.

22 Lombardi, D., Bloch, R., Vasarhelyi, M.

evolving real-time global economy. Given that decision usefulness is a primary criterion for effective financial reporting, the need for a more timely and proactive auditing methodology is apparent. To remain relevant, auditing must take advantage of technological advances and provide assurances that are meaningful to real-time financial statement users. This study reports on expertsâ&#x20AC;&#x2122; consensus about the current status of the audit profession and forecasts of what the profession might resemble in a decade. Agreement about the present state of the profession was obtained through a brainstorming session and a forecast of the future was generated via the Delphi technique. This study employs a formal methodology to assess the current and future status of the audit profession, and has utility for at least two distinct groups: 1) audit professionals in helping them determine how to best structure their assurance practices, and 2) regulators in anticipating audit regulations that may be needed for audits in a real-time economy. Audits are performed to provide assurance that financial statements properly follow current accounting standards and accurately reflect the financial position of a company. Historically, this paradigm has been useful to investors and creditors who had little information available beyond the financial statements. However, in recent years, technology has taken the lag out of conducting business such that events may be captured instantaneously, and, in some cases, markets are able to react to the constant updating of real-time information (Vasarhelyi et al. 2010). Thus, the business environment has evolved more rapidly than the audit profession (Eilifsen et al. 2001; Humphrey et al. 2009), and audited accounting information is now in a substantially disadvantaged state relative to other forms of timely information. For example, company news is readily available in the financial press and investors can differentiate between competitors by reading about product quality and other industry information posted by consumer product groups. Most important, much of this information is generated very soon after event occurrence and readily available for online consumption and processing. Auditors must be prepared and properly trained to handle the new challenges associated with collecting, processing, and incorporating new forms and large volumes of data, many of which will likely require the application and understanding of sophisticated technologies. To better clarify and address this matter, this study seeks to determine the current state of auditing and what experts believe the future auditing landscape might resemble. The study is performed in two discrete stages: 1) a brainstorming exercise is used to determine the current state of the profession, and 2) the Delphi method is employed to obtain predictions concerning the future of auditing. The Delphi method has been suggested as a methodology that provides value and rigor to research in the fields of auditing (Garsombke and Cerrulo 1984), accounting, and accounting information systems (Worrell et al 2013), and it has been found to accurately forecast future events and trends (Bell, 1967, Mehr and Neumann1970, Dalkey and Helmer 1963, Baldwin-Morgan 1993, Holstrum et al. 1986, Brancheau et al.1996; Rowe and Wright 1999). Furthermore, it has been used extensively to predict the direction of specific industries (Melnyk et al. 2009; Ogden et al. 2005; Singh 2005; Chen 2005; Baldwin-Morgan 1993; Cegielski 2008). The next section provides background literature and the ensuing section explains the methodology. Results and analysis are presented in the third section, and the last section contains the conclusions.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 21-32

www.jistem.fea.usp.br

The Future of Audit

BACKGROUND

The literature predicting changes to the audit profession can be divided into those using generic prediction efforts (Hunton and Rose 2010, Holstrum et al. 1988, Elliott 1994, 2002, Violino 2004), and papers using a formal methodology to predict specific aspects of the profession (Baldwin-Morgan 1993), rather than looking at the profession as a whole. Elliott (1994) emphasized potential opportunities and threats within the auditing profession, specifically noting that information technology provides users with a plethora of information sources beyond the traditional financial statements, and impacts the preparation, audit, and use of financial statements. Elliot’s position was that the current audit profession is threatened because audited financial statements are becoming less relevant to users such as investors, creditors, and analysts. However, he also noted that there are opportunities for the auditing profession to evolve by providing a new set of assurances on information acquired via real-time information. Elliott (2002) recommended that the academic community study the changes needed in the assurance domain to help practitioners prepare for the future. Elliott implied that reliance on information technology (IT) may supersede the need for traditional audited financial statements, and that future users may be decision-makers beyond just investors and creditors. He noted: “Every aspect of the accounting profession is being pervasively affected by advances in information technology”. Violino (2004) discussed trends in IT audits, noting that they are “moving into the mainstream as regulatory compliance, risk management, and information security become higher corporate priorities”. Hunton and Rose (2010) argued that auditors will begin to transition from manually collecting data to managing complex decision support systems, and will thus have to become comfortable with trusting these systems. 3.

METHODOLOGY

The methodology applied in this study was originally used at AT&T Bell Laboratories and RAND Corporation (Bell 1967). Two rounds of brainstorming and Delphi were performed during a six-month interval, and contained some variation of expert participation between sessions. The experts in this study had in-depth backgrounds and experience in auditing and accounting, and were considered the first movers and thought leaders of the field by other audit and accounting professionals. Refer to Figure 1 for a description of the experts who participated at both sessions.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 21-32

www.jistem.fea.usp.br

24 Lombardi, D., Bloch, R., Vasarhelyi, M.

Figure 1 - The Expert Participants For a procedural overview of the brainstorming and Delphi methods performed at both sessions, refer to Figure 2 below.

Figure 2 - The Overall Methods Used in this Study

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 21-32

www.jistem.fea.usp.br

The Future of Audit

Brainstorming Method - The Current State of Audit The brainstorming sessions used open-ended topics developed by a team of researchers with varied backgrounds in accounting and auditing. See Appendix 1 for brainstorming topics presented at each session. Delphi Method - The Future of Audit The Delphi methodology employed multiple questionnaire rounds to obtain response stability (Baldwin-Morgan 1993). Both sessions were recorded and participant discussions were transcribed to allow for a more accurate analysis and quotation of participant responses. Refer to Appendix 2 for questions asked during both Delphi sessions. 4.

RESULTS AND ANALYSIS

The results from both sessions indicate an overall group consensus on both the current state and future of auditing. In addition, responses provided in the brainstorming method closely matched the topics of questions used during the Delphi; which provides methodological validation that the questions used in the Delphi focused on pertinent areas impacting the future of audit. Brainstorming Method Results - The Current State of Audit The three major areas highlighted by participants were: the audit model, technology and automation tools, and audit education. The audit model The audit model is evolving from traditional audits (i.e. historical data at a cutoff date) to a more continuous audit of the entire business process and associated risks, which enables current business issues to be more adequately addressed. Participants indicated that audits have changed from periodic to continuous, covering a much higher percentage of company data and monitoring of business processes. The real-time economy requires real-time assurance, and continuous auditing provides this assurance through verification and quality control. Continuous monitoring of business processes can identify emerging issues, and possibly lead to non-audit opportunities within companies. Conversely, sampling currently used in the audit process provides assurance on historical data and does not evaluate business processes. Technology and automation tools Automation tools, such as decision aids, currently assist auditors in analyses and risk assessments. Furthermore, these tools allow for increased usage of quantitative analyses, such as probability evaluations. As a result of increased audit automation, auditors can now spend more time reviewing analyses and interpreting results rather than performing tasks. Audit automation tools evaluate inherent risks for a particular audit, so the auditor can spend more time reviewing and interpreting analyses (rather than performing tests) and determining the desired course of action. Automated tools currently used also simulate audit procedures to determine if they are robust. Once the JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 21-32

www.jistem.fea.usp.br

26 Lombardi, D., Bloch, R., Vasarhelyi, M.

simulation procedures have been constructed, they may continually run and the auditor can reconfigure the process at any time. Some of the top technologies used currently in auditing are analytics, data bases, and sampling. Accounting education Accounting education has significantly progressed and now incorporates more technology, analytics, fraud detection, risk analysis, forensics, and International Financial Reporting Standards into the classroom. More electives are now available to students, as well as collaboration among courses to cover these very timely topics. Delphi Method Results - The Future of Audit Participant responses were examined by the researchers and paraphrased below. The paraphrasing used in this study is selective, and aims to illustrate the tone and tenor of participant discussions. Participant responses from the Delphi indicate that consensus was reached. Overall, the ranges of responses decreased round over round, with an insignificant amount of dissenters among the groups. The questions used in the Delphi sessions are shown in Appendix 2. Questions are grouped by topic to better analyze the questionnaire sessions from both Delphi meetings. Audit automation, procedures, and judgment (Q1Q5) The overall consensus was that auditing is judgmental in nature, and, although automation can support the judgment process, it cannot replace human judgment. P3 said “Technology should reduce the barriers and repetitiveness of time-consuming sampling. We can leverage technology to get greater coverage on some areas than what we currently are doing. However, you never can automate the human judgment component. It is a part of auditing and I don’t know that we want that to be automated.” The use of automation also varies with company type and size, with larger companies driving the use of technology in their organizations and internal audit departments. Judgment is also needed in the audit process to ensure that what was recorded by the system appropriately represents the actual occurrence. “I have to be able to change the lens that I am looking through because I know that people will end up sensing a pattern and their behavior will either change or stay consistent with the pattern” (P3). P1 agreed, “The firm is supposed to capture what happens in the real world in their information system. The most important issue for auditors is whether what happened in the real world is reflected in the client’s and I don’t think this can be 100 percent automated.” "Judgment and expertise" are contingent on each other, meaning expertise is required to make a quality judgment (Mock et al. 1993). Overall, participants foresee auditor judgment in conjunction with automation as essential components of the future audit. “Judgment is something that comes with experience and training. It is driven by the use of technology. Ten years from now, more and more things are going to be automated. For what we use to sit down and analyze there will be different tools to do the analytics so you can use your judgment in a different way” (P4). To keep up with technology, more frequent auditor training will be needed, which will impose additional costs and the need for staff availability for training. Although newer staff has less experience than veteran staff, they tend to be more up-to-

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 21-32

www.jistem.fea.usp.br

The Future of Audit

date on technology and software because experienced staff might often not receive the necessary training. The continuing advancement of technologies is leading to a better evaluation of audit evidence. Auditors will be able to use sensors, biometrics, voice recognition, meta-information exchange, clustering, and expert systems to evaluate clients and analyze relationships among companies. A meta-information exchange will be developed from data to efficiently and effectively examine relationships among similar companies. This includes capabilities to drill down for more detailed information. Through the use of these emergent technologies, data will be delivered straight to auditors' desktops and inconsistencies will be easily identified. Last, the use of electronic reporting, eXtensible Business Reporting Language (XBRL), may serve to expedite and improve certain audit functions, such as benchmarking and analytical review. XBRL data allows auditors to examine risks across clients in similar industries, resulting in an overall meta-analysis. This information can be provided repetitively and instantly impounded into analytic models. In addition, XBRL allows various levels of data to be tagged and subsequently used in customized reports to analyze and compare different industries and companies. These aggregation techniques allow for the combination of qualitative and quantitative data as well as historical and real-time data for a dynamic assessment including probabilities for fraud or potential error. Furthermore, these techniques allow for a more flexible, customized audit plan and a better audit by exception. Internal auditors taking over some functions from external auditors (Q6, Q7) The participants discussed a shift in the role of internal audit resulting from demands of end-users and regulatory bodies. For example, when the Sarbanes-Oxley Act was enacted, companies did not want external auditors doing non-audit-type work that was required by the new regulation. Currently, audit firms are more aggressive about maintaining this work, and some entities are either providing the services for free or steeply discounting them to keep other firms out. It is still unclear to the participants how this will ultimately be handled in the future. The participants did postulate that the relationship between internal and external audit will evolve in the future so that the roles and responsibilities of internal audit will increase relative to external audit. New internal audit technologies, such as automated controls monitoring, will allow external auditors to place more reliance on the work of internal auditors. Some of this is already in place and being applied by external auditors,; such as internal journal entry analysis and audit assessments provided by internal auditors. If this trend continues, internal audit will ultimately have more responsibility than external audit regarding assurance on the quality of data for low-risk areas. However, high-risk areas must remain the responsibility of external audit because of independence issues. Ultimately, the American Institute of Certified Public Accountants (AICPA) will have to provide updated guidance concerning reliance on internal audit work by external auditors, and internal auditors have to be properly trained to meet these new challenges

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 21-32

www.jistem.fea.usp.br

28 Lombardi, D., Bloch, R., Vasarhelyi, M.

Frequency of externally audited financial statements, audit fees, and changing audit platform (Q8, Q9, Q10) Participants debated the diminishing role of externally audited financial statements and the constant availability of externally audited financial statements due to real-time reporting and continuous reporting. â&#x20AC;&#x153;People are making individual types of decisions whether to invest or not invest, and the type of information they need to make decisions differs. A system designed from scratch would permit them to take a view of the information thatâ&#x20AC;&#x2122;s relevant to their decision problem instead of forcing everything into a one size fits all model. Ten years out infrastructure changes would be necessary to change the information paradigm that is in place. The only thing that prevents it from happening now and ten years from now is that the SEC and the AICPA and others have a vested interest in this one size fits all modelâ&#x20AC;? (P1). There were varying opinions from participants regarding the audit fee model. If the same parties responsible for hiring the auditors remain responsible for agreeing and paying the audit fees, a bias will exist for either the company or the audit firm. The auditor will have an incentive to agree with the client in order to increase the likelihood of being fully compensated for current work and retained as the auditor in future engagements. This creates obvious problems relative to auditor independence and objectivity. It was suggested that the PCAOB either take on the responsibility of setting audit fees or handle the process of selecting company auditors in order to assist with ensuring that auditor independence is consistently maintained. This topic also relates to the potential for rolling the audit function into a general management monitoring and control platform. Within this framework, segregation of duties would need to be achieved to better prevent and deter decreased auditor professional skepticism. Furthermore, this would presumably enhance auditor independence and objectivity as well. The main profile of services provided by large CPA firms in the future will incorporate forensics into the traditional audit, with an increase in revenue from this service. Assurance services will continue, but the nature of assurance will shift to an emphasis on real-time assurance via continuous auditing. Clients will require more risk-based services and wider forms of assurance services. Utilization of XBRL/GL, (Q11) There was an initial variation in responses which was mainly due to the lack of knowledge and confusion of XBRL/GL and its intended uses and capabilities. IT education must stem from the classroom, so that students gain the basic knowledge and skills necessary for performing in the field. Educators must account for XBRL/GL by both familiarizing students with the most current standard retrieval methodologies available, and allowing them to discover future tools later on in their careers. In addition, participants noted that a common data model needs to be created across all ERP systems, so every general ledger ERP system knows the needed fields; such as, payment amount, date, payee, and payer. With the recent mandate from the Securities and Exchange Commission for public companies using XBRL for financial statements, the necessity for learning and training on XBRL/GL exists and needs to be addressed by companies as well as educational institutions.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 21-32

www.jistem.fea.usp.br

The Future of Audit

CONCLUSIONS

Expert consensus was reached relative to both the current state of audit and the future of audit over the next decade. For a summary of highlights and recommendations resulting from both Delphi sessions, refer to Table 1.

First Session

Second Session

• External auditors will rely more on internal audit work in the future • Although use of automation will increase, judgment and decisionmaking cannot be automated • The view of many of the topics would vary depending upon the evolution of the financial statements • Audit will be cycled over the year, instead of only at year-end • There is a need for a more global perspective

• Client technology is leading audit procedures • The use of technology depends upon proper safeguards for privacy (i.e. HIPAA) • Automation can be used for more tedious tasks so that auditors can use their expert judgment for more pressing issues

Table 1 - Highlights and Recommendations Provided by Participants from the Delphi Method Participants agreed that the audit model is currently incorporating continuous audits of business processes, and that auditors are using automation tools such as decision aids and risk assessment tools to assist with their audits. This will continue to evolve over the next decade as meta-analysis information exchanges with drill down capabilities will become more widely used for this purpose. Audit education has progressed and now increasingly incorporates the use of technology as well as offers new electives to help students better prepare for real-world audit. To keep up with technology, additional auditor technology training will be needed, especially for experienced auditors. Over the next decade, the participants forecasted that technology will continue to be an important part of the audit process, but that it will not take over the judgment that is inherent in the auditing process. Judgment from audit experts will continue to be essential. The participants also stated that the relationship between internal and external audit will continue to evolve with more responsibility shifting to the internal audit function. However, there was some debate as to whether the traditional audited financial statements will be replaced with more predictive, real-time statements. Although some participants noted that financial statements are competing with other forms of information, others felt that too many regulatory bodies are invested in the traditional model for it to significantly change. JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 21-32

www.jistem.fea.usp.br

30 Lombardi, D., Bloch, R., Vasarhelyi, M.

Participant responses during the brainstorming sessions validated the questions posed during the questionnaire rounds. Participants' insights during the brainstorming sessions related to the same areas of audit the Delphi addressed during both sessions. Overall, results suggest it is not only likely, but necessary that the traditional audit undergoes changes to make it more relevant in this real-time economy. Auditing needs to stay in tune with continuous advancements in the profession in order to effectively meet the needs of the users of information. REFERENCES Anderson, U. (2012). Assurance Coordination: Managing the Organizationâ&#x20AC;&#x2122;s Assurance Network, presentation, 2/22/12. Baldwin-Morgan, A. (1993). The impact of expert system audit tools on auditing firms in the year 2001: A Delphi investigation. Journal of Information Systems, 7(1), 16-34. Bell, W. (1967). Technological forecasting - what it is and what it does. Management Review,56(8), 64. Brancheau, J. C., Janz, B.D. & Wetherbe, J.C. (1996). Key issues in information systems management: 1994-95 SIM Delphi results. MIS Quarterly, 20(2), 225-242. Cegielski, C. G. (2008). Toward the development of an interdisciplinary information assurance curriculum: Knowledge domains and skill sets required of information assurance professionals. Decision Sciences Journal of Innovative Education, 6(1), 2949. Chen, M. (2005). Ethics: An urgent competency in financial education. Journal of American Academy of Business, Cambridge, 6(2), 74-79. Dalkey, N., & Helmer, O. (1963). An experimental application of the Delphi method to the use of experts. Management Science, 9(3), 458-467. Eilifsen, A., Knechel, R. & Wallage, P. (2001). Application of the business risk audit model: A field study. Accounting Horizons, 15(3), 193-207. Elliott, R. K. (1994). The future of audits. Journal of Accountancy, 178(3), 74-82. Elliott, R. K. (1996). Auditing Reborn. CA Magazine, 129, 36-8. Elliott, R. K. (2002). Twenty-first century assurance. Auditing, 21(1). 139. Garsombke, H. & Cerullo, M. (1984): Auditing advanced computer systems, The EDP Auditors Journal, 2(1): 1-11. Holstrum, G.L.; Mock, T. J. & West, R.N. (1986): The impact of technological events and trends on audit evidence in the year 2000: Phase I, Auditing Symposium VIII. Proceedings of the 1986 Touche Ross-- University of Kansas Symposium on Auditing Problems, Lawrence, KS: University of Kansas, 125-146. Holstrum, G. L., T.J. Mock, & West, R.N. (1988). The impact of technology on auditing: Moving into the 21st century. Institute of Internal Auditors Research Foundation, Altamonte Springs, Florida. Humphrey, C., A. Loft, and M. Woods. (2009). The global audit profession and the international financial architecture: Understanding regulatory relationships at a time of financial crisis. Accounting, Organizations & Society, 31(6/7), 810-825.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 21-32

www.jistem.fea.usp.br

The Future of Audit

Hunton, J. E., and J.M. Rose. (2010). 21st Century Auditing: Advancing Decision Support Systems to Achieve Continuous Auditing. Accounting Horizons, 24(2), 297312. Mehr, R. I., and S. Neumann. (1970). Delphi forecasting project. Journal of Risk & Insurance, 37(2), 241-246. Melnyk, S. A., R.R. Lummus, R.J. Vokurka, L.J. Burns, and J. Sandor. (2009). Mapping the future of supply chain management: A Delphi study. International Journal of Production Research 47(16): 4629-4653. Mock, T. J., P.R. Watkins, P. Caster, and K. Pincus. (1993). A review of the audit judgment symposium: 1983-1992. Auditing 12(2): 3. Ogden, J. A., K.J. Petersen, J.R. Carter, and R.M. Monczka. (2005). Supply management strategies for the future: A Delphi study. Journal of Supply Chain Management: A Global Review of Purchasing & Supply 41(3): 29-48. Rowe, G., and G. Wright. (1999). The Delphi technique as a forecasting tool: Issues and analysis. International Journal of Forecasting 15(4): 353-375. Singh, A. J. (2005). Future events and their impact on financial management in the US lodging industry: Delphi study to predict changes in 2007 and 2027. Journal of Retail & Leisure Property 4(3): 236-254. Vasarhelyi, M., R. Teeter, and J.P. Krahel. (2010). Audit Education and the Real-Time Economy. Issues in Accounting Education 25(3): 405-423. Violino, B. (2004). You bought it, now audit. CFO 20(8): 17-19. Worrell, J. L., Di Gangi, P. M., & Bush, A. A. (2013). Exploring the use of the Delphi method in accounting information systems research. International Journal Of Accounting Information Systems, 14(3), 193-208. APPENDIX 1 QUESTIONS ASKED TO EXPERT PARTICIPANTS DURING THE BRAINSTORMING SESSIONS First Delphi Meeting's Brainstorming session Non-audit opportunities (assurance services for CPAs that uses CPA competencies that has expanded the scope of provided services, e.g. provide independent continuous monitoring services) E-Audit (Ways the audit service has been extended through electronic media e.g. perform large part of the audit remotely through video and voice and desktop sharing) Audit automation (ways the audit has been automated, e.g. drive most of audit objective by data alarms) Audit process (ways the way we audit has changed e.g. create automatic pinging) Second Delphi Meeting's Brainstorming Session Choose and rank five top technologies â&#x20AC;&#x201C; most important to least important â&#x20AC;&#x201C;used in audit. How do you estimate the fee structure/base of billing? Please list issues and trends in what you see as the most important first. Please also specify the cause of any changes that have led to changing the fee structure. How has the relationship changed between internal audit and external audit?

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 21-32

www.jistem.fea.usp.br

32 Lombardi, D., Bloch, R., Vasarhelyi, M.

How has accounting/auditing education changed to satisfy requirements of the new audit environment? How has the litigation environment changed? What are the main functions (profiles) of services being provided by the larger CPA firms and how do they interplay? APPENDIX 2 QUESTIONS ASKED TO EXPERT PARTICIPANTS DURING THE DELPHI SESSIONS First Delphi Meeting's Questionnaire Session What percentage of the external audit will be automated? What percentage of the internal audit will be automated? What will happen to usage of sampling techniques in auditing? What will happen to usage of analytic procedures in auditing? What will happen to audit judgment in auditing? To what extent will internal auditors take over responsibility for the IT auditing now undertaken by the external auditor? What is the likelihood that external auditors will offer opinions on financial statements more frequently than once a year? What is the likelihood that a different model of auditor compensation will become prominent in practice? What is the likelihood that inside the firm the audit function will be rolled into a general management monitoring and control platform? What is the likelihood that XBRL-GL will emerge as the common platform for both reporting and assurance? Assuming that eventually there will be continuous auditing, will there be more frequent reporting? What is the likelihood that continuous assurance will be synonymous with auditing, as far as audit practice education is concerned? Second Delphi Meeting's Questionnaire Session What percentage of the external audit will be automated? What percentage of the internal audit will be automated? What will happen to usage of sampling techniques in auditing? What will happen to usage of analytic procedures in auditing? What will happen to human audit judgment in auditing? To what extent will internal auditors take over the responsibility for the IT auditing which is currently undertaken by the external auditor? To what extent will internal auditors take over the responsibility for financial auditing which is currently undertaken by the external auditor? What is the likelihood that external auditors will offer opinions on financial statements more frequently? What is the likelihood that a different model of audit fees will become prominent in practice? What is the likelihood that inside the firm the audit function will be rolled into a general management monitoring and control platform? What percentage of companies will utilize XBRL/ GL? Assuming that eventually there will be some form of continuous auditing, will there be more frequent reporting?

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 21-32

www.jistem.fea.usp.br

JISTEM - Journal of Information Systems and Technology Management Revista de Gestão da Tecnologia e Sistemas de Informação Vol. 11, No. 1, Jan/Apr., 2014 , pp. 33-52 ISSN online: 1807-1775 DOI: 10.1590/S1807-17752014000100003

FACTORES CRÍTICOS DE ÉXITOS EN BUSCA DE LA VENTAJA COMPETITIVA DENTRO DEL COMERCIO ELECTRÓNICO: UN ESTUDIO EMPÍRICO EN LAS EMPRESAS PARAGUAYAS CRITICAL SUCCESS FACTORS FOR A COMPETITIVE ADVANTAGE IN ELECTRONIC COMMERCE: EMPIRICAL STUDY IN PARAGUAYAN COMPANIES Chap Kau Kwan Chung Universidad Americana, Asunción, Paraguay Rosario García Cruz Universidad de Sevilla, Sevilla, España ______________________________________________________________________ ABSTRACT Electronic commerce has been one of the most active fields of investigation in Information Technology in the last few years. It would be interesting to find out which critical factors keep electronic commerce from achieving success and which factors would be regarded as a sustainable competitive advantage for companies. In this investigation we have developed a model for the achievement of success in electronic commerce based on what is proposed by DeLone and McLean regarding the current model to achieve success in Information Systems, tested by the AMOS (Analysis of Moment Structures), through a sample of 55 Paraguayan companies. As for the bibliography used, we have performed a meta-analysis study with the purpose of identifying the most relevant journals, with their respective current authors and the indicators that represent the importance to the project. The results have yielded valuable conclusions to the Paraguayan reality as only two of the 10 indentified critical factors for success were detected: “Experts in information technology (IT) and infrastructure” and “Interoperability of the system”. These are regarded as factors which have been converted into a timely competitive advantage but not into a sustainable competitive advantage through time, due to reasons such as a lack of experts in information technology (IT), particularly regarding the knowledge of electronic commerce and the sluggishness of development of electronic commerce which carries out online activities.

Keywords: critical factors, electronic commerce, model, sustainable competitive advantage, sostenible

Resumen El comercio electrónico ha sido unas de las áreas de investigación más activa dentro del campo de Sistema de Información en los últimos años. Sería interesante descubrir cuáles son los factores críticos que afecta al comercio electrónico para llegar al éxito, y cuales podrían considerarse como una ventaja competitiva sostenible para las empresas. En esta ____________________________________________________________________________________________Manuscript first received/Recebido em 27/10/2012 Manuscript accepted/Aprovado em:17/01/2014 Address for correspondence / Endereço para correspondência Chap Kau Kwan Chung. Doctora en Administración y Dirección de Empresas, Universidad Americana, Asunción, Paraguay. Address: Mcal Estigarribia 1806 c/ Mayor Fleitas. Tel: 595-991-838838. E-mail: wendy505@hotmail.com Rosario García Cruz. Doctora en Ciencias Económicas y Empresariales, Universidad de Sevilla, Sevilla, España. Address: C/San Fernando 4, Post Code: 41004. Tel: 34-954557515. E-mail: rosacruz@us.es Published by/ Publicado por: TECSI FEA USP – 2014 All rights reserved.

34 Chung, C. K. K.,Cruz, R. G.

investigación hemos desarrollado un modelo para el éxito del comercio electrónico basado en la propuesta de los autores DeLone y McLean sobre el actual modelo del éxito de Sistema de Informacion, testado con AMOS (Analysis of Moment Structures), sobre una muestra de 55 empresas paraguayas. En cuanto a la bibliografía utilizada hemos realizado el estudio del meta-analisis con el fin de identificar las revistas científicas más relevantes, con sus respectivos autores del momento y los indicadores que representen importancia para el proyecto. Los resultados han arrojado conclusiones valiosas para la realidad paraguaya, puesto que de los 10 factores críticos de éxito identificados se han encontrado que dos de ellos: “Expertos en tecnología de información (TI) e infraestructura” e “Interoperabilidad del sistema” son considerados como factores que se han convertido en una ventaja competitiva temporal y no ventaja competitiva sostenible a través del tiempo, por razones tales como carencia de los expertos en tecnología de información (TI) especialmente en cuanto al conocimiento del comercio electrónico y la lentitud del avance de los comercios que realizan actividades vía online. Palabras clave: Factores críticos de éxito, Comercio electrónico, modelo de éxito y ventaja competitiva sostenible

INTRODUCCIÓN

Desde los comienzos de la era computacional, investigadores científicos del área de Sistemas de Información (SI) habían pronosticado que con la implementación de la tecnología de computadores, éstos arrojarían unas series de efectos positivos dentro de las empresas (McFarlan, 1984). En los últimos años, el comercio electrónico (CE) ha sobresalido como una de las áreas de investigación más activa dentro del campo del SI (Hong y Zhu, 2006). Aunque, la mayoría de los temas derivados del CE han sido de gran interés en la investigación científica, la mayor parte de éstas investigaciones fueron estudios analizados en países como los Estados Unidos de Norteamérica y de otros países Europeos, y solamente una cantidad limitada de éstos han sido estudiados en los países Asiáticos como en China, Hong Kong, Taiwán, Japón, Malasia, Corea del Sur y Singapur (Teo y Ranganathan, 2004). Muchos tópicos derivados del CE que se han investigado durante la última década como son los beneficios del comercio electrónico en las pequeñas y grandes empresas, que han sido bien establecidas y fundamentadas por muchos investigadores del campo (Joseph et al., 2001), la evaluación de las páginas electrónicas (Liu y Arnett, 2000), la privacidad (Clark, 2000), la confiabilidad en la red (Manchala, 2000), la lealtad de los consumidores en la red (Reichheld y Schefter, 2000), entre otros también han sido de gran interés dentro del campo, y sólo unos pocos estudios han examinado los factores críticos de éxito (FCE) del CE en su conjunto (Sung, 2004). El aumento de interés sobre el CE como tema de investigación es aún bastante novedoso para un tercer mundo como es el Paraguay. La adopción del CE en las empresas es un tema que los empresarios paraguayos no deben ignorar, por el simple hecho de que realizar comercio en internet ya no es un tema de novedad, lo nuevo será el modo de adoptarlo e implementarlo con éxito para llegar a obtener una ventaja competitiva sostenible (VCS). Por lo tanto, la presente investigación tiene como objetivo general identificar los FCE para determinar el éxito del CE en las empresas paraguayas.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 33-52

www.jistem.fea.usp.br

Factores críticos de éxitos en busca de la ventaja competitiva dentro del Comercio Electronico: Un estudio empírico en las empresas paraguayas

MODELO PROPUESTO

El modelo propuesto se basa principalmente en las investigaciones de DeLone y McLean (2004) y Teo et al. (2006). En lo que se refiere a los FCE hemos adoptado los factores de impedimientos del estudio de Teo et al. (2006) agrupados en lo organizacional, tecnológico y entorno y con respecto a las dimensiones del éxito del CE hemos adoptado del estudio de DeLone y McLean (2004). Ver Figura1 y Figura 2.

Figura 1. Modelo propuesto para la investigación

Fuente: Elaboración propia

Figura 2. Modelo específico de la investigación

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 33-52

www.jistem.fea.usp.br

36 Chung, C. K. K.,Cruz, R. G.

Fuente: Elaboración propia

Los FCE del modelo están clasificados en las siguientes formas: Contexto Organizativo (CONORG): Cambio organizacional (fe0100), Estrategia del comercio electrónico (fe0200), Apoyo a la alta gerencia (fe0300), Gestión de proyectos (fe0400) y Análisis de costos-beneficios (fe0700). Contexto Tecnologico (CONTEC): Expertos de la TI e infraestructura (fe0500), Problemas técnicos (fe0600) e Interoperabilidad del sistema (fe0800). Contexto del entorno (CONENT): Aspecto legales (fe0900) y Dudas y temores (fe1000). En cuanto a las dimensiones del éxito del CE se relacionan estos con Calidad del sistema (CALSIS) (f2001 y f2002), Calidad de la información (CALINF) (f2003), Calidad del servicio (CALSER) (f3001 y f3002), y Beneficios netos (BENETO) (f6001 y f6002). 2.1

FACTORES CRÍTICOS DE ÉXITO

Después de un análisis sistemático en la literatura se ha podido identificar 119 FCE en 26 trabajos de investigación que afectan a la adopción del éxito del CE. Ver Tabla I y II. Tabla I. Definición de variables Variables

Descripción

Cambio organizacional

La capacidad de adaptación de las organizaciones a las diferentes transformaciones del orden interno o externo.

Estrategia del comercio electrónico

Es la formulación de objetivos y métodos para llegar al éxito del comercio electrónico dentro de la empresa.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 33-52

www.jistem.fea.usp.br

Factores críticos de éxitos en busca de la ventaja competitiva dentro del Comercio Electronico: Un estudio empírico en las empresas paraguayas

Apoyo de la alta gerencia

Colaboración, liderazgo y disposición de la alta gerencia para el apoyo del desarrollo del comercio electrónico.

Gestión de proyectos

Es la disciplina de organizar y administrar recursos de manera tal que se pueda culminar todo el trabajo requerido en el proyecto dentro del alcance, el tiempo, y coste definidos.

Expertos en tecnología de información (TI) y su infraestructura

Recursos humanos calificados y equipos adecuados para desarrollar las actividades del comercio electrónico.

Mecanismos inadecuados para la protección contra la Problemas técnicos inseguridad, encriptación, autenticación en el sistema de pagos online. Análisis de costosbeneficios

Poseer una herramienta útil y válida con un mecanismo de control para medir los beneficios del comercio electrónico.

Interoperabilidad del sistema

Integración de los sistemas actuales con las aplicaciones del comercio electrónico.

Aspectos legales

Son todos los aspectos que protegen las actividades del comercio electrónico. Ejemplo: leyes impositivas, protección de propiedad intelectual, autenticación en la red, leyes sobre la transferencia de pagos o pagos electrónicos a través de los bancos, etc.

Dudas y temores

El grado por el cual los clientes y proveedores están dispuestos y listos en realizar actividades de negocios en forma electrónica.

Fuente: Elaboración propia

Tabla II. Resumen de las hipótesis y referencias bibliográficas Variables Contexto Tecnológico: Expertos en tecnología de información (TI) y su infraestructura, Problemas técnicos e Interoperabilidad del sistema. Contexto Organizativo: Cambio organizacional, Estrategia del

Hipótesis

Referencia bibliográfica

H1a-d: El contexto tecnológico (CONTEC) tienen un impacto positivo en el éxito del sistema del comercio electrónico medido en calidad de sistema, calidad de la información, calidad de servicio y beneficios netos.

Zhu et al. (2003), Soliman (2003), Grewal et al. (2001), Debreceny et al. (2002), AlNaeem et al. (2005), Evans (2001), Czuchry et al. (2001), Gebauer et al. (1998), Clay y Strauss (2002), Liu y Shen (2004) y Claycomb et al. (2004).

H2a-d: El contexto organizativo (CONORG) tiene un impacto positivo en el éxito del sistema del comercio electrónico medido en calidad de sistema, calidad de la información, calidad de

Zhu et al. (2003), Iyer et al. (2004), Webb (2002), Chircu y Kauffman (2000), Barua et al. (2004), Alt y Fleisch (2001), Czuchry et al. (2001), Hempel y Kwong (2001), Rajkumar (2001), Liu y Shen (2004),

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 33-52

www.jistem.fea.usp.br

38 Chung, C. K. K.,Cruz, R. G.

comercio electrónico, Apoyo de la alta gerencia, Gestión de proyectos, Análisis de costosbeneficios`.

Contexto del Entorno: Aspectos legales y Dudas y temores.

Calidad del sistema, calidad de la información, calidad del servicio que afectan a los beneficios netos del sistema del CE

servicio y beneficios netos.

Claycomb et al. (2004), Evans (2001), Czuchry et al. (2001), Gebauer et al. (1998), Srinivasan et al. (2002), Debreceny et al. (2002) y Chatterjee et al. (2002).

H3a-d: El contexto del entorno (CONENT) tienen un impacto positivo en el éxito del sistema del comercio electrónico medido en calidad de sistema, calidad de la información, calidad de servicio y beneficios netos.

Evans (2001), Gibbs et al. (2003), Debreceny et al. (2002), Czuchry et al. (2001), Irion (2000), Hempel y Kwong (2001), Clay y Strauss (2002), Zhu et al. (2003), Barua et al. (2004), Alt y Fleisch (2001), Srinivasan et al. (2002) y Claycomb et al. (2004).

H4: La calidad del sistema tiene un impacto positivo en los beneficios netos

DeLone y McLean (2004), Bailey y Pearson (1983), Srinivasan (1985), Molla y Licker (2001).

DeLone y McLean (2004), H5: La calidad de la Bailey y Pearson (1983), información tiene un impacto Miller (2005), Miller y Doyle positivo en los beneficios netos (1987), Srinivasan (1985) y Molla y Licker (2001). H6: La calidad del servicio tiene un impacto positivo en los beneficios netos

Cronin y Taylor (1992), DeLone y McLean (2004), Landrum et al. (2007), Molla y Licker (2001).

Fuente: Elaboración propia

Contexto Organizacional  Cambio organizacional: la adopción del CE no significa solamente la adopción de una nueva tecnología, más bien, es toda una nueva forma de hacer negocio reemplazando al modelo tradicional, es decir, una reingeniería en los procesos de negocios. Sung y Gibson (1998) dieron a conocer el impacto positivo de la reingeniería del negocio, como unos de los FCE, en el rendimiento empresarial.  Estrategia del comercio electrónico: El proyecto de realizar actividades comerciales a través de Internet requiere de un profundo análisis de los procesos operativos, es decir, la empresa debe tener una clara visión de la estrategia, formular un plan estratégico y alinear los planes corporativos con los planes electrónicos, todos estos son considerados como aspectos críticos en la estrategia del CE. Investigaciones de autores como Eid y Trueman (2004), Sung (2004) y Sung y Gibson (1998, 2005) han considerado a la planificación de la estrategia del comercio electrónico como un FCE.  Apoyo de la alta gerencia: Chan y Swatman (2000) y Kim (2006), confirman que el factor apoyo y predisposición de la alta gerencia es un factor clave JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 33-52

www.jistem.fea.usp.br

Factores críticos de éxitos en busca de la ventaja competitiva dentro del Comercio Electronico: Un estudio empírico en las empresas paraguayas

para una exitosa implementación de los procesos de innovación de una empresa, puesto que se requiere de conocimientos personales y gerenciales sobre el potencial del marketing en Internet y que éstos estén pro activamente involucrados en su difusión interna de la empresa. Ngai y Gunasekaran (2004) identifican que para obtener una exitosa implementación del EDI, el factor del apoyo de la alta gerencia es indudablemente unos de los tantos FCE que necesita la empresa.  Gestión de proyectos: es considerado como un FCE en la adopción del CE por muchos autores, Chircu y Kauffman (2000) y Zhu et al. (2003). También los autores Teo et al. (2006) han argumentado que unos de los principales obstáculos para la adopción del CE es justamente la gestión de proyectos. Por lo tanto, una carencia de entrenamiento inadecuado del staff de la organización en cuanto al CE (Chircu y Kauffman, 2000 y Zhu et al., 2003), falta de tiempo para desarrollar nuevas actividades (Zhu et al. 2003), falta de consultores externos u otros, son consecuencias de una inadecuada gestión de proyectos Teo et al. (2006).  Análisis de costos-beneficios: se ha identificado a la percepción de costos – beneficios como una variable importante en la introducción y adopción de nuevas tecnologías de información, especialmente en los sistemas ínter organizacionales (Teo y Ranganathan, 2004; Tsai et al., 2005; Zhu y Kraemer, 2002). Los autores Sung (2004), Sung y Gibson (2005) y Dinesh y Jaideep (2001) han identificado como un FCE del CE al costo operativo. Contexto Tecnológico  Expertos en tecnología de información (TI) y su infraestructura: este factor está argumentada como unos de los FCE del CE por muchos autores como ser: Wang y Tsai (2002); Kim (2006), Sung (2004), Sung y Gibson (2005), Eid y Trueman (2004); Ngai y Gunasekaran (2004), Molla y Licker (2005).  Problemas técnicos: estudios relacionados con este factor podemos encontrarlos en investigaciones realizadas por los autores Gebauer et al. (1998), Clay y Strauss (2002), Evans (2001), Czuchry et al. (2001), entre otros. Sung y Gibson, (2005) consideran a la estabilidad del sistema, procesos de pagos y la seguridad del sistema como FCE en el CE.  Interoperabilidad del sistema: Al-Naeem et al. (2005) identificaron que el factor clave de éxito en la implementación de la aplicación del B2B es la integración de los procesos de negocios y servicios e información a través de la empresa incluyendo los esfuerzos de integración dentro de la empresa. Y los autores Gebauer et al. (1998) determinan que la falta de interoperabilidad y la carencia de estándares dificulta a los compradores y vendedores la utilización de un protocolo en común. Contexto del Entorno  Aspectos legales: estudios relacionados con este factor son: representaciones legales (Cloete et al, 2002; Dedrick y Kraemer, 2001), protección de la propiedad intelectual (Dedrick y Kraemer, 2001), leyes sobre tasas del comercio electrónico (Dedrick y Kraemer, 2001), autenticación en la red (compradores, vendedores, productos) (Basu y Muylle, 2003), leyes sobre la transferencia de pagos o pagos electrónicos a través de los bancos (Dedrick y Kraemer, 2001).  Dudas y temores: la inseguridad de la respuesta de los clientes, proveedores, socios comerciales y miedo en abrir los sistemas propios a dichos agentes son factores

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 33-52

www.jistem.fea.usp.br

40 Chung, C. K. K.,Cruz, R. G.

de gran importancia. Otros estudios relacionados con la inseguridad en la respuesta de los socios comerciales son: Alt y Fleisch (2001) y Zhu et al, (2003) con la inseguridad en la respuesta de los clientes son: Srinivasan et al. (2002) y Zhu et al, (2003). 2.2

DIMENSIONES DEL ÉXITO DEL COMERCIO ELECTRÓNICO 

Calidad del Sistema. Según el modelo propuesto por DeLone y McLean (2004) las características para medir la calidad del sistema son: utilizable, disponibilidad, tiempo de respuesta, fiabilidad y adaptabilidad.  Calidad de la Información. Esta dimensión captura el contenido del CE (DeLone y McLean, 2004). La página web debe ser personalizada, completa, relevante, fácil de comprensión y seguro en sus transacciones comerciales.  Calidad del Servicio. Este factor siempre ha sido unas de las dimensiones más importante en cualquier industria, puesto que afecta a la lealtad del consumidor y a la satisfacción y es considerada al mismo tiempo como la más crucial dentro del mundo del CE (DeLone y McLean, 2004), puesto que la audiencia es más bien el cliente que el personal de la empresa, por lo tanto, una carencia de soporte al consumidor se traduce en pérdida de clientes y pérdida de oportunidades de ventas.  Beneficios Netos. Esta dimensión es la medida de éxito más importantes dentro del sistema del CE (DeLone y McLean, 2004). En dicha dimensión se puede capturar el balance entre el impacto positivo y negativo del CE en los consumidores, proveedores, personales, organización, mercado, industrias, economías e inclusive una sociedad entera. 3.

VALIDACIÓN EMPÍRICA

En ese trabajo no hemos analizado la validez de las escalas de medida puesto que el cuestionario utilizado para la encuesta ya estaba aprobado por otros autores en sus respectivas investigaciones. Ver Tabla III. Tabla III. Fiabilidad de las escalas de medida Ítems

α de Cronbach

Cambio organizacional

0.9746

Estrategia del comercio electrónico

0.9699

Apoyo de la alta gerencia

0.9654

Gestión de proyectos

0.9669

Expertos en tecnología de la información (TI) y su infraesructura

0.9495

Problemas técnicos

0.9772

Análisis de costos-beneficios

0.9525

Interoperabilidad del sistema

0.9743

Variables

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 33-52

www.jistem.fea.usp.br

Factores críticos de éxitos en busca de la ventaja competitiva dentro del Comercio Electronico: Un estudio empírico en las empresas paraguayas

Aspectos legales

0.9762

Dudas y Temores

0.9646

Calidad del sistema

0.8990

Calidad de la información

0.8641

Calidad de servicio

0.9111

Beneficios netos

0.8286

Fuente: Elaboración propia

Entre los FCE tenemos: Escala de medida adoptada de Teo et al. (2006) -

El cambio organizacional: 8 ítems (α = 0.90).

La estrategia del comercio electrónico: 3 ítems (α = 0.97).

El apoyo de la alta gerencia: 3 ítems (α = 0.94).

Gestión de proyectos: 6 ítems (α = 0.83).

Los análisis de costos-beneficios: 2 ítems (α = 0.61).

Los expertos de la TI e infraestructura: 4 ítems (α = 0.81).

Los problemas técnicos: 4 ítems (α = 0.89).

La interoperatibilidad del sistema: 3 ítems (α = 0.76).

Aspectos legales: 4 ítems (α = 0.85).

Dudas y temores: 4 ítems (α = 0.78).

Entre el éxito de implantación del comercio electrónico tenemos: Escala de medida adoptada de Gable et al. (2003) en Calidad del Sistema e Informacion. Escala de medida adoptada de Cronin y Taylor (1992) en Calidad del Servicio. Escala de medida adoptada de Wade y Nevo (2005) em Beneficios Netos. -

Calidad del sistema: 9 ítems (α = 0.90).

Calidad de la información: 6 ítems (α = 0.91).

- Calidad del servicio: 22 ítems incluyendo las dimensiones de tangibilidad, fiabilidad, capacidad de respuesta, seguridad y empatía. 4.

Beneficios netos: 14 ítems (α = 0.81).

RECOLECCIÓN DATOS

En el presente trabajo de investigación hemos utilizado como estrategia de investigación “el estudio de campo o estudio empírico”, basado esencialmente en datos primarios como las entrevistas a empresas previamente seleccionadas y encuestas mediantes cuestionarios a empresas que utilizan el CE o no y datos secundarios como datos de archivos y bases de datos dentro de la red. El sector que hemos utilizado para la selección en el estudio son: 1. La lista de empresas registradas en la UIP (Unión Industrial Paraguaya), 2. La lista de miembros de la CAP (Cámara de anunciantes del

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 33-52

www.jistem.fea.usp.br

42 Chung, C. K. K.,Cruz, R. G.

Paraguay) y 3. La lista de empresas registradas en el buscador paraguayo “Yagua.com” con más popularidad en la red. Para comenzar con dicho trabajo de campo, nos hemos propuesto contactarnos con algunas empresas que se encuentraban registradas dentro de nuestra base de datos ya previamente construída, y que a la vez contamos con su cuenta de email. Todo el trabajo fue realizado con el fin de ver la posibilidad de poder acercarnos a dicha empresa y pedir su colaboración para el cuestionario. Ver Tabla IV. Tabla IV. Ficha técnica del estudio empírico Población

1413 empresas

Ámbito geográfico

Asunción – Paraguay

Tipo de muestreo

Muestreo probabilística - Aleatorio simple

Tamaño muestral

177 empresas (12.5 %)

Tasa de respuesta

31.6 %

Nivel de confianza

95 %; Z= 1.96; p=q=0.5

Diseño muestral

Entrevista personal y correo electrónico

Fuente: Elaboración propia

Los pasos que se siguieron fueron: Primero, hemos enviado un correo electrónico a las empresas seleccionadas en la muestra a pedir su colaboración y la manera de contactar con las personas que se encargan de las tecnologías de informaciones y específicamente a los encargados del desarrollo del CE. Segundo, se realizó una entrevista personal a las empresas que nos había permitido dicha realización. Y tercero, una vez realizado el cuestionario definitivo lo hemos enviado por correo electrónico a las empresas colaboradas, mientras que la recolección de dichos documentos fueron algunos devueltos vía email y otros fueron realizado en forma personal. 5.

ANÁLISIS DE DATOS

Se ha utilizado el SPSS para obtener informaciones relevantes con los datos de las muestras proporcionadas. En el primer resultado de los datos obtenidos a través de nuestro cuestionario se han podido identificar los tipos de empresas a quienes fue seleccionada la muestra, en las que pudimos observar que las empresas dedicadas al comercio obtuvieron el mayor porcentaje de entrevistas, llegando a más de la mitad de la muestra total (52,7 %), seguidas de las empresas dedicadas al servicio (30,9 %) y por último, las industrias con sólo (16,4 %). Otros datos interesantes que se pudo rescatar son la cantidad de empresas que cuentan con una página electrónica. Aunque existan 25 empresas que cuentan con una página electrónica, solamente 5 de ellas comercializan a través de la red, mientras que el resto no cuentan con dicha actividad. Finalmente, realizando una clasificación obtuvimos que 3 de dichas páginas son estáticas, 19 interactivas y 3 transitivas. Según el análisis de datos hemos podido contrastar y refutar las hipótesis formuladas con anterioridad y ellas son: Las hipótesis que fueron confirmadas son 7: H1a, H1b, H1d, H2c, H3a, H3c e H6 y las hipótesis que no fueron confirmadas son 8: H1c, H2a, H2b, H2d, H3b, H3d, H4 e H5. VER Tabla V.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 33-52

www.jistem.fea.usp.br

Factores críticos de éxitos en busca de la ventaja competitiva dentro del Comercio Electronico: Un estudio empírico en las empresas paraguayas

Tabla V. Análisis de la contrastación de hipótesis Valor

Sig.

Conf.

+0.59 +0.65 -0.34 +0.11

n n n n

+ + +

-0.61 -0.68 +0.15 -0.11

n n n n

+ -

Sig.

Conf.

+0.15 -0.25 +0.51 -0.15

n n n n

+ + -

Calidad del Sistema (CALSIS) tiene impacto positivo sobre -0.127 Beneficios Netos (BENETO)

-0.16

Calidad de la Información (CALINF) tiene impacto positivo -0.248 sobre Beneficios Netos (BENETO)

-0.06

Calidad del Servicio (CALSER) tiene impacto positivo sobre +0.610 Beneficios Netos (BENETO)

+0.24

Contexto Tecnológico (CONTEC) tiene impacto positivo sobre: · Calidad del Sistema (CALSIS) 2.075 · Calidad de la Información (CALINF) 5.514 · Calidad del Servicio (CALSER) -0.984 · Beneficios Netos (BENETO) 2.869 Contexto Organizativo (CONORG) tiene impacto positivo sobre: · Calidad del Sistema (CALSIS) -1.933 · Calidad de la Información (CALINF) -5.156 · Calidad del Servicio (CALSER) +0.389 · Beneficios Netos (BENETO) -2.396

Valor Contexto de Entorno (CONENT) tiene impacto positivo sobre: · Calidad del Sistema (CALSIS) +0.071 · Calidad de la Información (CALINF) -0.275 · Calidad del Servicio (CALSER) +0.230 · Beneficios Netos (BENETO) -0.349

Fuente: Elaboración propia

Despues del análisis de datos sobre los FCE destacamos que: 1. En cuanto al CONORG: el factor Cambio Organizacional tiene una media relativamente alta de (2.59), y el ítem más destacado fue “los gerentes operativos dan poco apoyo” con una media mucho más elevada que los otros ítem (2.82). En cuanto a la estrategia del CE, este factor tiene una media de (2.38), los encuestados opinan que no existe un claro plan estratégico dentro de la empresa para el comercio electrónico, dicho ítem tiene una media de (2.42). La variable Apoyo de la alta gerencia, ha arrojado una media que centra en (2.42) y su ítem con la media más elevada es de “No hay liderazgo de la alta gerencia” (2.45). El factor gestión de proyectos cuenta con una media de (2.32) y que la media del ítem “No existe apoyo de consultores externos” es de (2.44), siendo considerado como unos de los ítems más importante y preocupante

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 33-52

www.jistem.fea.usp.br

44 Chung, C. K. K.,Cruz, R. G.

para los encuestados. Asi mismo, la variable Análisis de costos-beneficios cuenta con una media de (2.29) y con dos ítems cuyas medias son también las más elevadas “Es difícil justificar las inversiones y beneficios obtenidos a través del comercio electrónico” y “Es complejo medir los beneficios obtenidos a través del comercio electrónico”, ambos con una media de (2.29). 2. En cuanto al CONTEC: la variable Expertos en Tecnología de información (TI) y su infraestructura, cuenta con una media de (2.45), los encuestados creen que la carencia de una adecuada infraestructura de tecnología de información es de suma importancia para una empresa (2.64). Los Problemas técnicos cuenta con una media de (2.23) y cuenta con dos ítems cuyas medias son las más altas dentro del variable. El ítem “Sistema de pagos de trasacciones online es inadecuado” (2.29) y la “Falta de una infraestructura robusta y estable” (2.29). Finalmente, la Interoperabilidad del sistema cuenta con una media de (2.32) y el ítem con la media más alta “Es difícil la integración de las aplicaciones de e-comercio con los sistemas actuales” de (2.47). 3. En cuanto al CONENT: los Aspectos legales cuenta con la media más baja entre todas las variables medidas con tan solo (2.16), y la media del ítem más elevada es que “No existen leyes impositivas consistentes con relación al e-comercio” (2.18). Y las Dudas y temores con una media de (2.17), y el ítem con la media más elevada es que “No hay seguridad de respuesta en los clientes” (2.25). 4. Entre las variables del éxito del CE “Calidad del sistema” encontramos con la variable cuya media es la más baja, de sólo (6.09), la Calidad de la información cuenta con una media de (6.25), la variable Calidad del servicio cuenta con una media de (6.23) y la última variable Beneficios netos cuenta con la media más alta entre ellas, de (6.34). Lo más relevante a los efectos de la interpretación de los resultados obtenidos es que la mayoría, el 90 % no cuenta con CE efectivamente implementado, por lo que los resultados necesariamente reflejarán el grado de creencia, y no experiencia, con relación al sistema de los FCE del CE. En nuestra investigación hemos adoptado el modelo de ecuaciones estructurales AMOS para contrastar las hipótesis del trabajo. 6. RESULTADOS DE LA INVESTIGACIÓN Dentro del análisis descriptivo de ítems encontramos que la más alta valoración media de los FCE fueron: El Cambio organizacional, Expertos en tecnologia de informacion (TI) y su infraestructura y apoyo de la alta gerencia. Y entre los factores que determinan el éxito del CE, tenemos los ítems con más alta valoración media en Calidad del sistema: “En cuanto a las características del sistema”, en Calidad de la informacion: a “la relevancia de dichas informaciones”, en Calidad del servicio: “Dispuestos a ayudar a los usuarios” y en los Beneficios Netos: “alcanzar todos los objetivos en general”. En cuanto al análisis empírico, hemos realizado los cálculos correspondientes de los datos para encontrar la fiabilidad de las escalas. El resultado de la fiabilidad fueron todos positivos, puesto que todos los FCE tuvieron un alfa de Cronbach bastante alta, con más del 0.9 y las variables del éxito del comercio electrónico con más de 0.8, calificándolos así en Excelentes y Muy buenos según los autores George y Mallery (1995). Nuestro modelo confirma la fiabilidad de las escalas de medidas encontradas en los estudios de Teo et al. (2006), Wade y Nevo (2005), Gable et al. (2003) y Cronin y Taylor (1992). Para realizar la contrastación de hipótesis

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 33-52

www.jistem.fea.usp.br

Factores críticos de éxitos en busca de la ventaja competitiva dentro del Comercio Electronico: Un estudio empírico en las empresas paraguayas

del estudio hemos recurrido al modelo de ecuaciones estructurales AMOS en nuestro trabajo de investigación. En cuanto al análisis de las hipótesis, siete son las que fueron confirmadas y el resto fueron refutadas. Para concluir con los análisis empíricos, veremos a continuación si el sistema del comercio electrónico puede ser considerado como una fuente de VCS en empresas del Paraguay. Con el fin de examinar dicha hipótesis nos emergimos en la literatura de nuestra investigación, utilizando la teoría de los recursos y capacidades para contestar si dicho sistema es afirmativo o negativo. A continuación, contestaremos las tres preguntas fundamentales para responder si el sistema del CE es una VCS o no. a. ¿Es el sistema del comercio electrónico valioso? Si la respuesta es negativa entonces la empresa quedará en una desventaja competitiva frente a sus competidores, pero si la respuesta es positiva, es decir valioso, entonces la empresa al menos puede obtener una paridad competitiva dentro del mercado. A esta respuesta vemos que los resultados de nuestro análisis empíricos los Beneficios netos concluyen que dicho sistema puede llegar a convertirse en una ventaja competitiva para la empresa. Mostramos a continuación los cinco principales beneficios de nuestro estudio empírico. Aunque se haya contestado positivamente a la primera pregunta, el sistema del CE es una condición necesaria pero no suficiente para obtener una VCS para la empresa, por lo tanto, se hará la segunda pregunta correspondiente. b. ¿Los factores del sistema del CE están heterogéneamente distribuidos a través de los competidores? Si la respuesta es negativa entonces la empresa no tiene una paridad competitiva frente a sus competidores, pero si la respuesta es afirmativa, entonces podemos decir que puede llegar a obtener una ventaja competitiva en forma temporal. Siguiendo a nuestro estudio empírico, hemos detectado que el contexto tecnológico es la única variable latente que tiene un impacto positivo en los beneficios netos, y las variables que los componen son “Expertos en tecnología de información (TI) e infraestructura”, “Problemas técnicos” y la “Interoperabilidad del Sistema”. Estos FCE del CE pueden estar distribuidos homogéneamente entre los competidores, por lo que no puede considerarse como una ventaja competitiva. Aunque en la teoría de los recursos y capacidades se ha dicho que los factores que estén homogéneamente distribuido a través de los competidores no pueden llegar a convertirse en una ventaja competitiva, la realidad Paraguaya dice lo contrario “por el momento” (en el momento de la realización del cuestionario), puesto que las empresas analizadas, la mayoría de ellos, aunque cuenten con una presencia dentro de la red, aún no desarrollan actividades relacionadas con el CE. Las razones son varias, una de ellas es la carencia de los expertos en tecnología de información (TI) especialmente en cuanto al conocimiento del comercio electrónico. Por lo tanto, los factores dentro del contexto tecnológico pueden ser considerados como que están heterogéneamente distribuido a través de los competidores por el momento. c. La tercera pregunta a contestar es si dichos factores son imperfectamente movibles o no. Si la respuesta es No, entonces se tiene una ventaja competitiva temporal, pero si la respuesta es Si, entonces tendríamos una VCS. Los factores mencionados del contexto tecnológico definitivamente son movibles, por lo que no pueden ser considerados como una VCS, sino más bien como ventaja competitiva temporal.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 33-52

www.jistem.fea.usp.br

46 Chung, C. K. K.,Cruz, R. G.

En conclusión, el sistema del comercio electrónico es considerado valioso para las empresas puesto que los resultados valoran en forma positiva a los beneficios netos de implementar dicho sistema dentro de la empresa. Las variables que conforman el CONTEC son los únicos que tienen un impacto positivo con los beneficios netos. Actualmente, los tres FCE que componen el CONTEC están heterogéneamente distribuidos a través de los competidores, y esto lo confirmamos a través de las pocas empresas que utilizan el CE en sus actividades comerciales. Aunque dichos FCE estén colocados en forma heterogéneamente entre los competidores, no podemos afirmar que puedan convertirse en VCS por ser perfectamente movible con el tiempo según la teoría de los recursos y capacidades. Ver Figura 3.

Figura 3. Sistema del comercio electrónico en Paraguay y ventaja competitiva

Fuente: Elaboracion propia

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 33-52

www.jistem.fea.usp.br

Factores críticos de éxitos en busca de la ventaja competitiva dentro del Comercio Electronico: Un estudio empírico en las empresas paraguayas

CONCLUSIONES

Aunque el CE esté bastante difundido, en los países cuyas barreras como las TIC ya no se consideran un problema, existen aún países cuyo CE es considerado como algo novedoso y que aún no se encuentran en pleno uso. Según los autores Wareham et al. (2005), solamente un 2.7 % de los estudios científicos investigados han sido sobre los FCE del CE. En primer lugar, hemos adoptado los FCE del CE propuestos por los autores Teo, Ranganathan y Dhaliwal (2006) a nuestra realidad paraguaya, es decir, analizar si dicho FCE pueden llegar a considerarse como una VCS dentro de la realidad paraguaya. En segundo lugar, hemos obtenido resultados interesantes en el modelo como: - El único que obtuvo una relación positiva con los beneficios netos fue el factor del contexto tecnológico, mientras que el contexto organizativo y el contexto del entorno cuentan con una relación negativa con los beneficios netos. - Dentro del CONTEC, las variables con más peso fueron “Expertos en tecnología de información (TI) e infraestructura” y la “Interoperabilidad del Sistema”. En tercer lugar, los FCE que componen el CONTEC son las variables: “Expertos en tecnología de información (TI) e infraestructura”, “Problemas técnicos” e “Interoperabilidad del sistema”, y al mismo tiempo dentro de nuestro análisis empíricos hemos identificados que dichas variables tienen una relación positiva con los beneficios netos. Desde la perspectiva de la teoría de los recursos y capacidades, las variables mencionadas son consideradas como una ventaja competitiva temporal y no sostenible, por la simple razón de que dichos factores no pudieron obtener una respuesta afirmativa en cuanto a la tercera pregunta de la teoría de los recursos y capacidades, la de ser imperfectamente movible y difícil de imitar. Como hemos mencionado anteriormente, los FCE dentro del contexto tecnológico no son considerados como VCS dentro de la realidad paraguaya, pero dichos factores pueden ser utilizados en este mismo instante por las empresas (como pioneras) para adquirir nuevos mercados y ser una ventaja competitiva temporal, comparando con otras empresas en el mismo sector. Creemos que aunque estos FCE no puedan mantenerse por mucho tiempo dentro de una empresa como una VCS frente a sus competidores, la realidad paraguaya nos demuestra que hasta el momento no existe una conciencia en cuanto al uso de la herramienta del CE, por lo que esta inconciencia sobre los beneficios que el CE puede transformarse en una ventaja competitiva temporal para las empresas que sí utilizan el CE. Esta ventaja competitiva podrá ser sostenible en el tiempo, sólo si los competidores de la empresa no sobrepasan en conocimientos o tenga una mejor estrategia que ella, es decir, la empresa siempre debe sobresalir de entre sus competidores. Conclusiones finales sobre el CE en la realidad paraguaya: 1. Las empresas paraguayas que cuentan con el CE son prácticamente contadas, puesto que éstas no cuentan con una presencia en la red, es decir, no tienen una página electrónica corporativa, segundo, de las que tienen presencias son pocas las que utilizan el CE y por último, las que utilizan el CE no cuentan con un plan estratégico definido para su negocio ni tampoco cuentan con el recurso humano

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 33-52

www.jistem.fea.usp.br

48 Chung, C. K. K.,Cruz, R. G.

capacitado sobre el tema. Las dos barreras más notables del CE en las empresas paraguayas ya sea quizá por un lado, la falta de capital y por el otro lado, las carencias de recursos humanos. Por el primero entendemos que son los costos de inversión y están constituídos por los costos de conexión de Internet, costos de adquisión de equipos adecuados como hardware y software, costos de modificación organizacional, costos de implementación, costos de mantenimiento (Lefebvre et al., 2005) y finalmente los costos de entrenamiento que recalcan los autores Wang y Tsai (2002) y Claycomb et al. (2005). La razón principal se basa en que muchas de las empresas paraguayas son de tipo PYMES por lo que la carencia de capital para invertir en las infraestructuras tecnológicas son consideradas como un obstáculo para su implementación dichas tecnologías. Al mismo tiempo, también existe una carencia de conocimientos sobre el CE en forma general, y específicamente sobre el proceso de una transacción dentro de la red. 2. Las empresas paraguayas que poseen una página electrónica están en su mayoría posicionados en la categoría de los agentes que no adoptan el comercio electrónico en el presente, pero se encuentran en un estado de ver y esperar que pasará en el futuro, ésta clasificación pertenece a las investigaciones realizadas por Iivari y Janson (2003) de los procrastinadores, en donde las empresas cuentan con un bajo nivel de conocimientos sobre la tecnología del CE y del estudio de Lefebvre et al. (2005) sobre la trayectoria de la adopción del CE, en donde dichas empresas están situadas en el escenario 0, pertenecientes a los que no adoptan la tecnología pero que están interesados en incorporarlo en un futuro cercano. 3. En cuanto a los resultados del análisis se dieron a conocer que solamente el factor “CONTEC” ejercía una relación positiva sobre los “BENETO”, pero según la teoría de los recursos y capacidades, el recurso o capacidad que pueda ser movido e imitado, no puede considerarse como una VCS para las empresas (Mata et al., 1995). Por lo tanto, podemos concluir que los factores de mayor preocupación en las empresas paraguayas para conseguir el éxito del CE sean los factores que constituyen el CONTEC (Infraestructura Tecnológica del comercio electrónico, los expertos de la tecnología de información, los problemas sobre la seguridad y protección de datos e informaciones durante la transacción online y la interoperabilidad del sistema). Aunque el análisis haya arrojado dicho resultado, en el momento de la encuesta, estamos seguros que las empresas cambiarán su punto de vista una vez que el CE haya llegado a su madurez y que los factores también cambiarían con el tiempo, pudiendo sumarse otros recursos para convertirse en otro FCE del CE. 4. Sorprendentemente, el CONORG no ha tenido una relación positiva con los beneficios netos puesto que es considerado como una variable inmovible e inmitable por el hecho de posser las variables como el cambio organizacional, la estrategia del CE y el apoyo de la alta gerencia, pudiendo de este modo considerarse como una VCS, mientras que el contexto tecnológico que ha tenido un efecto positivo en los beneficios netos sólo puede ser considerado como una ventaja competitva temporal. El hallazgo de dicha investigación, considerado como interesante para algunos y por ser el pionero en el Paraguay sobre cuestiones de los FCE del CE dentro de la realidad paraguaya, podrá ser utilizado como una referencia valiosa e interesante para las empresas paraguayas que actualmente están presentes en la red como también para aquellas que aún no se encuentran en el proceso de implementar dicha tecnología, sino que se encuentran en la categoría de espera y ver que pasará en el futuro con sus competidores actuales y potenciales.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 33-52

www.jistem.fea.usp.br

Factores críticos de éxitos en busca de la ventaja competitiva dentro del Comercio Electronico: Un estudio empírico en las empresas paraguayas

5. Por ultimo, sería interesante agregar nuevas dimensiones al proyecto como ser la economía actual y futura, el cambio en los hábitos del consumidor en cuanto a las compras, el avance de la tecnología en el país u otros factores que también puedan afectar a las organizaciones actuales. 8.

LIMITACIONES

Al igual que todas las investigaciones, esta cuenta con varias limitaciones a saber: carencia de estudios que trate sobre los FCE del CE dentro de un sector industrial en particular (comercio, industrias u otros), escasez de estudios o modelos sobre el éxito del comercio electrónico, carencia de datos estadísticos sobre el CE. Y en cuanto a la realidad, las empresas paraguayas recién comienzan con la implantación del CE, por lo que las experiencias obtenidas de ellas no son las suficientes o adecuadas para profundizar el estudio empírico.

REFERENCES Alt, R. y Fleish, E. (2001). Business Networking Systems: Characteristics and Lessons Learned. International Journal of Electronic Commerce, vol. 5(2), 7-27. Al-Naeem, T; Rabhi, F. A.; Benatallah, B. y Ray, P. K. (2004/2005). Systemactic approaches for designing B2B applications. International Journal of Electronic Commerce, vol. 9(2), 41-71. Bailey, J. E. Y Pearson, S. W. (1983). Development of a tool for measuring and analyzing computer user satisfaction. Management Science, vol. 29(5), May, 530-545. Barua, A.; Konana, P y Whinston, A. B. (2004). An Empirical Investigation of NetEnabled Business Value. MIS Quarterly, vol. 28(4), 585-620. Basu, A. y Muylle, S. (2003). Authentication in E-Commerce. Communications of the ACM, vol. 46(12), 159-166. Chan, C. y Swatman, P. M. C. (2000). B2B e-commerce implementation: The case of BHP steel. Journal of Internet Research, vol. 10(1), 72-82. Chatterjee, D.; Grewal, R. y Sambamurthy, V. (2002). Shaping up for e-commerce: Institutional enablers of the organizational assimilation of web technologies. MIS Quarterly, vol. 26(2), 65-89. Chircu, A. M. y Kauffman, R. J. (2000). Limits to Value in Electronic CommerceRelated IT Investments. Journal of Management Information Systems, vol. 17(2), 5980. Clark, D. (2000). E-Commerce (A Special Report): The lessons we’ve learned-privacy: You have no Secrets-It’s the Dark side of E-Commerce: You leave a trail of personal information wherever you go. The Wall Street Journal, Octubre, 32.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 33-52

www.jistem.fea.usp.br

50 Chung, C. K. K.,Cruz, R. G.

Clay, K. y Strauss, R. P. (2002). Institutional barriers to electronic commerce: An historical perspective. New Institutionalism Strategic Management, vol. 19, 245-271. Claycomb, C.; Iyer, K. y Germain, R. (2005). Predicting the level of B2B e-commerce in industrial organizations. Industrial Marketing Management, vol. 34(3), 221-235. Cloete, E.; Courtney, S. y Fintz, J. (2002). Small business acceptance and adoption of ecommerce in the Western Cape Province of South Africa. Electronic Journal on information systems in Developing countries, vol. 10(4), 1-13. Cronin, J. J. y Taylor, S. A. (1992). Measuring Service Quality: A Reexamination and Extension. Journal of Marketing, vol. 56, 55-68. Czuchry, A. J.; Yasin, M. M. y Bayes, P. (2001). Are you in control of your ecommerce strategy?. Information Strategy, vol. 17(3), 6-11. Debreceny, R.; Putterill, M.; Tung, L. L. y Gilbert, A. L. (2002). New tools of the determination of e-commerce inhibitors. Decision Support Systems, vol. 34(2), 177-195. DeLone, W. H. y McLean, E. R. (2004). Measuring e-Commerce Success: Applying the DeLone & McLean Information Systems Success Model. International Journal of Electronic Commerce, vol. 9 (1), 31-47. Dedrick, J. y Kraemer, K. (2001). China IT report. The Electronic Journal on information systems in Developing countries, vol. 6(2), 1-10. Dinesh, A. M y Jaideep, M. (2001). Understanding Small Business Electronic Commerce Adoption:An Empirical Analysis. Journal of Computer Information Systems, vol. 41(3), 70-74. Eid, R. y Trueman, M. (2004). Factors affecting the success of business-to-business international Internet marketing (B-to-B IIM): an empirical study of UK companies. Industrial Management & Data Systems, vol. 104(1), 16-30. Evans, C. (2001). An E-Strategy for Online E-business. Information Systems Management, vol. 18(4), 8-22. Gable, G. G.; Sedera, D. y Chan, T. (2003). Enterprise systems success: a measurement model. Proceedings Twenty-Fourth International Conference on Information Systems, 576-591. Gebauer, J.; Beam, C. y Segev, A. (1998). Impact of the internet on procurement. Acquisition Review Quarterly, vol. 5(2), 167. George, D. y Mallery, P. (1995). SPSS/PC+Step by step: a simple guide and reference. California. Wadsworth Publishing Company. Gibbs, J.; Kraemer, K. L. y Dedrick, J. (2003). Environment and policy factors shaping global E-commerce diffusion: A cross-country comparison. Information Society, vol. 19(1), 5-18. Grewal, R.; Comer, J. M. y Mehta, R. (2001). An Investigation into the Antecedents of Organizational Participation in Business-to-Business Electronic Markets. Journal of Marketing, vol. 65(3), 17-33. Hempel, P. S. y Kwong, Y. K. (2001). B2B e-commerce in emerging economies: Imetal.comâ&#x20AC;&#x2122;s non-ferrous metals exchange in china. Journal of Strategic Information Systems, vol. 10(4), 335-355.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 33-52

www.jistem.fea.usp.br

Factores críticos de éxitos en busca de la ventaja competitiva dentro del Comercio Electronico: Un estudio empírico en las empresas paraguayas

Hong, W. y Zhu, K. (2006). Migrating to internet-based e-commerce: Factors affecting e-commerce adoption and migration at the firm level. Information & Management, vol. 43, 204-221. Iivari, J. y Janson, M. (2003). Analysis of Electronic Commerce Adopter Categories in Retailing: The Case of Automobile Dealerships. Journal of Organizational Computing and Electronic Commerce, vol. 13(1), 25-55. Iron, J. K. (2000). Tax risks every e-commerce executive should know. Information Strategy, vol. 16(3), 18-29. Iver, K. N. S.; Germain, R. y Frankwick, G. L. (2004). Supply chain B2B e-commerce and time-based delivery performance. International Journal of Physical Distribution & Logistics Management, vol. 34(8), 645-661. Joseph, W.; Chen, H. y Hwang, S. (2001). E-commerce Web site design: strategies and models. Information Management & Computer Security, vol. 9(1), 5 – 12. Kim, J. B. (2006). Determinants of B2B Adoption in South Korean Fishery Wholesale Firms: An Exploratory Study. Reviews in Fisheries Science, vol. 14, 285-301. Landrum, H.; Prybutok, V. R. y Zhang, X. (2007). A comparison of Magal’s service quality instrument with SERVPERF. Information & Management, vol. 44, 104-113. Lefebvre, L.; Lefebvre, E.; Elia, E. y Boeck, H. (2005). Exploring B-to-B e-commerce adoption trajectories in manufacturing SMEs. Technovation, vol. 25, 1443-1456. Liu, D. y Shen, M. (2004). Business-to-business workflow interoperation based on process-views. Decision Support Systems, vol. 38(3), 399-420. Liu, C. y Arnett, K. P. (2000). Exploring the factors associated with Web site success in the context of electronic commerce, Information & Management, vol. 38(1), 23-33. Mata, F. J; Fuerst, W. L. y Barney, J. B. (1995). Information Technology and sustained competitive advantage: A resource-based analysis. MIS Quarterly, vol. 19(4), 487-505. Molla, A. y Licker, P. (2001). Electronic Systems Success: An attempts to extend and respecify the Delone and Maclean Model of IS success. Journal of Electronic Commerce Research, vol. 2(4), 131-141. Molla, A. y Licker, P. (2005). Perceived E-Readiness Factors in E-Commerce Adoption: An Empirical Investigation in a Developing Country. International Journal of Electronic Commerce, vol. 10(1), 83-110. Miller, H. (2005). Information quality and market share in electronic commerce. Journal of Services Marketing, vol. 19(2), 93-102. Miller, J. y Doyle, B. A. (1987). Measuring the Effectiveness of Computer-Based Information Systems in the Financial Services Sector. MIS Quarterly, Marzo, 107-124. Ngai, E. W. T. y Gunasekaran, A. (2004). Implementation of EDI in Hong Kong: an empirical analysis. Industrial Management & Data Systems, vol 104(1), 88-100. Rajkumar, T. M. (2001). E-procurement: Business and technical issues. Information Systems Management, vol. 18(4), 52-60. Reichheld, F. F. y Schefter, P. (2000). E-Loyalty: your secret weapon on the web. Harvard Business School Press. July-August, 105-113.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 33-52

www.jistem.fea.usp.br

52 Chung, C. K. K.,Cruz, R. G.

Soliman, K. (2003). Internet-based Business-to-Business electronic commerce: A CIOâ&#x20AC;&#x2122;s perspective. Information Systems Management, vol. 20(1), 35-41. Srivinasan, R.; Lilien, G. L. y Rangaswamy, A. (2002). Tecnological Opportunism and Radical Technology Adoption: An Application to E-Business. Journal of Marketing, vol. 66(3), 47-60. Sung, T. K. (2004). E-commerce critical success factors: East vs. West. Technological Forecasting and Social Change, vol. 73(9), 1161-1177. Sung, T. K. y Gibson, D. V. (1998). Critical Success Factors for Business Reengineering and Corporate Performance: The Case of Korean Corporations. Technological Forecasting and Social Change, vol. 58(3), 297-311. Sung, T. K. y Gibson, D. V. (2005). Critical Success Factors in Electronic commerce: Korean Experiences. Journal of Organizational Computing and Electronic Commerce, vol. 15(1), 19-34. Teo, S. H. y Ranganathan, C. (2004). Adopters and non-adopters of business-tobusiness electronic commerce in Singapore. Information & Management, vol. 42(1), 89102. Teo, S. H.; Ranganathan, C. y Dhaliwal, J. (2006). Key Dimensions of Inhibitors for the Deployment of Web-Based Business-to-Business Electronic Commerce. IEEE Transactions on engineering Management, vol. 53(3), 395-411. Tsai, H. T.; Huang, L. y Lin, C. G. (2005). Emerging e-commerce development model for Taiwanese travel agencies. Tourism Management, vol. 26(5), 787-796. Wade, M. R. y Nevo, S. (2005). Development and Validation of a Perceptual Instrument to Measure E-Commerce Performance. International Journal of Electronic Commerce, vol. 10(2), 123-146. Wang, J. C. y Tsai, K. H. (2002). Factors in Taiwanese Firmsâ&#x20AC;&#x2122; Decisions to Adopt Electronic Commerce: An Empirical Study. The World Economy, vol. 25(8), 11451167. Wareham, J., Zheng, J. G. y Straud, D. (2005). Critical themes in electronic commerce research: a meta-analysis. Journal of Information Technology, vol. 20(1), 1-19. Webb, K. (2002). Managing channels of distribution in the age of electronic commerce. Industrial Marketing Management, vol. 31(2), 95-102. Zhu, K. y Kraemer, K. (2002). e-Commerce metrics for net-enhanced organizations: Assessing the value of e-commerce to firm performance in the manufacturing sector. Information Systems Research, vol. 13(3), 275-295. Zhu, K., Kraemer, K. y Xu, S. (2003). Electronic business adoption by European firms: A cross-country assessment of the facilitators and inhibitors. European Journal of Information Systems, vol. 12(4), 251-268.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 33-52

www.jistem.fea.usp.br

JISTEM - Journal of Information Systems and Technology Management Revista de Gestão da Tecnologia e Sistemas de Informação Vol. 11 No.1,Jan/Apr 2014, pp. 53-64 ISSN online: 1807-1775 DOI: 10.1590/S1807-17752013000100004

CONVERGENCE IN INFORMATION AND COMMUNICATION TECHNOLOGY (ICT) USING PATENT ANALYSIS Eunhee Kim Jaejon Kim Joon Koh Chonnam National University, Buk-gu, Gwangju, Korea __________________________________________________________________________

ABSTRACT Since the 1990s, information and communication technology (ICT) has been perceived as the critical technology for economic development, and the ICT industry itself has been growing exceptionally fast. Moreover, technology convergence in ICT has received particular attention. ICT innovations diffuse into existing products and thus come to form a new integral part of the goods. This is an exploratory research to examine technology convergence of the supply side as a firm level in the ICT sector using International Patent Classification (IPC) of 43,636 sample patents from 1995 to 2008. This study finds a degree of merger and relationships between different technology domains through the association rule mining of patent coclassification. This type of analysis helps companies to take strategies under the environment of technological trajectory change. Keyword: technological convergence; information and communication technology (ICT); patent; international patent classification (IPC); co-classification; association rule mining

_____________________________________________________________________________________ Manuscript first received/Recebido em 01/01/2013 Manuscript accepted/Aprovado em: 01/10/2013 Address for correspondence / Endereço para correspondência Eunhee Kim, is an assistant professor in the Department of Business Administration, Chonnam National University in Korea. She holds a Ph.D. in Technology Management from Seoul National University, a MS in Technology Management from State University of New York at Stony Brook, and a BS in Computer Science from Ewha Womans University. She worked at LG-EDS and EDS Korea as a system engineer and at Industrial Bank of Korea’s Economic Research Institute as a research fellow. E-mail: eheekim@chonnam.ac.kr Jaejon Kim, is a professor in the Department of Business Administration, Chonnam National University. He received his Ph. D. at the Arizona State University. His research interests are in the DSS, GDSS, and e-business areas. E-mail: jaejonkim@hotmail.com Joon Koh, is an associate professor in the Department of Business Administration, Chonnam National University. He received his Ph.D. in MIS from the Korea Advanced Institute of Science and Technology (KAIST). His research interests are in the areas of virtual community, communities-of-practice, and knowledge management. Chonnam National University, 300 Yongbong-dong, Buk-gu, Gwangju, Korea. E-mail: kjoon@chonnam.ac.kr Published by/ Publicado por: TECSI FEA USP – 2014 All rights reserved.

54 Kim, E., Kim, J., Koh, J.

1. INTRODUCTION Since the 1990s, information and communications technology (ICT) has been perceived as the essential technology for economic development, and ICT industries themselves have been growing exceptionally fast. Moreover, technology convergence has received particular attention. ICT innovations diffuse into existing products and thus come to form a new integral part of the goods. The creation of synergies, blurring of industry boundaries, integration, and overlapping of markets are all used to describe convergence. The convergence phenomenon has been mainly observed and discussed in ICT sectors. ICT innovations diffuse into existing products and thus come to form a new integral part of the goods. Patents play an increasingly important role in innovation and patent data are used to indicate innovative activity of companies, industries and countries. Patent analysis can be regarded as one of the most effective methods to keep in touch with technology trends (Karvonen and Kassi, 2010). Scholars used patent analysis to discuss convergence phenomenon (Duysters and Hagedoorn, 1998; Gambardella and Torrisi, 1998; Bröring, 2005; Curran and Leker, 2011; Kavon and Kassi, 2010). This is an exploratory research to examine technology convergence of the supply side as a firm level in the ICT sector using patent analysis. International Patent Classification (IPC) of 46,363 sample patents from 1995 to 2008 is employed to find a degree of merger and relationships between different technology fields through the association rule mining of patent co-classification. The paper is structured as follows: Section 2 outlines shortly the background of convergence and presents the use of patent analysis. Section 3 presents the research setting, including research data and methodology. Section 4 indicates the empirical results. Finally, a discussion of this study and future research strands conclude this paper in Section 5. 2. BACKGROUND Convergence is an often used but rarely defined concept. Ideas such as the creation of synergies, disappearance of industry boundaries, integration, or overlapping of markets, are all used to describe this phenomenon. Technological convergence is the tendency for different technological systems to evolve towards performing similar tasks (Wikipedia). Convergence can refer to previously separate technologies such as voices, data and videos that now share resources and interact with each other, synergistically creating new efficiencies. The phenomenon of convergence occurs when innovations emerge at the intersection of established and clearly defined industry boundaries, thereby sparking off an evolutionary development with a much broader impact. In recent industry developments within information technology (IT), bio-technology (BT) and nano-technology (NT), the convergence of technologies and knowledge bases has induced a variety of industrial points of inflection. Hence, industry boundaries have become blurred, and innovation does not take place within previously existing industrial silos anymore, but rather between them (Hacklin et al., 2009). The dictionary definition of convergence is ‘tendency to meet at a point’ or ‘gradual change so as to become similar or develop something in common’. The first use of the term convergence can be traced back to Rosenberg (1963) who introduced the label

JISTEM, Brazil Vol. 11 No.1,Jan/Apr 2014, pp. 53-64

www.jistem.fea.usp.br

Convergence in Information and Communication Technology (ICT) using patent analysis

‘technological convergence’ as a way to describe the evolution towards a specialized machine tool industry in the US in the late 1800s. Rosenberg’s notion of technological convergence appears to have re-emerged in recent decades as way of describing the apparent merger of telecom, data communication, IT, media and entertainment into a giant ICT and multimedia industry (Gambardella and Torrisi, 1998). During the 90s, convergence was mainly discussed in the context of the merger of the IT, telecommunications, media and entertainment industries into a giant ‘infocom’ sector (Lind, 2004). According to a study (Greenstein and Khanna 1997), it is possible to define two basic forms of convergence as substitutes and complements. Competitive convergence as a ‘Substitutes Paradigm’ occurs when products or services become interchangeable one for another to fulfill a set of certain user needs through bundling of functions. Complementary convergence as a ‘Cooperative Paradigm’ occurs when products or services from different industries are merged to meet a larger or new set of consumer needs simultaneously. Pennings and Puranam (2001) classified industry convergence from two dimensions such as substitution/complementation and supply/demand. Stieglitz (2003) suggested a similar classification but the dimension of supply/demand with the dimension of technology–based/product-based. Studies of Hacklin (2008) and Hacklin et al (2009) developed and discussed a process of four sequential convergence stages, which are knowledge, technology, application and industry convergence, with an evolutionary perspective. Curran and Leker (2011) discussed how to measure convergence under the sequential process which is evolving when scientific disciplines and technologies and/or markets have converged. Starting with scientific disciplines that begin to use more and more research results of one another, a scientific convergence will start with cross-disciplinary citations and eventually develop further into closer research collaborations. After the distance between basic science areas has been decreasing for some time, applied science and technology development should follow leading to technology convergence (Meyer, 2000; Murray, 2002; Bainbridge, 2006). Pennings and Puranam (2001) argued that based on a validity assumption for classification schemes like the IPC (International Patent Classification), convergence can be found in patent data through growing overlapping among IPCs and through an increase in patent citations between different classes. Patent analysis has been employed in the context of technology-driven convergence of electronics, computers, and telecommunication (Duysters and Hagedoorn, 1998; Gambardella and Torrisi, 1998; Bröring, 2005) as patents are often regarded as outcome indicators for organizations' R&D activities (Ernst, 1995; Fai and Tunzelmann, 2001). Curran and Leker (2009; 2011) discussed convergence indicators using patent data. Also Xing and et al (2011) tried to measure industry convergence with input-out analysis. 3. RESEARCH METHODS AND SAMPLES In this research, association rule mining is used to analyze technology integration and diversification in a firm level. Data mining, which is referred to as a knowledge discovery in a database, is a process of nontrivial extraction of implicit previously unknown and potentially useful information such as knowledge rules, constraints and regularities from data in a database (Chen et al., 1996). Data mining, which differs from traditional statistics in that formal statistical inference, is assumption-driven in the sense that a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery-driven in the sense that patterns and hypotheses are automatically extracted from data (Zhang and Zhang, 2002). Data mining has made broad and significant

JISTEM, Brazil Vol. 11 No.1,Jan/Apr 2014, pp. 53-64

www.jistem.fea.usp.br

56 Kim, E., Kim, J., Koh, J.

progress since its early beginning in the 1980â&#x20AC;&#x2122;s. Today, data mining is used in a vast array of areas, and numerous commercial mining systems are available (Han et al. 2006). Association mining is one of the best-studied methods in data mining (Agrawal et al., 1993; Agrawal and Srikant, 1994; Chen et al., 1996; Han and Kamber, 2001). Since its introduction in 1993 (Agrawal et al.), the area of association rule mining has received a great deal of attention. Association rule mining has been developed mainly to identify the relationships strongly associated among item sets that have high frequency and strong correlation. Association rules are produced by finding the interesting associations or correlation relationships among a large set of data items (Jiao and Zhang, 2005), and enable us to detect the items that frequently occur together in an application (Zhang and Zhang, 2002). An association rule (Agrawal et al., 1993) is a probabilistic relationship, of the form A -> B, where A, B are disjoint itemsets. The intuitive meaning of such a rule is that the transactions (or tuples) that contain itemset A also tend to contain itemset B. An association rule indicates that the occurrence of a certain itemset in a transaction will imply the occurrence of another itemset in the same transaction. The rule suggests that a strong relationship exists between the itemsets. The association analysis is applicable to market basket data, bioinformatics, medical diagnosis, Web mining, and scientific data analysis (Tan et al., 2005). The importance of a rule is usually measured by two numbers, support and confidence. These two properties provide the empirical basis for derivation of the inference expressed in the rule and a measure of the interest in the rule. The support for the association rule A -> B is the percentage of occurrences that contain both itemset A and B among all transactions. The confidence for the rule A -> B is the percentage of transactions that contain an itemset B among the transactions that contain an itemset A (Tan et al., 2005). The rule A -> B holds with support s if s% of transactions in the database contain both itemset A and itemset B. The rule A -> B holds with confidence c if c% of transactions that contain itemset A also contain itemset B. Association rule mining finds all rules in the database that satisfy some minimum support and minimum confidence constraints (Agrawal and Srikant 1994). Additionally, lift value (Brin et al., 1997) is used to judge the strength of an association rule. The lift of a rule A -> B is defined as support (AUB) / (support (A)*support (B)). A lift ratio greater than 1.0 suggests that there is some usefulness to the rule. The larger the lift ratio, the greater the strength of the association (Kim and Park, 2006). In this study an itemset is regarded as a set which includes primary patent classification code and secondary patent classification codes of a patent, and a transaction means granting a patent. A patent has its classification code, which indicates its technology area. Patent information includes one primary classification code and additionally second classification codes. The patent co-classification shows clearly convergence between different technological domains (Curran and Leker, 2011). In this research, the association rule mining is applied to discover the linkage patterns, which show strongly related among various technology areas, based on the patent coclassification information. In order to examine the technological convergence, 46,363 granted patents by IBM from 1995 to 2008 in USPTO are used in this study. IBM holds more patents than any other technology company, and has topped the list of the worldâ&#x20AC;&#x2122;s most inventive companies from 1993.

JISTEM, Brazil Vol. 11 No.1,Jan/Apr 2014, pp. 53-64

www.jistem.fea.usp.br

Convergence in Information and Communication Technology (ICT) using patent analysis

The International Patent Classification (IPC) provides for a hierarchical system of language independent symbols for the classification of patents and utility models according to the different areas of technology to which they pertain (WIPO). Each patent grant is assigned to IPC to determine the nature of the patent. One patent can be assigned to more than one IPC if the patent finds application in various domains. Each company has a few subclasses to which most of their patents are assigned. These subclasses describe their core technological competencies. If a company has granted patents only in a few subclasses, it can be said that the technologies employed by the company are highly focused on a narrow field of expertise. On the other hand, if all the patents are not concentrated in a few subclasses, research can be said to be diversified. The IPC codes in a firm’s patent records are identified and classified into technology fields representing the firm’s technology domains. Patent application in each field indicates an accumulation of knowledge and advancement in the technological trajectory (Fai and Tunzelmann, 2001). IPC codes are a hierarchical way of assigning the category to which every patent belongs. There are 8 sections, 120 classes, 628 subclasses and about 70,000 groups. The 628 subclasses are aggregated into 35 technological fields, and for descriptive purposes these are further aggregated into five main categories: electrical engineering, instruments, chemistry, mechanical engineering and others, and 35 sub categories in the appendix, the IPC and Technology Concordance Table (WIPO, 2008). The subclasses of the sample patents are analyzed using association rule mining in this research. Among the 43,636 sample patents used for this analysis, 13,338 patents were assigned to more than one IPC.. 4. ANALYSIS In this study, a software package, R, is used to analyze the patent dataset. For the association rule mining, the ‘apriori( )’ algorithm, which is well known and included in the R package, ‘arules’ is executed. Thresholds for mining this dataset are 0.05% for minimum support and 90% for minimum confidence. The lift values of all association rules as the results of the association rule mining are greater than 1.0, indicating the usefulness of the rules and the strength of the association. The table 1 shows the IPC statistics of the sample patent dataset. During fourteen years from 1995 to 2008, IBM’s patents were diversified within 355 different technology fields. The major IPC subclass codes of the dataset are ‘G06F’ and ‘H01L’ as indicated in table 1. More than 60% of the company’s granted patents during this period are included in these two subclasses. In this study, the conditions for association rule mining of IPC co-classification, minimum support is 0.05% which is lower than a general threshold value. Support value to detect association rules is related to the frequency of occurrence. The main purpose of this study is to examine the technological convergence in a patent using its IPC coclassification. As displayed in the table1, more than half of the company’s patents are focused on two technological domains. And others are dispersed into more than 300 different subclasses. In order to detect the association rules between different technology subclasses, the minimum support value should be lowered in this study. Table 2 shows the results of association rule mining for co-classification of the IPC dataset. The association rules detected from the patent dataset shows clearly the relationships between different fields of technology. As mentioned above, subclasses of IPC can be grouped into 35 technology fields as IPC and Technology concordance table (WIPO, 2008). The 32 association rules can be interpreted based on the concordance

JISTEM, Brazil Vol. 11 No.1,Jan/Apr 2014, pp. 53-64

www.jistem.fea.usp.br

58 Kim, E., Kim, J., Koh, J.

table. The detected 32 association rules can be divided into five types. First, the association rules from #1 to #17 present the relationships between ‘computer technology’ and other fields such as ‘telecommunications’, ‘digital communication’ and ‘semiconductors’ in electrical engineering. Second, the association rules from #18 to #26 show the technological combinations between electrical engineering and instrument. Third, the association rule #27 indicates the technological integration among three domains: electrical engineering, instruments, and mechanical engineering fields. Fourth, the association rules #28, #29, and #30 show the technological combinations between electrical engineering and mechanical engineering. Fifth, the last two association rules #31 and #32 present the relationships between ‘organic fine chemistry’ and ‘biotechnology’ in chemistry. These two rules do not designate the convergence between ‘computer technology’ which the company are focused on and technology fields in chemistry. They indicate the combinations between ‘organic fine chemistry’ and ‘macromolecular chemistry, polymers’ fields. Table 1. Patent Statistics for major IPC subclasses of sample dataset (n=43,636) Year

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

Total

155

364

1,666

3,495

4,411

4,432

4,067

3,599

3,996

3,695

3,602

3,451

4,443

4,987

46,363 100.0%

Count of patent Major IPC subclass G06F

183

876

1,720

2,093

2,087

1,778

1,781

2,070

1,946

1,872

1,763

2,289

2,647

23,182

50.0%

H01L

288

438

452

455

386

548

542

543

538

671

626

5,616

12.1%

H04L

108

113

157

155

165

131

153

146

135

125

144

174

1,720

3.7%

G11B

206

184

185

153

135

1,424

3.0%

G01R

112

102

105

125

137

1,045

2.2%

101

106

103

163

133

1,040

2.2%

G11C H05K

107

821

1.7%

G06K

732

1.5%

H03K

618

1.3%

G09G

105

103

553

1.1%

509

1.1%

G06Q G10L H04M

500

1.0%

463

1.0%

Table 2. Association rules* for IPC co-classification of sample dataset** No.

Antecedent

Consequent

Support

Confidence

{G06F,G06K,H04K,H04N}

{H04L}

0.0005

1.0000

{G06F,G06K,H04L,H04N}

{H04K}

0.0005

1.0000

{G06K,H04K,H04L,H04N}

{G06F}

0.0005

1.0000

{G06K,H04K,H04N}

{H04L}

0.0005

1.0000

JISTEM, Brazil Vol. 11 No.1,Jan/Apr 2014, pp. 53-64

www.jistem.fea.usp.br

Technology categories*** I

Convergence in Information and Communication Technology (ICT) using patent analysis 5

{G06K,H04K,H04N}

{G06F}

0.0005

1.0000

{G06K,H04L,H04N}

{H04K}

0.0005

1.0000

{G06K,H04L,H04N}

{G06F}

0.0005

1.0000

{G06F,H04K,H04N}

{H04L}

0.0007

1.0000

{H04K,H04L,H04N}

{G06F}

0.0007

1.0000

{G06F,G06K,H04K}

{H04L}

0.0010

1.0000

{G06F,G08C,H04L}

{H04J}

0.0016

0.9565

{G06K,H04K}

{H04L}

0.0010

0.9333

{G06K,H04K,H04L}

{G06F}

0.0010

0.9286

{G06N}

{G06F}

0.0092

0.9248

{G08C,H04J,H04L}

{G06F}

0.0016

0.9167

{H04K,H04N}

{H04L}

0.0007

0.9000

{H04K,H04N}

{G06F}

0.0007

0.9000

{G01R,G06F,G08C,H04L}

{H04J}

0.0014

1.0000

{G01R,G08C,H04J,H04L}

{G06F}

0.0014

1.0000

{G01R,G08C,H04L}

{H04J}

0.0014

1.0000

{G01R,G08C,H04L}

{G06F}

0.0014

1.0000

{G01R,G08C,H04J}

{G06F}

0.0016

1.0000

{G01R,G08C}

{G06F}

0.0017

1.0000

{G01R,G06F,G08C}

{H04J}

0.0016

0.9565

{G01R,G08C}

{H04J}

0.0016

0.9565

{G08B,H04L}

{G06F}

0.0013

0.9000

{G06F,G21K}

{G03F}

0.0006

1.0000

I, II,IV

{G21C}

{G06F}

0.0012

1.0000

I,IV

{F28F,H01L}

{H05K}

0.0015

1.0000

{F28F}

{H05K}

0.0043

0.9661

{C07H}

{C12Q}

0.0005

1.0000

{C12M}

{C12Q}

0.0006

1.0000

I, II

III

* Thresholds for association rule mining; minimum support - 0.05%, minimum confidence 90% ** The number of patents with more than one IPC: 13,338 *** 5 technology categories: I. Electrical engineering II. Instruments III. Chemistry IV. Mechanical engineering

5. DISCUSSION AND FUTURE DIRECTIONS This study aims at examination of technology convergence of the supply side in the ICT sector through patent IPC analysis. Based on IPC of 46,363 sample patents from 1995 to 2008 in ICT, association rule mining of the sample patentsâ&#x20AC;&#x2122; IPC is used to find degree of overlaps and relationships between different technology domains. The results of association rule mining of the ICT firmâ&#x20AC;&#x2122;s patent co-classification show clearly JISTEM, Brazil Vol. 11 No.1,Jan/Apr 2014, pp. 53-64

www.jistem.fea.usp.br

60 Kim, E., Kim, J., Koh, J.

convergence between different technological domains. Technological convergence can trigger market convergence with new product and firms begin to merge with each other, completing the convergence process with industry fusion, considering sequential process of convergence. Curran and Leker (2011) discussed the phase of convergence based on the assumption of an idealized time series of events: scientific/knowledge convergence, technology convergence, market/applicational convergence, and industry convergence. Also, as scholars suggested in the previous studies, convergence can be considered in several dimensions: supply/demand, substitution/complementation, and product-based/ technology-based. The context of this study is the ICT technology convergence in a firm level as a technology supplier, considering degree and scope of technology convergence. The results imply that the technology convergence in a firm occurs mainly within its dominant technology areas and that the major technologies tend to be merged with other areasâ&#x20AC;&#x2122; technology, expending the scope of convergence. However, the results do not indicate the technological convergence change technological paradigm in a firm level. The scope of the convergence can be related to the firmâ&#x20AC;&#x2122;s capability of innovativeness and technological competitiveness. As discussed in previous literature (Curran and Leker, 2011), convergence starts with knowledge convergence. Additionally, this study examines the usefulness of association rule mining to indicate technological convergence. Due to the technology development, new phenomena have appeared in the world. Some of them can be difficult to be described or analyzed by conventional methods. The association rule mining approach is appropriate for describing the complicate relational data and discovering important patterns among them. The association rule mining analysis can be applied to indicate not only technology convergence but also knowledge convergence. Technological convergence, as an emerging research field, is being studied by many scholars. The impact of this new phenomenon is enormous. For example, after introducing the smartphone which is a representative outcome of the convergence, lots of new business models and applications related to the smartphone have appeared, and our life style has changed rapidly. In order to improve the capability of forecasting the technology development, in-depth understanding of the technological convergence phenomenon is essential. Even though, in this research, one factor, which is classification code of patent, is considered to discuss technological convergence, knowledge transfer with a perspective of open innovation can be deliberated together using the mining analysis. Therefore, the empirical analysis about the relationship between technology convergence and knowledge will be discussed in a further study.

JISTEM, Brazil Vol. 11 No.1,Jan/Apr 2014, pp. 53-64

www.jistem.fea.usp.br

Convergence in Information and Communication Technology (ICT) using patent analysis

REFERENCES Agrawal, R., Imielinski, T., Swami, A. (1993) ‘Mining association rules between sets of items in large databases’, Proceedings of ACM SIGMOD, 207-216. Agrawal, R., Srikant, R. (1994) ‘Fast algorithms for mining association rules’, Proceedings of the 20th international conference on very large databases, 487-499. Bainbridge, W.S., (2006) ‘Transformative Concepts in Scientific Convergence’, Annals of the New York Academy of Sciences 1093, 24–45. Brin, S., Motwany, R., Ullman, J.D. and Tsur, S. (1997) ‘Dynamic itemset counting and implication rules for market basket data’, Proceedings of ACM SIGMOD, pp.255–264. Bröring, S. (2005) The Front End of Innovation in Converging Industries: The case of nutraceuticals and functional foods, DUV, Wiesbaden, Germany, 342. Curran, C., Leker, J. (2009) ‘Seeing the Next iPhone Coming Your Way: How to Anticipate Converging Industries’, PICMET 2009 Proceedings, August 2-6, Portland, Oregon USA. Curran, C., Leker, J. (2011) ‘Patent indicators for monitoring convergence – examples from NFF and ICT’, Technological Forecasting & Social Change 78, 256-273. Duysters, G., Hagedoorn, J. (1998) ‘Technological convergence in the IT industry: the role of strategic technology alliances and technological competencies’, International Journal of the Economics of Business 5 (3), 355–368. Ernst, H. (1995) ‘Patenting strategies in the german mechanical engineering industry and their relationship to company performance’, Technovation 15 (4), 225–240. Fai, F., Tunzelmann, V. N. (2001) ‘Industry-specific competencies and converging technological systems: evidence from patents’, Structural change and Economic Dynamics (12), 141-170. Gambardella, A., Torrisi, S. (1998) ’Does technological convergence imply convergence in markets? Evidence from the electronics industry’, Research Policy 27, 445–463 Greenstein, S., Khanna, T. (1997) ‘What does industry convergence mean’, in Yoffie, D. (ed.) (1997) Competing in the age of digital convergence, Boston, MA, Harvard Business School Press. Hacklin, F. (2008) Management of Convergence in Innovation – Strategies and Capabilities for Value Creation Beyond Blurring Industry Boundaries, Physica-Verlag, Heidelberg Hacklin, F., Marxt, C., Fahrni, F. (2009) ‘Coevolutionary cycles of convergence: An extrapolation from the ICT industry’, Technological Forecasting & Social Change 76, 723–736. Han, J., Kamber, M., & Pei, J. (2006) Data mining: concepts and techniques. Morgan kaufmann. Jiao, J., Zhang, Y. (2005) ‘Product portfolio identification based on association rule mining’, Computer-Aided Design 37(2), 149-172. Karvonen, M., Kässi, T. (2010) ‘Patent Citations as a Method for Analysing Industrial Convergence’, PICMET 2011 Proceedings of Technology Management for Global Economic Growth. Kim, E. H., Park, Y. (2006) ‘An exploratory study of risks in information system development projects: using association rule mining’, International Journal of Technology Intelligence and Planning, 2(4), 404-417. Meyer, M. (2000) ‘Does science push technology? Patents citing scientific literature’, Research Policy 29 (3), 409–434.

JISTEM, Brazil Vol. 11 No.1,Jan/Apr 2014, pp. 53-64

www.jistem.fea.usp.br

62 Kim, E., Kim, J., Koh, J.

Lind, J. (2004) ‘Convergence: History of term usage and lessons for firm strategists’, In ITS 15th Biennial Conference, Berlin, Germany, International Telecommunications Society (ITS). Murray, F. (2002) ‘Innovation as co-evolution of scientific and technological networks: exploring tissue engineering’, Research Policy 31 (8–9), 1389–1403. Pennings, J. M., Puranam, P. (2001) ‘Market convergence & firm strategy: new directions for theory and research’, ECIS Conference, The Future of Innovation Studies, Eindhoven, Netherlands. Rosenberg, N. (1963) ‘Technological Change in the Machine Tool Industry, 1840–1910’, The Journal of Economic History 23(4), 414-443. Stieglitz, N. (2003) ‘Digital Dynamics and Types of Industry Convergence – The Evolution of the Handheld Computers Market in the 1990s and beyond’, in Christensen F. J., Maskell P. (eds.) The Industrial Dynamics of the New Digital Economy, London: Edward Elgar. Tan, P., Steinbach, M. Kumar, V. (2005) Introduction to Data Mining, Addison-Wesley. Wing, W., Ye, X., Kui, L.. (2011) ‘Measuring convergence of China’s ICT industry: An inputoutput analysis’, Telecommunications Policy 35, 301-313. WIPO (2008) World Patent Report: A Statistical Review Zhang, C., Zhang, S. (2002) Association Rule Mining: models and algorithms, Berlin: Springer.

Appendix IPC code descriptions of sample dataset IPC code

Description

PHYSICS

G01

MEASURING; TESTING

G01R

MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES

G06

COMPUTING; CALCULATING; COUNTING

G06F

ELECTRIC DIGITAL DATA PROCESSING

G06K

RECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS

G06Q

DATA PROCESSING SYSTEMS OR METHODS, SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTING PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTING PURPOSES, NOT OTHERWISE PROVIDED FOR EDUCATING; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS

G09 G09G G10

ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION MUSICAL INSTRUMENTS; ACOUSTICS

G10L

SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; AUDIO ANALYSIS OR PROCESSING

G11

INFORMATION STORAGE

G11B G11C

INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER STATIC STORES

ELECTRICITY

H01

BASIC ELECTRIC ELEMENTS

H01L

SEMICONDUCTOR DEVICES; ELECTRIC SOLID STATE DEVICES

H03

BASIC ELECTRONIC CIRCUITRY

H03K

PULSE TECHNIQUE

H04

ELECTRIC COMMUNICATION TECHNIQUE

H04L

TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION

H04M

TELEPHONIC COMMUNICATION

H05

ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR

H05K

PRINTED CIRCUITS; CASINGS OR CONSTRUCTIONAL DETAILS OF ELECTRIC APPARATUS; MANUFACTURE OF ASSEMBLAGES OF ELECTRICAL COMPONENTS

JISTEM, Brazil Vol. 11 No.1,Jan/Apr 2014, pp. 53-64

www.jistem.fea.usp.br

Convergence in Information and Communication Technology (ICT) using patent analysis

IPC and Technology Concordance Table (WIPO, 2008) Field of Technology I: Electrical engineering Electrical machinery, apparatus, energy Audio-visual technology Telecommunications Digital communication Basic communication processes Computer technology IT methods for management Semiconductors II: Instruments Optics Measurement Analysis of biological materials Control Medical technology III: Chemistry Organic fine chemistry Biotechnology Pharmaceuticals Macromolecular chemistry, polymers Food chemistry Basic materials chemistry Materials, metallurgy Surface technology, coating Micro-structural and nanotechnology Chemical engineering

Environmental technology IV: Mechanical engineering Handling Machine tools Engines, pumps, turbines Textile and paper machines Other special machines Thermal processes and apparatus Mechanical elements Transport V: Other fields Furniture, games Other consumer goods Civil engineering

Definition for energy technology Solar energy (includes solar photovoltaic power and solar thermal power)

Fuel cells technology Wind energy technology

International Patent Classification (IPC) Symbols F21#, H01B, H01C, H01F, H01G, H01H, H01J, H01K, H01M, H01R, H01T, H02#, H05B, H05C, H05F, H99Z G09F, G09G, G11B, H04N-003, H04N-005, H04N-009, H04N-013, H04N-015, H04N-017, H04R, H04S, H05K G08C, H01P, H01Q, H04B, H04H, H04J, H04K, H04M, H04N-001, H04N-007, H04N-011, H04Q H04L H03# (G06# not G06Q), G11C, G10L` G06Q H01L G02#, G03B, G03C, G03D, G03F, G03G, G03H, H01S G01B, G01C, G01D, G01F, G01G, G01H, G01J, G01K, G01L, G01M, (G01N not G01N-033), G01P, G01R, G01S; G01V, G01W, G04#, G12B, G99Z G01N-033 G05B, G05D, G05F, G07#, G08B, G08G, G09B, G09C, G09D A61B, A61C, A61D, A61F, A61G, A61H, A61J, A61L, A61M, A61N, H05G (C07B, C07C, C07D, C07F, C07H, C07J, C40B) not A61K, A61K-008, A61Q (C07G, C07K, C12M, C12N, C12P, C12Q, C12R, C12S) not A61K A61K not A61K-008 C08B, C08C, C08F, C08G, C08H, C08K, C08L A01H, A21D, A23B, A23C, A23D, A23F, A23G, A23J, A23K, A23L, C12C, C12F, C12G, C12H, C12J, C13D, C13F, C13J, C13K A01N, A01P, C05#, C06#, C09B, C09C, C09F, C09G, C09H, C09K, C09D, C09J, C10B, C10C, C10F, C10G, C10H, C10J, C10K, C10L, C10M, C10N, C11B, C11C, C11D, C99Z C01#, C03C, C04#, C21#, C22#, B22# B05C, B05D, B32#, C23#, C25#, C30# B81#, B82# B01B, B01D-000#, B01D-01##, B01D-02##, B01D-03##, B01D-041, B01D-043, B01D-057, B01D-059, B01D-06##, B01D-07##, B01F, B01J, B01L, B02C, B03#, B04#, B05B, B06B, B07#, B08#, D06B, D06C, D06L, F25J, F26#, C14C, H05H A62D, B01D-045, B01D-046, B01D-047, B01D-049, B01D-050, B01D-051, B01D-052, B01D-053, B09#, B65F, C02#, F01N, F23G, F23J, G01T, E01F-008, A62C B25J, B65B, B65C, B65D, B65G, B65H, B66#, B67# B21#, B23#, B24#, B26D, B26F, B27#, B30#, B25B, B25C, B25D, B25F, B25G, B25H, B26B F01B, F01C, F01D, F01K, F01L, F01M, F01P, F02#, F03#, F04#, F23R, G21#, F99Z A41H, A43D, A46D, C14B, D01#, D02#, D03#, D04B, D04C, D04G, D04H, D05#, D06G, D06H, D06J, D06M, D06P, D06Q, D99Z, B31#, D21#, B41# A01B, A01C, A01D, A01F, A01G, A01J, A01K, A01L, A01M, A21B, A21C, A22#, A23N, A23P, B02B, C12L, C13C, C13G, C13H, B28#, B29#, C03B, C08J, B99Z, F41#, F42# F22#, F23B, F23C, F23D, F23H, F23K, F23L, F23M, F23N, F23Q, F24#, F25B, F25C, F27#, F28# F15#, F16#, F17#, G05G B60#, B61#, B62#, B63B, B63C, B63G, B63H, B63J, B64# A47#, A63# A24#, A41B, A41C, A41D, A41F, A41G, A42#, A43B, A43C, A44#, A45#, A46B, A62B, B42#, B43#, D04D, D07#, G10B, G10C, G10D, G10F, G10G, G10H, G10K, B44#, B68#, D06F, D06N, F25D, A99Z E02#, E01B, E01C, E01D, E01F-001, E01F-003, E01F-005, E01F-007, E01F-009, E01F-01#, E01H, E03#, E04#, E05#, E06#, E21#, E99Z International Patent Classification (IPC) Symbols F03G 6/06, F24J 2/00, F24J 2/02, F24J 2/04, F24J 2/05, F24J 2/06, F24J 2/07, F24J 2/08, F24J 2/10, F24J 2/12, F24J 2/13, F24J 2/14, F24J 2/15, F24J 2/16, F24J 2/18, F24J 2/23, F24J 2/24, F24J 2/36, F24J 2/38, F24J 2/42, F24J 2/46, F03G 6/06, G02B 5/10, H01L 31/052, E04D 13/18, H01L 25/00, H01L 31/04, H01L 31/042, H01L 31/052, H01L 31/18, H02N 6/00, E04D 1/30, G02F 1/136, G05F 1/67, H01L 25/00, H01L 31/00, H01L 31/042, H01L 31/048, H01L 33/00, H02J 7/35, H02N 6/00 H01M 4/00, H01M 4/86, H01M 4/88, H01M 4/90, H01M 8/00, H01M 8/02, H01M 8/04, H01M 8/06, H01M 8/08, H01M 8/10, H01M 8/12, H01M 8/14, H01M 8/16, H01M 8/18, H01M 8/20, H01M 8/22, H01M 8/24 F03D*, B60L 8/00

JISTEM, Brazil Vol. 11 No.1,Jan/Apr 2014, pp. 53-64

www.jistem.fea.usp.br

64 Kim, E., Kim, J., Koh, J.

JISTEM, Brazil Vol. 11 No.1,Jan/Apr 2014, pp. 53-64

www.jistem.fea.usp.br

JISTEM - Journal of Information Systems and Technology Management Revista de Gestão da Tecnologia e Sistemas de Informação Vol. 11, No. 1, Jan/Apr., 2014 pp. 65-84 ISSN online: 1807-1775 DOI: 10.1590/S1807-17752014000100005

USE OF THE RFID TECHNOLOGY TO OVERCOME INEFFICIENCIES IN THE PRODUCTION PROCESS: AN ANALYSIS OF A MICROCOMPUTER COMPANY IN ILHÉUS – BAHIA Frederico Wergne de Castro Araújo Filho X. L. Travassos Paulo S. Figueiredo Integrated Center of Manufacture and Technology – National Service of Industrial Learning – DR, Salvador, Bahia - Brazil _______________________________________________________________________ ABSTRACT This paper presents a methodology for the integration of RFID technology into microcomputer assembly companies. Such a technology enables inventory control, tracking of parts/products and customer satisfaction. The methodology was developed to identify and evaluate the production process in an assembly company and to apply the process FMEA to evaluate potential faults and propose improvement actions for the use of RFID technology. The methodology was validated in a company from the Pólo de Informática de Ilhéus/BA. In this study, the viability of RFID technology for inventory control and traceability of the product was confirmed. Among the benefits obtained were the reduction of one of the workers from the inventory team, reduction in the costs of substituting a component, and reduction in lead-time. Keywords: RFID; Process FMEA; Site Survey; Process Automation; Inventory Management

INTRODUCTION

According to Whang (2010), Radio-Frequency Identification (RFID) has been hailed as a major innovation to enhance the efficiency of inventory management (Gaukler et al. 2004) and supply chain management (Heinrich 2005). The basic premise _____________________________________________________________________________________ Manuscript first received/Recebido em: 14/09/2012 Manuscript accepted/Aprovado em: 05/02/2014 Address for correspondence / Endereço para correspondência Frederico Wergne de Castro Araújo Filho. Integrated Center of Manufacture and Technology – National Service of Industrial Learning – DR. Frederico has an MSc from Senai Cimatec. His research interests are Automating Manufacturing Systems and RFID. Adress: Av. Orlando Gomes 1845 – Piatã, Salvador, Bahia - Brazil, 41650-050. Phone: 71.3462-9500/ Fax: 71.3462-9599 E-mail: wergne@fieb.org.br X. L. Travassos . Federal University of Santa Catarina (UFSC). He finished his PhD at the L’Ecole Centrale de Lyon (France). His research interests are Numerical Modeling and Industrial Engineering. Adress: Rua Presidente Prudente de Moraes, 406, Santo Antônio, 89218-000 - Joinville, SC - Brasil. Phone: 47.3461-5900 E-mail: lucas.travassos@ufsc.br Paulo S. Figueiredo. Integrated Center of Manufacture and Technology – National Service of Industrial Learning – DR. Paulo has a Doctoral Degree from Boston University and does research in Operations Management. His research interests include System Dynamics modeling and Product Pipeline Management. Adress: Av. Orlando Gomes 1845 – Piatã, Salvador, Bahia - Brazil, 41650-050. Phone: 71.3462-9500/ Fax: 71.3462-9599 E-mail: paulo_s_figueiredo@hotmail.com Published by/ Publicado por: TECSI FEA USP – 2014 All rights reserved.

66 Araújo Filho, F. W. C. , Travassos X. L. T., Figueiredo, P. S.

of RFID is that a radio frequency reader can read as many as 200 tags in a second without the line-of-sight requirement. Near-real-time tracking and tracing capabilities, together with item-level identification through EPC (electronic product code) standard, would allow a host of new improvements in inventory management (Whang, 2010). Among these improvements is increased efficiency by making products available at the right time and in the right place, with a lower operational cost. As a result, numerous manufacturing plants are adopting RFID technology for product tracking purposes, especially within sectors that move multiple parts through their facilities each day (Chow et al. 2006, Kach and Borzabad 2011). According to Stambaugh and Carpenter (2009), RFID systems are rapidly replacing Universal Product Codes (barcodes in many applications). According to Soares et al. (2008), inventories are the sectors that can potentially benefit the most from RFID technology. The benefits with its implementation go far beyond the mere change of the technology of automation of data capture. Its main objective is to increase the integration of the entire logistics chain, providing tools that enable tracking and control of the products that circulate in the chain (Fosso Wamba and Chatfield, 2009). This case study initiates in 2008, motivated by research done in many microcomputer assembly companies of the Pólo de Informática de Ilhéus/BA. This group of companies is comprised of 20 assemblers of microcomputers. The raw materials are practically all imported from Asian suppliers (China, South Korea and Thailand, among others) and the companies supply the Brazilian market, specially the north and northeast regions of the country. The difficulties faced by these companies in managing their inventories of raw materials and finished products, even using barcodes for quick identification during production, were the motivational factor behind this study. These difficulties motivated the development of the methodology of integration presented here. The most relevant difficulties found in companies of the region are described below:  Sales of products that did not have all the necessary components as ordered by the client. In such cases, the company frequently substituted the missing component(s) for a component of higher value. This might create a discrepancy between the inventory control system (database) and the actual inventory. RFID may help, in this case, by identifying and tracking all items contained in a product.  Traceability of components during the phases of assembly – when one component was damaged and was substituted for another one The control of this activity is performed, in most companies, by inspection, i.e. by manually filling a spreadsheet at the end of the working hours, and then inputting the information into the inventory control system (database). Such method was performed in the company being studied.  Problems while reading the barcodes during the phases of data insertion into the ERP (Enterprise Resource Planning) system of the companies. In case there were many metal pieces, the RFID reading was performed with redundancy by means of a portal and a manual reader; 

Manual control of the raw materials inventory.

This research has the objective of evaluating the specific conditions of adoption of an RFID system in a microcomputer assembly company, situated in the Pólo de Informática de Ilhéus/BA. The system aims at an adequate return on investment to JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 65-84

www.jistem.fea.usp.br

Use of RFID technology to overcome inefficiencies in the production process: An analysis of a microcomputer company in Ilhéus – BA, Brazil

modernize the company with an up-to-date solution. It is noteworthy that no other company in the region possesses this technology. The actual Internal Rate of Return (IRR) of the project was calculated by Araújo Filho et al. (2013) and its value is 15%. RFID technology presents some advantages when compared to the barcode technology, previously used in the company. The method used to read the RFID tags enables them to be read at great speed compared to the barcode. The reading of an RFID tag does not depend on the line-of-sight requirement, and it can be reutilized or operated in hazardous environments or environments contaminated by dirt. The reutilization process is simple. It is just necessary to remove the used tags from the items and reattach them to new items. This paper outlines the activities that were undertaken by a Brazilian microcomputer assembly company while implementing RFID technology. We explore the steps taken by the firm and outline where the company met its expectations or not. The microcomputer industry relies heavily on logistical techniques and inventory management, moving many parts throughout their system, received from multiple suppliers. As such, we believe our analysis can be useful for other firms hoping to use RFID technology to reduce their systematic inefficiencies. The cost of the tag is a problem to be faced, but it is not insurmountable. Tag prices tend to reduce over time and tags can be reutilized. The innovative side of this research is the creation of a methodology for the integration of the RFID technology into the productive system of a company, to enable the adoption of such technology in companies that aim to increase and improve inventory control and product traceability. It is important to point out that this methodology aims to reduce potential impacts on the structure of the company, in order to facilitate integration. To our knowledge, this is the first methodology that integrates Failure Mode and Effect Analysis (FMEA) into the RFID adoption process. FMEA is used to identify the process failures, which might compromise the integration of the new system with the previous system. Other benefits inherent to the utilization of this technology were observed: reduction in lead-time, reduction of failures in the process and reduction of defects (since it became possible to identify and trace problems before the product was sent to customers). Such benefits were enabled due to the real time control, the capacity to read many tags at the same time, and the suppression of the manual reading of barcodes. Furthermore, all the logistic chain to which the company belongs receives benefits, as demonstrated by Fosso Wamba and Chatfield (2009), because with real time inventory control, company suppliers will be able to monitor their inventory and fulfill their clients’ needs more precisely. Therefore, the fundamental question for this study is “will implementing an RFID-based system reduce inefficiencies in the production process of the company in question?” 2.

RFID TECHNOLOGY

RFID technology has its origins in the Second World War, with the IFF – Identification Friend-or-Foe identification system, which allowed Allied units to distinguish between Allied and enemy aircraft (Hunt et al. 2007). RFID technology

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 65-84

www.jistem.fea.usp.br

68 AraĂşjo Filho, F. W. C. , Travassos X. L. T., Figueiredo, P. S.

works on the same basic principle. A signal is sent to a tag, which is activated and reflects back the signal (passive system) or transmits its own signal (active system). According to Pinheiro (2006), the considerable advances of RFID technology are undeniable, however many challenges remain in order to enable its widespread use. These challenges can be perceived by the analysis of applications for the RFID gadgets. For some applications, the technology is reasonably consolidated, while for others there is still a need of new gadgets, security protocols and reductions in the price of tags. In this section, a literature review is presented, with a list of advantages and disadvantages of RFID technology. Such factors were taken into consideration when the company decided to implement RFID technology. Advantages of the use of RFID technology In a multitude of industrial contexts, implementing RFID technology has shown to improve operations, supply chain and logistical systems by reducing forecasting errors and increasing delivery times (Dutta et al. 2007, Kach and Borzabad 2011). The use of RFID technology seems particularly useful within fast moving consumer goods markets, since it allows for real-time visibility of the supply chain (Bottani et al. 2010). Higher material transparency within organizational systems allows for more accurate forecasting, effectively reducing any inefficiencies that may exist within the supply chain (Delen et al. 2007, Kach and Borzabad 2011). RFID can also be beneficial in managing supply chains. Relying on an increasingly large number of suppliers can make the information flow across all parties difficult. RFID tags and the technology behind them allow for rapid automated item identification without the need of a physical operator, increasing inventory allocation efficiencies. Through the provision of real-time data, RFID technology helps in systematically reducing information gaps across the supply chain. It prevents items from being lost, since they are tracked continuously as they move through the chain, and enables inventory control, since the company knows exactly what is contained in its inventory, in real time (Attaran 2007; Gaukler et al. 2007, Kach and Borzabad 2011). Disadvantages of the use of RFID technology RFID technology does not guarantee improved processes across all functions of the supply chain (Wu et al. 2006). Just like any technology, RFID has downsides and risks (Kharif 2005; Taghaboni-Dutta and Velthouse 2006). To begin with, there are costs and capital requirements for implementing the physical technology and software needed for RFID (Ustundag and Tanyas 2009). Justifying the startup costs may be a prohibitive barrier for implementing RFID technology (Kach and Borzabad 2011). Second, there are cases that suggest how RFID may not be so effective and other cases where the implementation of the system was not successful (Kharif 2005). Additionally, lack of timing and technology coordination between suppliers when implementing RFID practices can create problems across the supply streams (Whang 2010). When to implement RFID technology is an important question facing many practitioners and researchers alike (Kach and Borzabad 2011). In relation to the problem identified in the case study, the main advantages and disadvantages hold the premise that the technology can be implemented; however, it is necessary to identify the process failures, which might compromise the integration with the previous system. This was done by applying FMEA. Therefore, in order to enable the integration process, it is necessary to identify the process failures (listed below in item 3) so that they be evaluated, eliminated or in the worst scenario, minimized. In order to do so, a Failure Mode and Effect Analysis FMEA study was carried out and is JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 65-84

www.jistem.fea.usp.br

Use of RFID technology to overcome inefficiencies in the production process: An analysis of a microcomputer company in Ilhéus – BA, Brazil

presented and explained below. FMEA was one of the first systematic techniques for failure analysis. It was developed by reliability engineers in the 1950s to study problems that might arise from malfunctions of military systems. An FMEA is often the first step of a system reliability study. It involves reviewing as many components, assemblies, and subsystems as possible to identify failure modes, and their causes and effects (Santos and Cabral, 2008). 3.

PROCESS FMEA

Among the many tools used for fault analysis (Ishikawa diagram, Fault Tree Analysis, Pareto Chart, 5 Whys, Quality Control story, among others), FMEA was the most adequate for the company, because the use of this methodology shows the process systematically, aggregates more information in the descriptions of the process and identifies causes of faults in the process of microcomputer assembly (Santos and Cabral, 2008). Failure Mode and Effects Analysis (FMEA) was one of the first systematic techniques for failure analysis. It was developed by reliability engineers in the 1950s to study problems that might arise from malfunctions of military systems. An FMEA is often the first step of a system reliability study. It involves reviewing as many components, assemblies, and subsystems as possible to identify failure modes, and their causes and effects. For each component, the failure modes and their resulting effects on the rest of the system are recorded in a specific FMEA worksheet. There are numerous variations of such worksheets. An FMEA is mainly a qualitative analysis According to Stamatis (2003), the factors, severity, occurrence and detection are calculated by experts according to the scale of values, which in most cases is set from 1 to 10, based on criteria of evaluation defined in common agreement by the team. The RPN (Risk Priority Number) is a measure of risk of a fault in a certain step of the process, and should be used to classify faults and prioritize the actions. Actions are recommended with priority given to those faults that achieved the highest RPN. Using this concept, a methodology was developed to integrate RFID technology into the supply chain of the microcomputer assembly company, which will be presented below. This methodology has not been tested before. This was the first attempt to systematize it. The methodology can be applied to the other companies in the Pólo de Informática de Ilhéus, since no other company possessed RFID technology at the time the study was done. In the global market, there are still many microcomputer companies that still have not made use of RFID technology. These companies could benefit from its many advantages, as listed in section 5. 4.

METHODOLOGY

Figure 1 presents the flowchart of the methodology that was developed, detailing each step of the technical viability analysis. It reports the procedures that guided the case study in the company (which will be named XYZ for confidentiality purposes). This company was chosen because it had all the limitations and characteristics of other companies of the region, as mentioned previously. Even though the methodology contemplates a return on investment analysis, this step will not be presented in this paper for brevity purposes. It is suffice to say that the JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 65-84

www.jistem.fea.usp.br

70 Araújo Filho, F. W. C. , Travassos X. L. T., Figueiredo, P. S.

Internal Rate of Return (IRR) was 15%, ensuring a large NPV (Net Present Value) for the firm (Araújo Filho et al., 2013). For other papers focusing on economic feasibility, check Balocco et al.(2011) and Jones et al. (2007). The problem of the tag price was minimized by choosing an economically feasible tag and by reutilizing the tags as much as possible. Describe the Manuf. Process

Perform Site Survey

Describe the factory and its manuf. process

Identify interrogation zones

Perform static capture

Develop the flowchart of the manuf. process

Eliminate noise Yes

Any eletromagnetic noise?

Identify clients’ needs

Perform itinerant capture Apply Process FMEA

Eliminate Noise

Perform test on the- spot

Detail the recomm. actions

Create selected infrastructure

Select the RFID Equipment

Yes

Any eletromagnetic noise?

Use the FMEA form

Perform Pilot Test

Select antenna and reader as needed by the process

Does it work? Select - Potential tags

Yes Perform financial viability study

Make a performance test on the tags

Calculate NPV & IRR No

Does it work?

Validate technically -economically

Yes Yes

Is it viable?

Figure 1 – Flowchart of the Methodology

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 65-84

www.jistem.fea.usp.br

Validate Technically

Use of RFID technology to overcome inefficiencies in the production process: An analysis of a microcomputer company in Ilhéus – BA, Brazil

4.1. Description and evaluation of the process In this section, a description of the factory of company XYZ will be presented, together with a description of the microcomputer assembly process from the receipt of raw materials to the checkout of finished products. 4.1.1. Description of the Factory The computer factory was built in 2008 in the Pólo industrial de Informática in Ilhéus/BA, with the objective of producing desktop computers and laptops to fulfill the needs of the north and northeast regions of Brazil. The company has a flexible assembly line, which manufactures 5 desktop models and 1 laptop model. The monthly production of the factory is approximately 12 thousand machines. The factory has a total area of 1134 m2, with a production area of 895 m2. Around 200,000 machines will be traced and tracked. For details on costs, please refer to Araujo Filho et al. (2013). We assumed that all the equipment would be tagged. 4.1.2. Description of the Process of Receiving Supplies The process of receiving supplies is initiated with the arrival of the product in the inventory sector. A worker checks the data on the invoice and the data in the request for parts. After the checking of documents, a physical checking of the products is performed, as well as a checking of the packaging and possible damages caused by transportation. When the new process is initiated, the components will receive an RFID tag when they are first inspected. All the checking data are registered in the stamp of inspection and acceptance. In case there is no problem with the material, either physical or documental, it is taken into the inventory. If there is a problem, a communication to the Board of Directors of the company is performed, which in turn will decide if the goods will be given back to the supplier or if they will be accepted into the inventory. After the goods are put into the inventory, barcodes are generated and printed for the components. Just after the printing of barcodes, they are attached to the components and read by an optical reader, so that the information enters the Integrated Enterprise Control System (IECS). Finally, the components are stored in the inventory. 4.1.3. Description of the Process of Computer Assembly The process of computer assembly is initiated with the generation of a sales order (SO) by the ERP system of the sales team located in the southeast of the country. The inventory manager generates the Production Order (PO) in an ERP system different from the one used by the sales team. Therefore, the factory is obliged to feed the system manually because there is no integration between databases. This was changed radically with the new process. Once the RFID began to be used, the factory fed the system automatically by using new software that was integrated with the existing system. In case a few items are not in the inventory, they are substituted for equivalent or superior items. In this phase, the barcodes for the power supply unit and for the lot of products (serial number) are generated. The RFID tags will help in this phase because it will be possible to know exactly what is in stock. After this phase, the PO is printed and taken to the inventory team. The team separates the cases and the power supply units, according to the PO, and the other parts (Motherboard, Hard Disc, Processor and DVD) in the inventory. Once the RFID system is used, all the parts will arrive already with their tags. JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 65-84

www.jistem.fea.usp.br

72 Araújo Filho, F. W. C. , Travassos X. L. T., Figueiredo, P. S.

The cases and power supply units are taken to the Case Preparation sector (which also attaches the power supply unit). Parallel to this activity, the other parts are checked by the “KIT preparation team”. In this stage, the motherboard, processor, cooler and memory are attached. The inspection process uses software to determine the condition of the parts. After the case is prepared with the power source, the stages of assembly are performed:  1st Post: Attach HD, DVD and serial number to the case and attach barcode (BC) of the power supply unit. 

2nd Post: Attach front part of the case.

 3rd Post: Connect frontal cables (LEDs, USB, POWER and RESET) on the motherboard.  4th Post: Attach KIT to the case and connect power source cable to the motherboard, HD and DVD. 

5th Post: Tie the power cables with clamps.



6th Post: Attach SATA cable to HD and DVD.



7th Post: Fasten sides of the case.

 8th Post: Insert Configuration Label on the side of the case, connect the power cable to outlet and perform a pre-test and setup of the machine (motherboard, processor, memory, DVD, frontal USB). In case the machine is approved, it is made available on the rack for run in. If it is reproved, it is taken to be repaired. Once the new process is implemented, RFID tags will be attached individually to groups of similar components, as described in section 4.3.3. The tags did perform well with parts made of metal. After the approval in the assembly line, the machines are made available to run. In this stage, the Operational System (OS) is installed and a fatigue test is performed. The next step is the final test for the machine. In this test, both hardware and software are tested. In the end of this step the machine is packaged together with a keyboard, loudspeakers, a mouse, manuals, CD of drivers and the label with its serial number is attached to the box. The machine is then put on the pallet. The pallets are stored in the inventory. When an invoice is generated, the inventory team ties (via barcode [BC] reader) the serial number of the machine to the invoice using the software system provided. As mentioned previously, once the new process begins, RFID tags will be attached to groups of similar components during the initial inspection. The results of the use of RFID tags are discussed in the final section. 4.1.4. Description of the process of Generating an Invoice for a Sales Request The process of generating an invoice for a sales request initiates with the sales request by the ERP system. The inventory team releases the serial number of the machines, which are on the pallet to compose the amount of machines requested in the sales invoice. After the machines are chosen, the sales invoice is created and the inventory team makes pallets available for removal, by manually checking the invoice numbers with the labels in the machines. Once the new process is initiated, the pallets will receive RFID tags and will be associated to the computers.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 65-84

www.jistem.fea.usp.br

Use of RFID technology to overcome inefficiencies in the production process: An analysis of a microcomputer company in Ilhéus – BA, Brazil

4.2. Applying FMEA to the manufacturing Process of the Company The FMEA form, used for the analysis of the manufacturing process of the company, was developed by a team of four experts (Microelectronics Lab of the Manufacture and Technology Integrated Center - CIMATEC). The most important results of this work were the possibility of evidencing the potential fault modes, their causes and the alternatives to prevent the occurrence of faults. The FMEA form is an established methodology with proven results (Santos and Cabral, 2008; Stamatis, 2003) . The methodology is applied by filling the FMEA form and analyzing it. In order to fill the FMEA form, the team followed the steps explained in section 3. The improvement actions were focused on the types of operations in which the Risk Priority Number (RPN) was above 125. On the FMEA form, 50 types of operations were identified. Seven of these operations (1, 11, 28, 30, 31, 34 and 43) were the ones with the highest RPN, according to Table 1 below:

KIT Preparation Reading error Delay in the (motherboard in barcodes KIT release to 11 , processor, 7 (BCs) of the assembly cooler and components line RAM Memory) Wrong input Inconsistent of BC tag of data in the the parent 28 Packing ERP system 7 serial number (Finished on the product) package

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 65-84

Raw material control in stock

Damaged tag

PRESENT PREVENTIVE CONTROL

PRESENT DETECTION CONTROL

Anual inventory ERP (IECS) (Manual / BC)

Improper manual 4 operation

www.jistem.fea.usp.br

RPN

Replacement by component Production Unavailable with Order Component in 5 equivalence Generation stock higher or lower than SO

POTENTIAL CAUSE (6M) MECHANISMS

DETEC.

POTENTIAL POTENTIAL FUNCTION EFFECTS OF FAULT MODE FAULTS

OCURR.

ITEM

SEVERITY CLASS.

Table 1 – FMEA Form

4 160

Visual Inspection 5 175 Visual / (ERP) System

Visual inspection

5 140

74 Araújo Filho, F. W. C. , Travassos X. L. T., Figueiredo, P. S.

Generation of Invoice

PRESENT DETECTION CONTROL

RPN

PRESENT PREVENTIVE CONTROL

Wrong selection of I/O (mouse, Packed board, machine out 8 speakers, user of spec. of PO manual, driver CDs)

Improper manual 4 operation

Visual inspection

6 192

Input in packing with insufficient number of Packed I/O(mouse, machine out 8 board, of spec. of PO speakers, user manual,driver, CDs)

Improper manual 4 operation

Visual inspection

6 192

Reading error of BC of Delay in the parent serial Raw material release of 34 number to set 7 entry invoice with the amount of the machines machines on the invoice

Machine 43 Repair

POTENTIAL CAUSE (6M) MECHANISMS

DETEC.

POTENTIAL POTENTIAL EFFECTS OF FAULT MODE FAULTS

OCURR.

FUNCTION

SEVERITY CLASS.

ITEM

Table 1 – FMEA Form

Reading error of BC of Delay in the component for component insertion in release the ERP system (IECS)

Damaged tag

Visual inspection / 5 175 (ERP) System

The recommended actions in the FMEA Process are listed below: 1. Real Time Inventory: There were faults due to unavailability of components in the inventory, caused by imprecise control of raw materials. It was found that 30% of the Production Orders were generated by substituting components for others with higher or lower value or grade. It was also found that the inventory of parts and finished products was performed once a year, and demanded 3 days to be executed. This happened because some steps of the inventory checking were performed manually and the results were compared with the values found in the ERP system. Therefore, real time inventory is recommended for the company. 2. Substitution of Barcodes for RFID: During the stages of KIT preparation, packaging, generation of invoice and acquisition of raw materials, a series of faults in the reading of barcodes was found. It happened with both the components and the parent serial number of the machines. It also occurred that a damaged label was attached to the JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 65-84

www.jistem.fea.usp.br

Use of RFID technology to overcome inefficiencies in the production process: An analysis of a microcomputer company in IlhĂŠus â&#x20AC;&#x201C; BA, Brazil

machine. Because of that, the recommended action is substituting the barcodes (BCs) for RFID tags, because the RFID tag is more resistant to dirt, impact and mechanical friction than the BC label. It is impossible to guarantee that all the tags were adequately read by the equipment, but the error rate of the RFID readers is extremely small according to specifications (less than 1%). 3. Identification of Parts with RFID Tags: Faults were detected in the selection of I/O products (mouse, keyboard, loudspeakers, and also user manual and CDs of drivers) and in the insertion of these parts in the packaging due to wrong manual operation. Therefore, the recommended action is to identify the I/O products with RFID tags and to group them with a RFID tag with the parent serial number of the packaging of the product, via ERP system. Even though the RFID tag may be expensive in relation to the cost of the items, they can be used many times (recycled), so their cost is reduced. The human cost for the process of attaching the tags to the body of the components was not calculated. For a more detailed analysis of the procedures, check Araujo Filho, Travassos and Figueiredo (2013). 4.3. Measurements and tests In this section, we will explain the steps of the technical viability analysis. The first subsection presents the selection of the RFID equipment. The second subsection presents the analysis performed in the environment to detect electromagnetic noise (site survey), at the points where the RFID equipment were to be installed. 4.3.1. Selection of the Most Adequate RFID Equipment For the selection of the proper and most adequate RFID equipment, a preselection and analysis were performed taking into consideration firstly the technical information of different RFID tags. Reading tests were then performed with the preselected list of available tags, antennas and tag readers. We tested most of the components available in the Brazilian market, but we do not show them here for brevity purposes. The tags were put on components, on the computer case and on the packaging box and the tests were performed. The choice of readers and antennas was restricted to the available material in the CIMATEC lab. This equipment had already been tested and proved to be cost efficient and reliable. However, a more detailed analysis was not performed. The choice of tag, however, was performed considering technical and economical criteria. A list of nine tags covering the major types available on the market was analyzed and a series of specifications were considered to enable the choice of the most adequate one, both in terms of performance and in terms of cost. Among such criteria were cost, dimensions, distance for reading, memory size, mounting type, among others. With such criteria in hand, it was possible to determine which tags were a viable alternative for components and for the packaging box. A pre-selection was made. Table 2 presents the kinds of tags that were tested.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 65-84

www.jistem.fea.usp.br

76 Araújo Filho, F. W. C. , Travassos X. L. T., Figueiredo, P. S.

Table 2: Choice of RFID Tags

Model

Survivor

Confidex Cruiser

Confidex Corona

Confidex Pino

SteelWave

SteelWave Micro

Ironside

Confidex Halo

ALN-9640

Manufacturer

CONFIDEX

ALIEN

Dimensions

224 x 24 x 8 mm

16 x 74 x 0,3 mm

100 x 20,32 x 0,3 mm

75 x 14 x 0,4 mm

45 x 35 x 6 mm

13 x 38 x 3 mm

51,5 x 47,5 x 10 mm

60 x 12 x 14 mm

101,6 x 50,8 x 0,3 mm

865 - 869 MHz 902 - 928 MHz 952 - 955 MHz

865 – 869 MHz 902 - 928 MHz

865 - 869 MHz 902 - 928 MHz 952 - 955 MHz

840 – 960 MHz

Frequency Range

865 – 960 MHz

860 – 960 MHz

865 – 869 MHz 902 - 928 MHz

Transmission Protocol

EPC Class 1 Generation 2

EPC Class 1 Generation 2 ISSO 18000-6C

EPC Class 1 Generation 2

EPC Class 1 Generation 2 ISO 18000-6C

Distance from Reader

8 - 12 m

4–5m

5-9m

3–4m

4-6m

2,5 m

6-7m

5-7m

4-6m

IC and Memory Size

96-bit EPC Impinj Monza 240-bit EPC NXP G2 + NXP G2XM

96-bit EPC Impinj Monza 240-bit EPC + 512 NXP G2XM

NXP G2XM Up to 240bit EPC + 512-bit extended user memory

96-bit EPC + 64 bit Impinj Monaco/64 240-bit EPC + 512 NXP G2XM

96-bit EPC Impinj Monza 512-bit EPC NXP G2

Impinj Monza3 96-bit EPC

96-bit EPC Impinj Monza 240-bit EPC + 512 NXP G2XM

Alien Higgs-3 96-bit EPC

Attachment

Mechanical Adhesive

Adhesive

Mechanical Adhesive

Mechanical

Adhesive

Mechanical Adhesive

Mechanical

Adhesive

Working Temperature

-35°C +85°C

-35°C +160°C

-35°C +200°C

-35°C +60°C

-20°C - +85°C

-20°C +85°C

-55°C +105°C

-35°C +85°C

-40°C +70°C

Typical usage

Industrial and Logistics

Tire manufacturing

Industrial solutions

Wood Pallets

Assembly lines and warehouses

Petrochemical industry, automotive and maritime

Metallic surfaces

Document tracking, pallets and boxes

Not viable for some components (motherboard, cooler, DVD player, memory)

VIABLE

Not viable for some components (motherboard, cooler, DVD player, memory)

Not viable due to the attachment (clamp)

Not viable for metallic surfaces

Viable

Not viable due to the attachment (clamp)

Viable

Usage in computer components

Not viable due to the tag dimensions

Not viable for mettalic surfaces

Not viable due to the tag dimensions

Not viable due to the attachment to the component

Usage in Boxes*

Viable

Not viable due to attachment to box

Performance tests were done in a sequence. Deavours et al. (2005) recommend a performance test of an RFID tag in order to evaluate the potential success of the RFID system under study. The test consists in adjusting the power of the reader to the maximum position, and making 100 reading attempts with the tag from a fixed distance. In order to simulate other distances, the power of the reader should be attenuated by 0.5dBm for every 100 reading attempts. The rate of response is the number of successful readings divided by the reading trials. After the pre-selection of tags, such performance tests were done with the SteelWave Micro tag for the components (motherboard, HD, DVD, RAM memory, power supply unit and case) and with the ALN-9640 tag for the packaging box. All these tests were performed in the microelectronics lab. We must admit that results may vary, JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 65-84

www.jistem.fea.usp.br

Use of RFID technology to overcome inefficiencies in the production process: An analysis of a microcomputer company in Ilhéus – BA, Brazil

since the manufacturing environment is slightly different from the microelectronics lab (we tried to mimic the conditions as much as possible). We believe that our choice of tag was the best considering the availability in the market. The tag can be affixed to all parts including the RAM memory. The fixed distance used for the tests was approximately 70cm +/- 2cm. To ensure more reliability from results, 4 experiments were run for each tag, totaling 10,800 reading attempts. Readings were successful with metal parts and with the closed computer case. Figure 2 presents the tests performed on the machine’s components (motherboard, HD, DVD reader, RAM memory, power supply unit, case and packaging box).

Figure 2 – Performance test on the machine’s components

4.3.2. Site Survey According to Sweeney II (2005), in order to perform a site survey it is necessary to follow these steps: 1. Define the reading places (or areas of inquiry) where the RFID system will be installed. These locations vary from store to store and require a power outlet and Ethernet connectivity for readers (unless you have players with wireless connectivity). Commonly, the RFID interrogation zones are placed on doors, assembly lines and racks of inventory; 2. Carry out site survey in local reading respecting all the business process cycle (24h or 48h). It is usually held on days of normal operation of the company, respecting all hours of operation. 3. Perform the static capture. To perform the static capture, you must configure the equipment as follows: a. Place the half-wave antenna (attached directly to the tripod) so that the center of the antenna is in the center of the target area; b. Connect the antenna to the input port of the spectrum analyzer (via coaxial cable);

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 65-84

www.jistem.fea.usp.br

78 AraĂşjo Filho, F. W. C. , Travassos X. L. T., Figueiredo, P. S.

c. Connect the laptop to the spectrum analyzer, using an RS-232 connector or Ethernet. If you use a laptop to record data, you must configure the virtual display to record the information every time and save it on your hard disk. Or else, you should shoot/capture the monitor spectrum analyzer every two hours during the normal cycle of operation d. Set the spectrum analyzer for the operating frequency to be used with the RFID system; 4. Perform Itinerant Capture. To perform the itinerant capture, you must configure the equipment as follows: a. Set the spectrum analyzer exactly the same way as configured in static capturing, except for the voltage source and the antenna that should be mobile; b. b. If any source of interference is found, its exact location must be identified. 5. Map the areas of interrogation in the project, locating sources of interference. 4.3.2.1. Site Survey at XYZ The site survey was performed in the areas of interrogation, according to Figure 3, on the places where the KIT preparation (Figure 4), generation of invoice (Figure 5) and entry of raw materials occur (Figure 6). For the static capture, images from the spectrum analyzer were recorded every 2 hours in each place, with a cycle of 24 hours (beginning at 08:00 until 08:00 of the next day). However, the last record of each place corresponded to the period between 18:00 of the last day until 8:00 of the following day, because the company did not have any activity after regular working hours. This was performed inside the company. We identified some areas of frequency spectra of interrogation, but all spectra were outside the working range of the UHF (Ultra High Frequency).For the itinerant capture, no spectrum in the range of 902 to 907.5 MHz and in the range of 915 to 928MHz was recorded by the spectrum analyzer,.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 65-84

www.jistem.fea.usp.br

Use of RFID technology to overcome inefficiencies in the production process: An analysis of a microcomputer company in Ilhéus – BA, Brazil

Dipole Antenna

Figure 4 – KIT preparation (Zone 1) Dipole Antenna

Figure 5 – Invoice issuing (Zone 2) Dipole Antenna

Figure 3 – Static capture zone. Figure 6 – Raw material issuing (Zone 3)

4.3.3. Tests in the Product After the site survey tests, in-place tests were performed. The stages of these tests include:  Input of Components: Through data integration with ERP, the system prompts the user to issue tags for the volumes that comprise a given set / batch of components. Through the information contained in the invoice launched in the ERP system, data will be associated for later traceability. At this point, the components will be credited into the system, releasing them for use if necessary.  Inventory: After proper labeling of RFID tags in the volumes of components, these will be stored as usual.  Handling: With the aid of a RFID Data Collector integrated with the ERP system, the operator will locate the Production Order that lists which components and which amounts must be separated and sent to the production line. At this time, the separate components are read by the RFID Collector and receive the status of Booked, which means that these components can be used at any time. JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 65-84

www.jistem.fea.usp.br

80 Araújo Filho, F. W. C. , Travassos X. L. T., Figueiredo, P. S.

 Production: Through the integration with the ERP system and upon information contained in the Production Order, RFID tags are printed with the necessary information and the Finished Product lot number in question. RFID tags are linked to the Production Order, whose number is also associated with the handling of components, ensuring the association of Finished Goods and Components.  Appointment of Production: All finished products, properly accommodated in a standardized way on a pallet, will go through an RFID portal. At this point, the confirmation of the RFID reading of the finished product will change the status of components used from Debited to Reserved, generating the debt of the component in the inventory system, while a credit of finished product will be held in it. 5.

RESULTS

After the in-place test was performed, the FMEA form was used once again to evaluate the efficiency of the RFID system. The RPN results for the items that were changed are shown below (Table 3): Table 3 - FMEA Form

I T EM

RECOMMENDED ACTIONS

Perform real-time inventory (RFID tags in the Inventory Replace CBs tags for RFID tags Replace CBs tags for RFID tags

Identify the I / O 30 devices with RFID tags and group them Identify the I / O 31 devices with RFID tags and group them Replace CBs tags 34 for RFID tags Replace CBs tags 43 for RFID tags Table 3 – FMEA Form

RESPONSABILITY FOR THE RECOMMENDED ACTIONS & FOR THE DEADLINES

ACTION TAKEN

CIMATEC/XYX Company jan/11 CIMATEC jan/11 CIMATEC/XYX Company jan/11 CIMATEC/XYX Company jan/11 CIMATEC/ Empresa jan/11 CIMATEC jan/11 CIMATEC jan/11

Validation tests performed Validation tests Validation tests performed Validation tests performed Validation tests performed Validation tests Validation tests

S E V E R I D .

O C O R R .

D E T E C .

R P N

These measures enabled the company to attain the following results with the implementation of the methodology for integrating RFID technology in its production system: JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 65-84

www.jistem.fea.usp.br

Use of RFID technology to overcome inefficiencies in the production process: An analysis of a microcomputer company in Ilhéus – BA, Brazil

 Reduction of one worker in the inventory team due to the substitution of manual reading of barcodes for RFID technology. RFID technology was so efficient in relation to the previous technology (barcode) that the time required to read a pallet in the process of generating an invoice was reduced from ninety seconds to seven seconds on average. These time savings are equivalent to the work of one employee (Araujo Filho, Travassos and Figueiredo, 2013).  Reduction in the practice of substituting a component for another of higher grade/cost, due to the real time inventory control. This can be measured because the company knows the amount of parts that were substituted in the past. There were, however, difficulties and challenges in the implementation of the new technology. The different areas of production were arranged in a way that made difficult the installation of the RFID portal, the training of workers demanded a lot of attention and care, and the integration of the RFID system with the ERP system demanded a lot of time. 6.

CONCLUSIONS

In this paper, it was evidenced that the methodology for implementation of RFID technology on microcomputer assembly companies was viable. Araujo Filho, Travassos and Figueiredo (2013) determined an internal rate of return of 15%. It was also observed that the utilization of the FMEA tool for identification and elimination of causes of failure modes in the production process was efficient (Araujo Filho, Travassos and Figueiredo, 2013). The tool details the process systematically and adds more information to the process descriptions. It also identifies the causes of failures in the production process. The difference between the process before and after the FMEA tool being implemented is that the most important failure modes are taken care of, with preventive measures being taken in order to avoid the failures. Another important point were the technical evaluations performed for tag selection, choice of RFID equipment, analysis of the environment and for detecting electromagnetic noises present in the factory. Such evaluations helped to ensure the technical efficacy of the chosen technology, during in-site tests. A good RFID reading was obtained at all points where data was collected. The technology was so efficient compared with the previous technology (barcodes) that the time required to read a pallet in the process of generating an invoice was reduced from ninety seconds to seven seconds on average. It is important to point out that there are other significant gains inherent to the use of RFID technology, which are, however, difficult to measure (Araujo Filho, Travassos and Figueiredo, 2013). Such gains are shared by the entire commodity chain, because with real time inventory control the suppliers of this company will be able to monitor the flow of parts and ensure that their customer (the company) will have its needs fulfilled. This real time monitoring between supplier and company is replicated in the remaining links of the chain, which ensures better service quality to the end consumer, and reduction of logistical and operational costs. In such a condition, the component can be tracked from the first supplier to the end consumer (Fosso Wamba and Chatfield, 2009). There is a need to conduct further research on the RFID supply chain management applications in the microcomputer sector as this area holds a great potential JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 65-84

www.jistem.fea.usp.br

82 Araújo Filho, F. W. C. , Travassos X. L. T., Figueiredo, P. S.

for performance improvements. Additionally, there is a need to conduct more in-depth research into the isolated impact of RFID technology in comparison to the change in management and process redesign that it generates. One key limitation of this research is the case study approach based on a single case. This paper, however, provides direction for practitioners on how to assess RFID’s potential impact in the microcomputer assembly process, from the receiving of supplies to the delivery of final products. This research contributes to our understanding of RFID’s potential in intra-organizational supply chain management processes. Hopefully, the experience and lessons learned from this case study can be shared with the readers, and they will be beneficial to those organizations that are contemplating the implementation of RFID systems.

REFERENCES Araújo Filho, F.W., Travassos, X.L., Figueiredo, P.S. (2013): “Uma Análise De Viabilidade Econômica De Um Sistema RFID Na Cadeia De Suprimentos De Fabricação De Microcomputadores: Há Algum Lucro?” Gestão da Produção, Operações e Sistemas (Forthcoming) Attaran, M. (2007): “RFID: An Enabler of Supply Chain Operations”, Supply Chain Management-an International Journal, (12:4), pp. 249-257. Balocco, R., Miragliotta, G., Perego, A., and Tumino, A. (2011): “RFID adoption in the FMCG supply chain: an interpretative framework”. Supply Chain Management: an International Journal, 16(5), p.299-315. Bottani, E., R. Montanari and A. Volpi (2010): “The Impact of RFID and Epc Network on the Bullwhip Effect in the Italian Fmcg Supply Chain”, International Journal of Production Economics, (124:2), pp. 426-432. Chow, H.K.H., K.L. Choy, W.B. Lee and K.C. Lau (2006): “Design of a RFID CaseBased Resource Management System for Warehouse Operations”, Expert Systems with Applications, (30:4), pp. 561-576. Delen, D., B.C. Hardgrave and R. Sharda (2007): “RFID for Better Supply-Chain Management through Enhanced Information Visibility”, Production and Operations Management, (16:5), pp. 613-624. Dutta, A., H.L. Lee and S.J. Whang (2007): “RFID and Operations Management: Technology, Value, and Incentives”, Production and Operations Management, (16:5), pp. 646-655. Deavours, D.D.; Ramakrishnan, K. M.; Syed, A. (2005): “RFID Performance Tag Analysis”. Technical Report ITTC-FY2006-TR-40980-01, Available at: <http://ittc.ku.edu/publications/documents/deavours2006_ITTC-FY2006-TR-4098001.pdf> Fosso Wamba, S. and A. T. Chatfield (2009) "A contingency model for creating value from RFID supply chain network projects in logistics and manufacturing environments," European Journal of Information Systems, vol. 18, pp. 615-636.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 65-84

www.jistem.fea.usp.br

Use of RFID technology to overcome inefficiencies in the production process: An analysis of a microcomputer company in Ilhéus – BA, Brazil

Gaukler, G.M., R.W. Seifert and W.H. Hausman (2007): “Item-Level RFID in the Retail Supply Chain”, Production and Operations Management, (16:1), pp. 65-76. Gaukler, G.M., O. Ozer, W. Hausman (2004). RFID and product progress information: Improved Dynamic ordering policies. Working paper, Stanford University, Stanford, CA. Heinrich, C. (2005): RFID and Beyond. Wiley Publishing, Indianapolis. Hunt, V.D., Puglia, A. and Puglia, M. (2007): “RFID: A Guide to Radio Frequency Identification”. John Wiley & Sons, Hoboken, N.J. Jones, E.C., Riley,M.W., Franca, R., Reigle, S. (2007): “Case Study: The Engineering Economics of RFID in Specialized Manufacturing. The Engineering Economist. 52: 285-303. Kach, A. and Borzabad, A.F. (2011): “Use of RFID Technology to Overcome Inefficiencies in the Supply Chain: An Analysis of Renault's Operations in Iran”. International Journal of Management, 28: 365-378. Kharif, O. (2005): “For Many Retailers, RFID Lacks Roi,” Business Week. Pinheiro, J. M. S. (2006): “Identificação por Radiofreqüência: Aplicações e Vulnerabilidades da Tecnologia RFID”. Caderno UniFOA, Volta Redonda, ano 1, n. 2. Available at: <http://www.unifoa.edu.br/cadernos/edicao/02/18.pdf>. Rausand M., Hoylan A. (2004): “System Reliability Theory, Models, Statistical Methods, and Applications”. Wiley Series in probability and statistics - second edition page 88. Santos, F.R.S., Cabral, S. (2008): “FMEA and PMBOK Applied To Project Risk Management”, Journal of Information Systems and Technology Management, vol.5, No.2, pp. 347-364. Soares, R. S. (2008): “O impacto da tecnologia de etiqueta inteligente (RFID) na performance de cadeias de suprimentos – Um estudo no Brasil”. Revista Jovens Pesquisadores, Ano V, No.9, Available at: <http://www.mackenzie.br/dhtm/seer/index.php/jovenspesquisadores/article/viewFile/8 20/344>. Stamatis, D.H. (2003) Failure Mode and Effect Analysis: FMEA from Theory to Execution. 2nd Ed. ASQ Quality Press, Milwaukee, 2003. Stambaugh, C.T. and F.W. Carpenter (2009): “Wireless innovation in inventory monitoring and accounting”. Strategic Finance Vol. 91(6), pp. 35-40. Sweeney II, P. J. (2005): RFID for Dummies. Ed. Wiley Publishing Inc., Indianapolis, Indiana, 409 p. Taghaboni-Dutta, F. and B. Velthouse (2006): “RFID Technology is Revolutionary: Who Should Be Involved in This Game of Tag?”, Academy of Management Perspectives, (20:4), pp. 65-78. Ustundag, A. and M. Tanyas (2009): “The Impacts of Radio Frequency Identification (RFID) Technology on Supply Chain Costs”, Transportation Research Part E-Logistics and Transportation Review, (45:1), pp. 29-38. Whang, S. (2010): Timing of RFID Adoption in a Supply Chain. Management Science, vol. 56, No. 2, pp. 343-355.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 65-84

www.jistem.fea.usp.br

84 Araújo Filho, F. W. C. , Travassos X. L. T., Figueiredo, P. S.

Wu, N.C., M.A. Nystrom, T.R. Lin and H.C. Yu (2006). “Challenges to Global RFID Adoption”, Technovation, (26:12) pp. 1317-1323.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 65-84

www.jistem.fea.usp.br

JISTEM - Journal of Information Systems and Technology Management Revista de Gestão da Tecnologia e Sistemas de Informação Vol. 11, No. 1, Jan/Apr., 2014 pp. 85-104 ISSN online: 1807-1775 DOI: 10.1590/S1807-17752014000100006

MEDICIÓN DE USABILIDAD DE TRÁMITES PÚBLICOS EN LÍNEA EN CHILE: UN CASO DE ESTUDIO EN GOBIERNO ELECTRÓNICO USABILITY MEASURE OF CHILEAN PUBLIC ONLINE SERVICES: AN E-GOVERNMENT CASE STUDY Marcelo Garrido Palma AMABLE - Laboratorio UX, Santiago, Chile Claudio Lavín Tapia CEAR, Facultad de Psicología, Universidad Diego Portales, Santiago, Chile Nelson Rodríguez-Peña AMABLE – E-gob., Santiago, Chile

____________________________________________________________ ABSTRACT The possibility of conducting public online transactions means an important help to users and a reduction of governmental resources. However, the usage of the digital channel can get obstructed by different variables where the usability of the service plays an important role. This research presents a usability evaluation of 60 online services offered by the Chilean government. Following a heuristic evaluation model, we conducted a diagnosis and propose lines of improvement to the problems detected. The evaluation showed that the online services do not offer enough control to users, the services cause uncertainty while interacting with them, and there is not enough guidance to users during relevant steps of the interaction. The evaluation shows that usability is an important variable when explaining the difficulties of public online _____________________________________________________________________________________________ Manuscript first received/Recebido em: 18/02/2013 Manuscript accepted/Aprovado em: 17/01/2014 Address for correspondence / Endereço para correspondência Marcelo Garrido Palma, Psicólogo Licenciado en Psicología por la Universidad de Concepción (Chile), Doctor en Ciencias Cogntivas por la Universidad Complutense de Madrid (España). Socio-Director de la consultora de investigación y diseño de estrategias digitales AMABLE: www.amable.com. Director y editor de la revista de experiencia de usuario Faz: www.revistafaz.org. Investigador afiliado al Centro de Investigación de la Web, de la Universidad de Chile. Dirección: Santo Domingo 550, Santiago, Chile. Teléfono: +56 2 26382679 Mail: mgarrido@amable.com Claudio Lavín Tapia, Centro de Estudios de Argumentación y Razonamiento, Universidad Diego Portales, Santiago, Chile. Psicólogo Licenciado en Psicología por la Universidad de Valparaíso (Chile), Master en Ciencias Sociales por la Universidad de Queensland (Australia). Docente e investigador afiliado a la Universidad Diego Portales. Investigador colaborador de la consultora de investigación y diseño de estrategias digitales Amable.Dirección: Vergara 275, Santiago, Chile Teléfono: +56 2 26762578 Mail: claudio.lavin@mail.udp.cl Nelson Rodríguez-Peña, Licenciado en Artes Plásticas por la Universidad de Chile. Director del Área de Gobierno Electrónico de la consultora de investigación y diseño de estrategias digitales Amable. Dirección: Santo Domingo 550, Santiago, Chile. Teléfono: +56 2 26382679 Mail: nrodriguezp@amable.info Published by/ Publicado por: TECSI FEA USP – 2014 All rights reserved.

86 Palma, M. F., Lavin, C., Rodriguez-Peña, N.

services spreading between users and it suggests that interventions in this sense could make more efficient the digital channel within the e-government context. Keywords: E-government, Usability, Online Services, Digital Transactions, Heuristic Evaluation

RESUMEN La posibilidad de realizar cada vez más trámites públicos en línea (TPL) supone una vital ayuda a los usuarios y una disminución en los recursos que el gobierno debe invertir en acercarse a éstos. No obstante, el uso de herramientas digitales por parte de los usuarios parece dificultado por una serie de variables donde la usabilidad de los sistemas juega un rol preponderante. Esta investigación presenta una evaluación de la usabilidad de 60 TPLs en Chile a partir de la metodología de evaluación heurística, y propone mejoras a partir de un diagnóstico objetivo. Esta evaluación muestra 3 problemas principales en el servicio de TPLs en Chile: (1) los usuarios no sienten control en las acciones que realizan, (2) el sitio genera incertidumbre, (3) no se guía a los usuarios al realizar tareas relevantes. Ésto demuestra que la usabilidad es una variable importante al momento de explicar los problemas de masificación de los servicios en línea de gobierno electrónico, y que intervenciones en este sentido pueden hacerlos más eficientes tanto en Chile como otros países hispanoparlantes. Palabras Clave: Gobierno Electrónico, Usabilidad, Trámites en Línea, Transacciones Digitales, Evaluación heurística

1. INTRODUCCIÓN Análisis recientes han evidenciado una evolución del gobierno electrónico (o egobierno) desde un enfoque de exposición de información o propaganda, hacia uno más transaccional e interactivo (Moon, 2002). En este sentido se plantea que los servicios ofrecidos por el gobierno a los ciudadanos a través de plataformas virtuales han evolucionado desde ser una iniciativa autoimpuesta por las autoridades de gobierno, a responder progresivamente a requerimientos cada vez más específicos de parte de los ciudadanos, y de la necesidad de hacer más eficiente la inversión de recursos del estado (Clancy, 2012; Gil-Garcia & Martinez-Moyano, 2007). Es justamente en este contexto donde los trámites públicos en línea (TPL) han adquirido creciente relevancia como un servicio que facilita el acceso de los usuarios a la institucionalidad gubernamental, y como un método mediante el cual se puede aumentar eficiencia de recursos tanto humanos como económicos. Los TPLs son definidos como un servicio ofrecido por un sitio web de gobierno a los ciudadanos, con el objeto de que éstos generen un acto administrativo (Fang, 2002). Los TPLs son una acción, o secuencia de interacciones que buscan concretar objetivos precisos como obtener información, entregar información o realizar una transacción. Estos trámites dentro del enfoque de e-gobierno tienen como objetivo la simplificación de procedimientos administrativos y a la vez proveer de un mejor servicio a los ciudadanos, funcionarios públicos y empresas privadas (Moon, 2002). Esto provee de JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

Medición de usabilidad de trámites públicos en línea en Chile: un caso de estudio en gobierno electrónico

beneficios no sólo a los usuarios sino que reduce los costos del gobierno central en el manejo de tales servicios. De esta manera los beneficios para el gobierno asociados a los TPLs son la reducción de costos operacionales por servicio, incremento en la cantidad y calidad de información, equidad de acceso, reducción de costos materiales y humanos, reducción de costo de oportunidad de trabajadores, entre otros (Norris & Moon, 2005). Al analizar los beneficios que los TPLs suponen tanto para los ciudadanos (en términos de la facilidad de acceso) como para el gobierno (en términos de la reducción de recursos), parecería difícil suponer problemas mayores en la implementación de estos servicios en línea. No obstante, el paso desde un enfoque expositivo a uno transaccional por parte de los sitios de gobierno, y expresado a través de los TPLs, ha sido bastante más lento de lo esperable debido a una serie de complejidades (Norris & Moon, 2005). Los principales problemas en este sentido tienen relación con los lentos cambios organizacionales que supone el traspasar servicios tradicionalmente presenciales a canales digitales, las dificultades en el uso técnico adecuado de las plataformas digitales, la desconfianza que este canal significa para algunos usuarios y los problemas de usabilidad que los sitios pueden presentar, entre otras razones (Balci, Dalci, Karaman, Kutluoglu, Yucel, Medeni, 2010; Hung, Chang, Yu, 2006; Peng, Ramaiah & Foo, 2004). Los TPLs no se usan en las cantidades esperadas, y su masificación es todavía insuficiente (Clancy, 2012). Esto grafica el hecho de que no basta con teóricamente tener claros los costos y beneficios que suponen un giro transaccional, sino que hay que considerar igualmente aspectos tanto idiosincráticos como técnicos que permitan explotar al máximo las posibilidades que el formato digital ofrece para todas las partes involucradas. El foco de esta investigación está en estudiar la usabilidad de los TPLs en el contexto chileno en tanto es una de las importantes fuentes de problemas en el uso y masificación de los TPLs (Balci et al., 2010). Usabilidad en la web En términos generales el concepto de usabilidad se utiliza para referirse a la facilidad de uso de cualquier sistema o herramienta con la que interactúa una persona. Dentro del ámbito de la interacción humano computador (o HCI, por el inglés human computer interaction), la usabilidad se ha entendido como la cualidad de las interfaces que determina la facilidad con que los usuarios pueden operar en ella. Como una muestra de la relevancia de este concepto, la usabilidad se ha hecho parte del conjunto de normas sobre calidad y gestión establecidas por la Organización Internacional de Normalización (ISO), la cual la ha definido como “el grado con que un producto puede ser usado por usuarios específicos para alcanzar metas con efectividad, eficiencia y satisfacción en un contexto de uso específico” (ISO 9241). A su vez la norma ISO/IEC define usabilidad como “la capacidad de un software de ser comprendido, aprendido, usado y ser atractivo para el usuario, en condiciones específicas de uso” (ISO/IEC 9126) (Citas tomadas de Balci, 2010, p. 20). Tomando en consideración la definición ISO y lo propuesto por diversas investigaciones en esta línea, se puede observar que la usabilidad es una propiedad no sólo deseable, sino también medible, cuantificable y con JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

88 Palma, M. F., Lavin, C., Rodriguez-Peña, N.

claras repercusiones en el devenir económico de un proyecto (Flavián, Guinalíu, & Gurrea, 2006; Hornbæk, 2006; Nakamichi, Shima, Sakai, & Matsumoto, 2006). Una interfaz será más fácil de usar en la medida en que permitirá a su usuario efectuar sus tareas en menor tiempo, disminuyendo los errores y generando mayor satisfacción. La aplicación del concepto de usabilidad al estudio de HCI ha tenido importantes repercusiones a nivel metodológico, y a partir de esta idea se han ido desarrollando pautas para generar diagnósticos adecuados y mejoras derivadas para un mejor aprovechamiento de los recursos digitales. Esto es de suma importancia para el desarrollo de sitios de gobierno usables, y de TPLs accesibles para el desarrollo de una orientación transaccional en gobierno electrónico. De esta manera este artículo se basa en la acumulación de evidencia que sugiere los beneficios de desarrollar metodologías para la medición de la usabilidad y accesibilidad. Esto al servicio de la incorporación de interfaces orientadas a los usuarios en proceso de diseño de interacción (Krug, 2000). Medición de usabilidad: la evaluación heurística Como se mencionó anteriormente, esta definición de usabilidad provee directrices con una clara repercusión metodológica para el diagnóstico de TPLs y sitios web en general. Si bien las evaluaciones tradicionales de usabilidad se basan en pruebas directamente con usuarios, se han implementado otros sistemas de medición basados en el conocimiento acumulado por la ergonomía que permite a expertos la calificación e identificación de problemas puntuales en sistemas interactivos, y un cuerpo de conocimiento acumulado en cuanto a directrices que guíen el desarrollo de interfaces (Nielsen y Mack, 1994). En este contexto se enmarca la evaluación heurística. Jacob Nielsen desarrolló una orientación de medición de usabilidad basada en heurísticos para el diseño de entornos informáticos (Nielsen y Mack, 1994). Estos heurísticos son definidos como soluciones aproximadas (contrarios a la noción de algoritmo) que se utilizan como herramientas de evaluación que requieren el contraste entre expertos. La evaluación que este estudio reporta se basa en la estructura de los heurísticos propuestos por Nielsen y Mack (1994), además del desarrollo de nuevas pautas tendientes a evaluar la usabilidad de los TPLs chilenos. Antes de detallar la metodología de este estudio, se hará referencia a otros estudios de usabilidad en gobierno electrónico y a los esfuerzos en este sentido por generar pautas de estandarización de transacciones en línea en este contexto. Evaluaciones de usabilidad en contextos de gobierno electrónico Aún cuando el estudio de la usabilidad de los sistemas de HCI se encuentra en pleno auge, existen ámbitos de estudio menos explorados en términos del impacto que involucraría la utilización de medidas de usabilidad para mejoras medibles en eficiencia y accesibilidad. Este es el caso de los aspectos transaccionales del gobierno electrónico donde Latinoamérica es un ejemplo concreto de esta falta de estudio (Clancy, 2012). Este es un aspecto de suma importancia ya que, como se ha planteado, la ineficiencia en la entrega de servicios en línea por parte del gobierno repercute directamente en la JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

Medición de usabilidad de trámites públicos en línea en Chile: un caso de estudio en gobierno electrónico

confianza de las personas, de la institución y genera exclusión progresiva de usuarios (Peng et al., 2004). De esta manera, a pesar de la implementación de servicios en línea como los TPLs, su masificación sigue siendo un problema, donde la confianza y los problemas generados por problemas de usabilidad pueden repercutir negativamente en los objetivos propuestos y la imagen global del gobierno en los ciudadanos (Peng et al., 2004). Dentro de los esfuerzos realizados en el ámbito de la evaluación de la usabilidad de sitios de gobierno y TPLs se encuentra el sitio usability.gov que se define como la principal fuente gubernamental para información sobre usabilidad y diseño centrado en el usuario de Estados Unidos. Asimismo, la publicación Research-Based Web Design and Usability Guidelines se ha reportado como una de las fuentes más exhaustivas y documentadas sobre criterios de usabilidad para e-gobierno (Leavitt y Shneiderman, 2006). Esta guía reporta más de 200 pautas de diseño, accesibilidad y usabilidad basado en la calificación de 16 especialistas y la evidencia basada en investigaciones publicadas en la materia. En Nueva Zelandia se han desarrollado estándares agrupados vale el e-Government Interoperability Framework (Gottschalk, 2009). Este documento tiene un carácter normativo y debe ser cumplido por todos los sitios web del sector público. Esta guía se propuso con el objeto de que los equipos a cargo del ofrecimiento de servicios en línea pudieran evaluar y corregir los problemas encontrados en sus sitios. En Australia también existen antecedentes de evaluaciones de servicios en línea con énfasis en la medición de efectividad y eficiencia, dos de los factores clave para la medición de usabilidad según la definición ISO (Gottschalk, 2009). Las mediciones llevadas a cabo a comienzos de la década del 2000 pusieron énfasis en la necesidad de mejorar los aspectos transaccionales de los sitios de gobierno de ese país. En Latinoamérica los ejemplos son escasos, aún cuando hay antecedentes de estudios de diagnóstico de usabilidad en sitios de gobierno en México. En este país se realizó un estudio de usabilidad sobre lo que ambiguamente se definió como Portales, aunque tuvo las características de una evaluación con pautas de usabilidad sobre aspectos generales del sitio y no aspectos transaccionales del mismo (Diagnóstico de Usabilidad y Calidad de la Información de los Portales de Internet de los Entes Públicos del Distrito Federal, 2007). En Argentina se realizó una Guía de Trámites como un sitio con el objeto de orientar a los ciudadanos en los trámites que realizaban ante la Administración Pública Nacional de ese país. Luego de una evaluación exhaustiva del sitio este se rediseñó el año 2011 siguiendo las pautas de la misma guía con el objeto de hacerlo más accesible (Clancy, 2012). En el contexto chileno, la preocupación por la usabilidad de los sitios de gobierno electrónico ha tenido algunos hitos que se tradujeron en dos instructivos presidenciales, la creación de desarrollo de sitios web, y una evaluación preliminar de TPLs en el año 2005. En Latinoamérica el panorama no es muy distinto. Los estudios son más bien locales y se centran en caracterizaciones de sitios más que en una evaluación de las propiedades de usabilidad de estos (Clancy, 2012). No obstante, no existe un estudio y seguimiento sistemático del estado de la usabilidad de los servicios transaccionales en línea a partir de los cuales tomar medidas para su masificación y explotación de los beneficios derivados de su implementación. De esta forma, y aún cuando existe un claro interés en modernizar los servicios ofrecidos por el gobierno a través de canales JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

90 Palma, M. F., Lavin, C., Rodriguez-Peña, N.

digitales (expresados por ejemplo en el Instructivo Presidencial de Gobierno Electrónico, 2001; como en el Instructivo Presidencial de Participación Ciudadana, 2008), existe una ausencia de datos concretos que informen respecto del estado específico de los TPLs en materia de usabilidad. Dados los beneficios que los TPLs suponen en términos de costos, eficiencia y administración de información tanto para oferentes (gobierno) como para usuarios (ciudadanos) (Moon, 2002), esta investigación tiene por objeto elaborar un diagnóstico de usabilidad de una muestra representativa de los TPLs en Chile a partir de una evaluación heurística. Además de esto, se discutirán sugerencias específicas de mejoras para los principales problemas evaluados y las posibles aplicaciones y beneficios de este tipo de evaluación dentro del contexto Latinoamericano. Siendo la usabilidad una propiedad esencial para el uso adecuado de un servicio electrónico parece de primera relevancia, no sólo suponer que problemas en esta línea pueden ser una importante barrera en la masificación de las transacciones públicas en línea, sino además saber en específico cual es el estado de los TPLs en términos de usabilidad, detectar problemas principales y elaborar propuestas de mejora. En las siguientes secciones entonces se detallará la metodología usada para la realización del diagnóstico de usabilidad, se detallarán los resultados obtenidos y estos serán discutidos a la luz de sugerencias de mejoras y de las posibilidades que este tipo de evaluaciones ofrece para otros contextos en Latinoamérica. 2. METODOLOGÍA La evaluación de los TPLs se realizó entre los años 2007 y 2009 sobre la base de una evaluación heurística de las tareas involucradas en la realización de los trámites. Este tipo de evaluación es una técnica de inspección de usabilidad que permite identificar la facilidad de uso de un sistema, de una aplicación o de un sitio web a partir de la conformidad con ciertos principios (Molich & Nielsen, 1990). La evaluación heurística se realiza siguiendo una serie de etapas, donde la primera consiste en la revisión individual del sistema por distintos evaluadores, en un segundo momento los evaluadores se reúnen y comentan sus hallazgos, y finalmente se genera un informe común (Nielsen, 1992). Siguiendo los lineamientos mencionados, la presente evaluación fue llevada a cabo por 2 equipos de 3 evaluadores cada uno, todos expertos en usabilidad y diseño web. La evaluación constó de las siguientes 3 etapas que pasarán a ser descritas en detalle en las siguientes secciones: (1) el muestreo de los trámites a evaluar por parte de los expertos, (2) la definición de las pautas de evaluación heurística, y (3) el procedimiento de aplicación de las pautas de evaluación a los trámites muestreados. Determinación de la muestra de TPLs a evaluar Los 60 trámites evaluados en este estudio fueron seleccionados utilizando una serie de criterios que permitieran contar con una muestra de los trámites más utilizados y más masivos en Chile. La fuente principal de información sobre trámites ha sido el listado de www.tramitefacil.gob.cl (alojado actualmente en el sitio JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

Medición de usabilidad de trámites públicos en línea en Chile: un caso de estudio en gobierno electrónico

www.chileatiende.cl) que cuenta con un listado superior a 400 trámites públicos en línea. En base a ellos se generó un listado inicial al que se aplicó una serie de criterios para seleccionar los 60 que serían evaluados. La selección de los TPLs a evaluar se realizó calificando a los trámites siguiendo tres criterios: Impacto, oportunidad y problemas operacionales. La noción de impacto se refiere a la cantidad de usuarios que realizan un trámite. Este factor es relevante por cuanto los problemas de usabilidad de un trámite realizado por muchos usuarios tienen una repercusión claramente mayor en el funcionamiento global del servicio. El concepto de oportunidad se refiere al factor de temporalidad de un trámite en términos de sus plazos y vigencia. La oportunidad afecta el alcance que tenga un trámite en un momento determinado y puede introducir problemas operacionales para la realización del trámite. La noción de problemas operacionales se refiere a posibles problemas de tipo logístico u operativo para la realización de un trámite. Este factor representa el nivel de complejidad técnica que pudiera dificultar la evaluación, midiendo la dificultad de generar las condiciones para evaluar. A partir de estos puntajes, se dejaron dentro los trámites que presentaran menores puntajes. Luego de esto se realizó una selección final de los 60 TPLs por parte de Ministerio Secretaría General de la Presidencia del gobierno de Chile (Segpres) considerando los criterios entregados en la evaluación más criterios estratégicos específicos del ministerio. El listado final se reporta en la tabla de resultados en la siguiente sección del artículo. Definición de las pautas de evaluación heurística La pauta de evaluación heurística tomó la forma de una escala tipo Likert (Jamieson, 2004) dado que ofrece una posibilidad de evaluación en un continuo de posibles valores más que una mera clasificación dicotómica, tradicionalmente usadas en este tipo de evaluación. También en el ánimo de superar las falencias de la evaluación heurística tradicional, se asignaron ponderaciones a cada una de las pautas o directrices cuyo grado de cumplimiento fue rastreado en los trámites. Dichas ponderaciones resultaron del cruce de los valores otorgados por expertos en usabilidad y por encargados de trámites públicos en línea, a la hora en que se les consultó por la relevancia de cada pauta para asegurar la usabilidad de un servicio de esta índole. Lo que se mantiene intacto respecto de una evaluación heurística tradicional es el procedimiento de aplicación, y el procedimiento de contraste de resultados entre evaluadores (ver próxima sección). El instrumento final corresponde a una escala Likert, en la cual 67 pautas de usabilidad actúan como reactivos. La evaluación consiste en señalar, en una escala estandarizada, el grado y/o frecuencia del cumplimiento de cada pauta en el contexto de un trámite en línea determinado. En concreto cada evaluador puede asignar las siguientes respuestas asociadas al cumplimiento de cada directriz de usabilidad: 0: La pauta no aplica 1: Nunca / Nada 2: Casi Nunca / Casi Nada JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

92 Palma, M. F., Lavin, C., Rodriguez-Peña, N.

3: Regularmente / Parcialmente 4: Casi Siempre / Casi Totalmente 5: Siempre / Totalmente La extracción de puntajes de las pautas de evaluación se estructuró de forma tal de evitar que las diferencias entre los evaluadores generaran tendencias diferentes en la utilización de estándares a la hora de juzgar el grado de cumplimiento de las pautas de usabilidad. Esto ya que los resultados finales tendrían tendencias disimiles, por ejemplo respecto al promedio de usabilidad de los trámites que cada grupo evalúa. Entonces para para subsanar este potencial inconveniente se recurrió a la estandarización de resultados apelando a puntajes Z que permiten la comparación entre grupos y la generalización de puntajes. Para obtener los puntajes Z se resta a cada puntaje bruto la media de los puntajes entregados por su grupo de valuadores respectivo, y este resultado se divide por la desviación estándar. Los puntajes de usabilidad estandarizados y finales, se obtienen de la estimación porcentual hecha a partir de los puntajes Z de cada trámite, respecto del máximo Z observado. Procedimiento de aplicación de la medición de usabilidad Una vez diseñado el instrumento se procedió a su aplicación en la muestra de 60 trámites seleccionados. La aplicación del instrumento fue realizada por dos equipos de tres evaluadores expertos cada uno, trabajando en paralelo. Cada equipo estaba liderado por un evaluador experimentado en usabilidad para gobierno electrónico. A cada equipo de evaluadores se le asignaron 30 trámites de manera aleatoria. Una vez asignados los trámites, los equipos utilizaron un procedimiento estándar para evaluar: 1. Asignación: se asigna a cada evaluador un trámite, con los detalles de evaluación necesarios: datos de acceso, procedimiento de evaluación, etc. 2. Evaluación individual privada: cada evaluador aplica el instrumento individualmente de modo aislado, sin intervención de los otros miembros del equipo. Se realizan comentarios particulares sobre las pautas cuando es pertinente y comentarios generales sobre el trámite. 3. Consolidación de resultados: se reúnen los tres miembros del equipo y contrastan sus resultados individuales, discuten los resultados y consolidan la evaluación como equipo. 4. Preparación del informe consolidado: una vez realizados los pasos anteriores, el jefe de equipo realiza un informe consolidado del trámite, que recoge las opiniones del equipo e incorpora los comentarios e imágenes ilustrativas. En suma, cada trámite es evaluado 3 veces, lo que en total significa que se han realizado 180 evaluaciones (3 evaluaciones por cada uno de los 60 trámites). Por el volumen de evaluaciones, todo el proceso se realiza utilizando una aplicación web desarrollada especialmente para esto, que permite controlar los tramites asignados, aplicar el instrumento, consolidar y generar los informes para cada trámite.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

Medición de usabilidad de trámites públicos en línea en Chile: un caso de estudio en gobierno electrónico

3. RESULTADOS Los resultados generales informan que los TPLs tienen problemas de usabilidad. En conjunto, los 60 TPLs evaluados presentaron un cumplimiento promedio de 63,78%, lo que equivale a un promedio de 4,3 en una escala de 1 a 7. Esto implica que la usabilidad global de los TPLs es apenas regular, siendo poco eficientes y eficaces para el cumplimiento de las tareas que los usuarios requieren realizar en ellos. De los 60 TPLs evaluados, 18 (30% del total evaluado) obtienen una calificación inferior a 4, lo que quiere decir que su usabilidad es insuficiente (mala o muy mala), 12 trámites (20% del total) obtienen una calificación igual o superior a 5, es decir, su usabilidad es buena, y la mitad de los trámites evaluados obtiene una calificación entre 4 y 4.9 indicando un nivel de usabilidad regular. Ningún trámite evaluado obtiene una calificación igual o superior a 6, es decir, ninguno tiene un nivel de usabilidad muy bueno. A continuación se expone el ranking resultante de la evaluación de los TPLs con información de la institución de dependencia, nombre del trámite e índice de usabilidad. Posición Institución 1. 2. 3. 4.

6. 7. 8.

10.

Autoridad Sanitaria Regional Autoridad Sanitaria Regional Correos de Chile Dirección de Compras y Contratación Pública Dirección del Trabajo Servicio de Registro Civil e Identificación (SRCeI) SBIF Instituto de Normalización Previsional Superintendencia Administradoras Fondos de Pensiones (SAFP) Servicio de Registro Civil e Identificación (SRCeI)

Trámite

Usabilidad (%)

Reporte Sanitario

84,03

Solicitud de Autorización Sanitaria para Establecimientos de Alimentos Obtención de Código Postal

81,80 80,13

Inscripción en Registro de Proveedores del Estado (ChileCompra)

77,33

Carta de aviso de Despido

77,15

Obtención Certificado de Nacimiento

76,49

Simulador de Créditos

75,17

Certificado Previsional del empleador

74,49

Consulta de Afiliación a AFP

73,99

Certificado de Anotaciones Vigentes de Vehículos Motorizados: Obtención

72,95

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

94 Palma, M. F., Lavin, C., Rodriguez-Peña, N.

11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24.

25.

26.

27.

28.

Servicio de Impuestos Internos Servicio de Impuestos Internos Servicio de Impuestos Internos Ministerio de Educación Instituto de Normalización Previsional Instituto Nacional de Estadísticas Instituto Nacional de Estadísticas Empresa de los Ferrocarriles del Estado Servicio de Impuestos Internos Ministerio de Educación Ministerio de Vivienda y Urbanismo Servicio de Impuestos Internos Servicio Electoral Subsecretaria Transporte Servicio de Registro Civil e Identificación (SRCeI) Servicio Nacional De Capacitación y Empleo Proyecto de Reforma y Modernización del Estado Subsecretaría de Transportes

Certificado de avalúo fiscal: solicitud

72,84

Iniciación de Actividades Persona Natural

72,76

Consultar antecedentes de un bien raíz 72,36 Consulta de Rol Nacional de Postulantes

72,21

Declaración y Pago de Cotizaciones

72,19

Encuesta a Sectores Productivos

70,67

Solicitud de información estadística

69,99

Pasajes Ferrocarriles del Estado: Obtención

69,69

Emisión de boleta de honorarios electrónica Postulación Fondo Solidario de Crédito Universitario Inscripción en el Registro Nacional de Contratistas

69,01 68,99 67,13

Anulación de Boletas de Prestación de 66,84 Servicios de Terceros Electrónicas Inscripción electoral: obtención de 66,81 datos Sugerencias, consultas y reclamos 66,73 sobre Transantiago Bloqueo gratuito de Cédula de Identidad

66,49

Ingreso de Ofertas de Trabajo (INFOEMPLEO)

66,08

Buscador de Trámite Fácil

65,72

Denuncias en el área de Transporte por calidad de Servicio, por Contaminación Vehicular o por Plantas de Revisión Técnica

65,39

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

Medición de usabilidad de trámites públicos en línea en Chile: un caso de estudio en gobierno electrónico

29. 30. 31. 32. 33.

SBIF Servicio Nacional del Consumidor Registro de Multas del Tránsito Instituto De Normalización Previsional Presidencia de la República

34.

FONASA

35.

SENAME Servicio de Impuestos Internos Servicio Nacional De Capacitación y Empleo Ministerio de Vivienda y Urbanismo Servicio de Impuestos Internos Servicio de Impuestos Internos Servicio de Impuestos Internos Subsecretaria Transporte Dirección del Trabajo Fonasa Servicio de Impuestos Internos Instituto de Normalización Previsional (INP)

36. 37.

38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 48. 49. 50.

Fonasa Dirección del Trabajo Sernac Servicio Nacional De Capacitación Y Empleo

Obtención Informe de deudas del Sistema Financiero

65,12

Ingreso de Reclamo

65,00

Certificado de Anotaciones de Multas del Tránsito No Pagadas

63,85

Certificado de Imposiciones

63,46

Escríbele a la Presidenta

63,26

Certificado de afiliación a FONASA: Obtención Denuncias de Maltrato Declaración o rectificatoria de IVA, PPM y Retenciones Organismos Técnicos de Capacitación (OTEC): Obtención de Formulario de Adhesión a Internet

63,15 62,43 62,27 62,08

Registro Nacional de Capacitación y Empleo

61,94

Registro y cambio de clave

61,79

Declaración de Renta

60,22

Consultar Rol asociado a su Rut

59,92

Transantiago de donde para donde Planificación de recorrido

59,64

Propuesta de Finiquito

58,85

Cartola Prestadores Inicio de actividades de segunda categoría

58,13

Certificado de Salud Fonasa

56,23

Búsqueda de Consultorios y Hospitales Constancias laborales para empleadores Seguimiento de Estado de Reclamo Ingreso de Currículum en proyecto Infoempleo

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

56,47

55,84 55,00 54,88 54,46

96 Palma, M. F., Lavin, C., Rodriguez-Peña, N.

51. 52.

Servicio Nacional De Capacitación y Empleo Serviu Valparaíso

Capacitación en las Empresas a través de Franquicia Tributaria

53,73

Certificados de No Expropiación Iniciación de Actividades o Persona Jurídica y otro tipo de Organizaciones: Declaración

53,66

53.

Servicio de Impuestos Internos

54.

Servicio de Registro Civil e Identificación (SRCeI)

Bloqueo Licencia de conducir

52,91

55.

JUNAEB

Beca pago de la inscripción de la Prueba de Selección Universitaria (PSU)

52,83

Consultas y estado de reclamos

51,60

56. 57. 58. 59. 60.

Superintendencia de Salud Superintendencia de Servicios Sanitarios Tesorería General de la República Tesorería General de la República Tesorería General de la República

Reclamos a la Superintendencia de Servicios Sanitarios Certificado de Deuda de Contribuciones de Bienes Raíces: Obtención

52,97

50,72 42,20

Certificado de deuda Fiscal: Solicitud

40,40

Pago de Contribuciones

36,20

Principales problemas de usabilidad detectados El panorama antes expuesto deja claro el diagnóstico global de los TPL en relación con su nivel de usabilidad. A continuación se ahondará en cuales son los asuntos que más inciden en la mala evaluación obtenida, para lo cual es necesario identificar los ámbitos de la usabilidad peor resueltos. Las pautas peor evaluadas en promedio en todos los trámites tienen relación directa con una mala solución para enfrentar tres aspectos críticos, que podemos identificar como los grandes problemas de usabilidad que se reiteran en los TPL y que deben ser mejorados. Existen problemas en: (1) entregar el control de las acciones a los usuarios, (2) hacerse cargo de las incertidumbres de los usuarios derivadas del cambio de lo presencial a lo virtual, y (3) guiar al usuario con la necesaria formalidad y cercanía que requiere la relación entre los ciudadanos y las instituciones públicas. Estos tres grandes temas tienen relación con una mala aplicación de siete principios identificados como necesarios para asegurar la eficacia y eficiencia de los sistemas y su orientación a la satisfacción de las necesidades de los usuarios a través de la pauta de evaluación generada. Estos temas son: El usuario no tiene –o siente que no tiene- el control de las acciones, porque los TPL en general: JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

Medición de usabilidad de trámites públicos en línea en Chile: un caso de estudio en gobierno electrónico

 No son auto-descriptivos; requieren de referencias externas que generalmente no son puestas al alcance del usuario.  No entregan al usuario las herramientas necesarias para superar los errores cometidos, por responsabilidad propia o de las aplicaciones.  No entregan el control al usuario, sino que lo obligan a adaptarse a jergas, códigos y formas de operar propias de la lógica interna de las organizaciones, pero que resultan crípticas para el ciudadano, externo a ellas. El usuario se mueve en constante incertidumbre, porque:  Los TPL en general no están adaptados a la tarea específica que el usuario sabe que debe realizar para satisfacer su necesidad; a veces el trámite es más complejo de lo que debiera o es tan poco asociado con la tarea, que resulta inútil su realización.  La interacción real con el trámite es diferente a la expectativa del usuario, por lo tanto este no sabe en que momento finaliza un proceso, si tiene que ejecutar acciones posteriores al trámite en línea, si intervienen otras personas, etc. El usuario no se siente asistido, porque:  No se le facilita el aprendizaje ni se le da la posibilidad de resolver dudas o interrogantes propias de la materia de los trámites en el momento en el que está operando.  Los TPL en general no cuentan con niveles de individualización de los usuarios, de modo que estos puedan sentirse tratados individualmente, tal como ocurre presencialmente en una ventanilla de atención. Estos problemas tienen repercusiones claras en el uso práctico que los usuarios le dan a los TPLs y que está previniendo a este servicio de servir a los propósitos para los que fue pensado. A continuación pasaremos a describir en detalle las razones prácticas identificadas detrás de los tres grandes problemas de usabilidad identificados. Problemas con el control de las acciones En términos del primer foco de problema, la falta de control de las acciones por parte de los usuarios repercute directamente en que la interacción de los usuarios con el sitio de TPLs se torna frustrante, poco efectiva y eficiente. En la mayoría de los trámites los textos de instrucciones y ayuda no utilizan un lenguaje cercano para el usuario y priman las jergas técnicas y legalistas. Esto ocurre aún en aquellos trámites orientados a cualquier ciudadano que no requiere ningún conocimiento particular sobre términos legales al relacionarse con la entidad. Es frecuente leer extractos de leyes para ejemplificar escenarios. Si el usuario no entiende, recurrirá a otro medio de comunicación paralelo para finalizar el trámite, o simplemente, ante la duda se abstendrá y no realizará el TPL. Algo similar ocurre con los mensajes de error que, si bien en la mayoría de los casos identifican los problemas, JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

98 Palma, M. F., Lavin, C., Rodriguez-Peña, N.

en general no entregan alternativas de solución. El usuario sabe que algo está mal, pero esa información no le resulta efectiva. Errores sin solución en asuntos delicados, como datos personales o transacciones de dinero, sin duda generaran desconfianza y el TPL difícilmente será completado. Otro foco de problema es el uso de etiquetas. Se utilizan etiquetas poco claras o inconsistentes para presentar servicios, lo cual genera que el usuario difícilmente anticipe que ocurrirá si hace clic en determinados vínculos. Esto genera pérdida de tiempo, ineficiencia y frustración, dado que el usuario seguramente sentirá que no domina el tema del trámite; lo asumirá como un problema propio, en circunstancias que es una mala gestión de la entidad responsable de ser clara. Esta falta de control se observa igualmente en los procesos de pagos online. Cuando hay procesos de pago en línea el usuario es llevado, sin previo aviso, a un sitio web diferente (el del proveedor del servicio). A veces estos sitios no cumplen con estándares de accesibilidad, por lo tanto el proceso puede quedar interrumpido drásticamente. En estos casos el usuario siente que hay una expectativa no cumplida. Un aspecto central observado en la evaluación heurística es que no se respetan como debieran las convenciones web. Este no es un tema menor, pues tiene que ver con aprovechar las experiencias previas de los usuarios, para hacer más efectivos los servicios. Si el usuario es obligado a aprender lógicas nuevas y particulares, pierde tiempo en asuntos que debieran ser intuitivos e invisibles y por lo tanto fácilmente controlables. Algunas convenciones que se pasan por alto reiteradamente: Botones como vínculos a otras páginas, que no necesariamente implican el inicio de una operación, radio buttons para opciones que no son excluyentes, checkbox para opciones excluyentes, no se distingue claramente que texto es hipervínculo dado que no siempre se cuida la convención del subrayado y letras de color azul para denotar la condición de link, se hace pasar al usuario por páginas intermedias informativas como flujo estándar que no van al caso en consultas simples, entre otras. Problemas de incertidumbre en el uso de sitio Los resultados indican que en los TPL faltan indicaciones que permitan al usuario dimensionar el trámite: en que consiste cabalmente, cuanto puede demorarse en realizarlo, cuantos pasos tiene, en que lugar del trámite se está, en que lugar del sitio se encuentra. Es decir, eliminar la incertidumbre. La evaluación muestra que el sentido mismo del trámite a veces no tiene nada que ver entre lo que se ofrece y lo que finalmente el usuario puede hacer. Un ejemplo de esto se observa cuando se ofrece un “certificado en línea” y la función que el usuario termina enviando un e-mail con una solicitud de certificado en vez de recibir efectivamente éste. O se ofrece un “seguimiento” y el usuario sólo tiene acceso a la información inicial que él mismo ingresó al momento de desarrollar una acción en el sitio. Este tipo de incertidumbre se fomenta igualmente con la inclusión de nuevas ventanas en el navegador dentro de la realización misma de un trámite. Esto potencialmente genera problemas como dificultad para regresar a pasos anteriores, imposibilidad de utilizar las herramientas “Atrás” y “Adelante” en el navegador y, en JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

Medición de usabilidad de trámites públicos en línea en Chile: un caso de estudio en gobierno electrónico

los casos más serios –los que utilizan pop ups-, el trámite no puede concretarse si el usuario tiene activado el bloqueador de estos recursos. Otro problema detectado es la inconsistencia de la imagen institucional (logo y tratamiento gráfico corporativo), debido en gran medida al cambio de ventanas. Un problema central detectado es la ausencia de referencias a los usuarios de cuanto podría demorarse la realización del trámite. No se le informa cuántos pasos tiene ni en que consiste cada paso, lo que puede elevar el nivel de incertidumbre y desconfianza hacia el medio digital para la realización de trámites importantes. Igualmente hay una falta de información sobre políticas de seguridad y de resguardo de los datos ingresados por el usuario, sobre todo en las páginas de formularios que solicitan información personal, como RUT, dirección o teléfono. Los prejuicios frente al medio digital son altos, si el usuario no percibe garantías de seguridad y confidencialidad de los datos, difícilmente completará un trámite. La incertidumbre no termina junto con la finalización del trámite, ya que la retroalimentación entregada al acabar los trámites es insuficiente. En la mayoría de los casos sólo puede hablarse de una correcta respuesta del sistema, lo cual no es lo mismo que retroalimentación. Preguntas como ¿Qué pasa ahora? ¿Me van a llamar? ¿Me enviarán un e-mail? ¿En cuánto tiempo? ¿Me van a pedir otros papeles? quedan sin respuesta en la mayoría de los casos. Esto sin duda obliga a los usuarios a usar igual otros medios, como el teléfono, para contactarse con la entidad, en circunstancias que, con un poco más de cuidado en la retroalimentación podría evitarse este innecesario uso de recursos tanto para los usuarios como para la institución. En definitiva, el trámite en línea termina siendo ineficiente y poco rentable para la entidad que lo promueve. Problemas por falta de asistencia Los trámite complejos requieren que el usuario realice operaciones, pero al mismo tiempo hay necesidades de información de asuntos relacionados y resolución de posibles dudas. Es el flujo lógico y tradicional, al que los ciudadanos están acostumbrados en las ventanillas de atención cuando realizan operaciones presencialmente: entregan papeles a una persona y a la vez hacen preguntas y le piden información. Quienes operan en línea buscan ahorrarse filas y burocracia, pero no por eso están dispuestos a perder las guías y la asesoría. En general los TPLs no se hacen cargo adecuadamente de esta manera natural del ciudadano de relacionarse con un trámite y es frecuente apreciar un divorcio entre las facetas informativa y operativa de los TPLs. El flujo propuesto al usuario (ej: todos los trámites del SII) es: primero sólo se informa y después sólo opera, pero no puede realizar las dos cosas al mismo tiempo. El formato no es efectivo ni eficiente y se manifiesta en que no hay un diálogo adecuado entre las zonas públicas –a la que todos tienen acceso- y privada –a la que tiene acceso el usuario sólo cuando ha proporcionado datos personales, como RUT o clave- de los sitios web. Las zonas públicas se enfocan en la información; y las privadas casi exclusivamente a la operación, lo cual obliga al usuario, cuando tiene dudas en medio de la realización de un trámite, a abrir nuevas ventanas para buscar información o a abandonar su sesión (y perder lo avanzado) y buscar la respuesta a sus dudas en la zona pública, para luego iniciar la sesión otra vez. En estos casos el trámite no está JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

100 Palma, M. F., Lavin, C., Rodriguez-Peña, N.

adaptado a la manera en la cual el usuario ejecuta la tarea y por lo tanto resulta poco efectivo. Esta falta de seguimiento detectada en la evaluación se observa igualmente en la falta de un tratamiento adecuado de la información contextual. Por ejemplo, los enlaces o comienzos de procesos que se ubican en zonas de la pantalla diferentes a la invitación al iniciar la operación. Esto obliga al usuario a aprender lógicas distintas entre trámites de una misma entidad, o entre distintas partes de un mismo trámite, lo cual es una práctica claramente ineficiente. Asimismo abundan promesas falsas de información relacionada. Hay vínculos puestos como información contextual, que en realidad no tienen que ver con el proceso. Por ejemplo, acceso a “Preguntas frecuentes” referidas a asuntos globales corporativos de la institución y no al trámite. De esta manera se puede observar que existen 3 focos principales de problemas que repercuten en el alejamiento del sitio evaluado respecto de los estándares de usabilidad requeridos. Las dificultades anteriores redundan directamente en que el sitio de TPLs y el servicio brindado acaban sin ser lo suficientemente efectivos, eficientes, ni cubren plenamente las necesidades de los usuarios que acuden a ellos para satisfacerlas. 4. CONCLUSIONES Tal como se planteó anteriormente, esta investigación tuvo como objetivo realizar una evaluación del estado de la usabilidad de los TPLs en Chile, reconociendo la necesidad de cuantificar este atributo. La usabilidad es un factor relevante para explicar el lento y complejo traspaso de transacciones desde lo presencial hacia lo digital en el contexto de gobierno electrónico. De esta manera la investigación se estructuró de modo de responder claramente a la necesidad de saber, en primer lugar si existen problemas de usabilidad en una muestra de los TPLs ofrecidos a los ciudadanos, y en segundo lugar cuales son efectivamente estos problemas. A partir de la pauta de evaluación generada, los resultados indicaron un nivel general bajo de usabilidad de los 60 trámites evaluados, concentrándose los problemas en el control que los usuarios poseen del proceso, las incertidumbres generadas y no resueltas por el sitio, y la poca asistencia que este provee a los usuarios. Estos tres factores atacan puntos relevantes respecto de las variables que explican la desconfianza e inseguridad que supone la realización de trámites y transacciones en línea para los usuarios. Tal como se indicó anteriormente, la ineficiencia en la entrega de servicios, la creación de incertidumbre y el incumplimiento de expectativas repercute directamente en la confianza de las personas en la institución y genera rechazo y abandono por parte de los usuarios potenciales de un sitio (Balci et al., 2010). Esto transforma a este tipo de mediciones en un instrumento esencial para evaluar las razones por las cuales este tipo de canales no adquieren la masificación deseada, y en segundo lugar, para realizar las mejoras necesarias para que esto ocurra. Las propuestas de mejoras de sitios como el de TPL en Chile requieren estar basadas en información confiable. A partir del diagnóstico realizado, y de los problemas detectados parece esencial focalizar esta información hacia los cambios que deben realizarse en el sitio para JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

Medición de usabilidad de trámites públicos en línea en Chile: un caso de estudio en gobierno electrónico

101

mejorar sus estándares de usabilidad, entendida en el contexto de HCI como un atributo que aumenta la efectividad, eficiencia y satisfacción respecto de un sitio o servicio web. En este sentido, y tal como se muestra en los resultados, un problema significativo son los niveles de incertidumbre al que los usuarios son sometidos una vez que interactúan con el sitio. De esta manera, se proponen para discusión una serie de medidas con las cuales sería posible atacar este sensible punto. Para que el usuario sienta que tiene control en su interacción con el sitio se deben poner indicadores que permitan dimensionar el trámite, es decir, en que consiste el trámite mismo, para que sirve; luego, saber cuantos pasos tendrá que realizar y a medida que lo ejecuta, saber donde está, cuantas acciones le quedan por realizar y anticipar lo que obtendrá de este proceso. Esto ayudaría al usuario a estar consciente del lugar donde se encuentra en el trámite y cuanto le falta. Otro aspecto igualmente relevante es respetar las promesas iniciales. Si se promete comunicación directa con una autoridad, el mensaje de respuesta debe incluir un reforzamiento de esto. Si se explicita que una determinada autoridad responderá las consultas, el mensaje de respuesta debe aludir a que dicha persona o alguien cercano a esta lo hará y así sucesivamente. Algo que es muy sensible en el control de la incertidumbre en el sitio es el flujo entre las zonas públicas y privadas de este. Se debe crear un flujo continuo entre las zonas pública y privada de los sitios web. De esta forma, debe posibilitarse un camino secuencial al usuario en la zona pública del sitio que le permita, mediante autenticación, llegar a la misma materia o tema en la zona privada. Y este mismo contexto las instrucciones, las ayudas, mensajes de advertencia y los mensajes de error deben ser claros y oportunos. Todos estos puntos ayudarían a que los usuarios tuvieran más claridad de lo que están haciendo en el sitio, de lo que pasará luego de la realización de un trámite y de cuáles son las razones por las que no pueden realizar ciertas operaciones. Esto además es un ejemplo para sitios de gobierno en general donde aspectos como la confianza y la imagen traspasada a los usuarios es de primera relevancia para sostener servicios de esta naturaleza en canales digitales. Esto es particularmente importante cuando los usuarios deben realizar procesos con consecuencias importantes o cuando hay dinero de por medio en la realización de un trámite. Tal como se sugiere en la literatura (ver Balci et al., 2010; Krug, 2000; Peng et al., 2004) la usabilidad es una de las variables relevantes que explican el impacto que tienen servicios digitales, contemplando a los TPLs dentro de estos, en los usuarios. De esta manera la realización de mediciones de usabilidad con metodologías rigurosas, y las líneas de mejoras extraídas de estas mediciones son fundamentales para cubrir al menos un aspecto relevante que determina el uso de estas tecnologías y los beneficios derivados de este uso. Por ejemplo, aunando las facetas informativas y operativas de un sitio de e-gobierno se respeta un flujo natural en el cual las personas desean informarse mientras realizan una operación. Con el objeto que los ciudadanos se relacionen fluidamente con las instituciones de gobierno por medio de Internet es importante tener en cuenta que es necesaria la consistencia entre los canales de comunicación, y entre las operaciones que les son ofrecidas. Igualmente es de suma importancia que si los servicios son ofrecidos como “online” entonces comiencen y terminen por medio de este canal y no sea necesario para los usuarios recurrir a otros canales. Esto disminuye la confianza y vuelve al canal digital subordinado y menos relevante. Así, mediante un

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

102 Palma, M. F., Lavin, C., Rodriguez-Peña, N.

cambio de enfoque basado en las medidas prácticas enumeradas se puede promover de mejor forma el uso de los canales digitales por sobre los presenciales. Según los resultados del estudio es posible concluir que seguir dichas sugerencias llevarían a una mejora importante en los niveles de usabilidad de los TPL. Como se señaló anteriormente, el gobierno electrónico ha evolucionado velozmente; pasando de la presentación en línea de las instituciones, a la interacción dinámica de éstas con los ciudadanos en Internet (Moon, 2002). No obstante, los resultados obtenidos muestran que, aún tras todos los esfuerzos realizados y las experiencias exitosas, queda mucho por mejorar, y que la usabilidad de los TPL sigue siendo una variable crítica en la que se juega parte importante de la estrategia del Gobierno Digital de Chile. El mantenimiento de trámites en línea poco eficientes, poco eficaces y que no generan satisfacción en los ciudadanos, tiene consecuencias para la imagen de las instituciones y de la administración pública en su conjunto, y además tiene consecuencias económicas (Tolbert & Mossberger, 2003). El gobierno digital es una promesa de comodidad, rapidez y confianza; su éxito como estrategia se basa, en su efectividad, eficiencia y en la satisfacción de los usuarios respecto de sus servicios, contexto donde la usabilidad es un factor de primera relevancia. Es por esto que presentar trámites en línea poco controlables, y difícilmente predecibles, puede alejar de dichos servicios a un número importante de ciudadanos (Tolbert & Mossberger, 2003). Mejorar la usabilidad de los trámites conlleva el beneficio de la reducción de costos en desarrollo y mantención de los TPL, así como el descongestionamiento de otros canales de comunicación entre los ciudadanos y las instituciones. Este estudio sirve como ejemplo para otros países en el contexto Iberoamericano en el que este mismo proceso desde lo informativo a lo transaccional está operando. Existen antecedentes de esfuerzos en esta línea en países iberoamericanos, como por ejemplo la creación de una Guía de Trámites del gobierno Argentino (Clancy, 2012), donde se ha intentado estandarizar la oferta de servicios gubernamentales en línea siguiendo pautas de usabilidad. No obstante, las evaluaciones objetivas y con metodología rigurosa del estado de la usabilidad de sitios transaccionales de egobierno, y el establecimiento de claras pautas de mejora a partir del diagnóstico son prácticamente inexistentes. De esta forma, las líneas de mejora de los TPLs en Chile puede servir de base para establecer mediciones focalizadas en otros países y los datos pueden usarse para establecer comparaciones respecto del estado de usabilidad en distintos contextos. Además, un aspecto importante en este sentido es el seguimiento de los sitios evaluados, con el fin de observar las mejoras reales y el monitoreo constante para explotar de una forma adecuada los beneficios que supone el enfoque transaccional en e-gobierno, donde este estudio sirve de insumo regional. La usabilidad es una variable crítica que, según los resultados obtenidos, podemos concluir que no está siendo tratada suficiente o adecuadamente en los TPL. Este trabajo es pionero, entregando información sobre el estado de dichos trámites, sugerencias de mejoramiento en lo particular y lo general, estándares de calidad, y una pauta válida y confiable para evaluar el nivel de cumplimiento de dichos estándares. Cada uno de estos aportes resulta por sí mismo significativo a la hora de plantear políticas de mejoramiento, y tomados en su conjunto representan una importante

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

Medición de usabilidad de trámites públicos en línea en Chile: un caso de estudio en gobierno electrónico

103

fortaleza al perseguir un gobierno electrónico más útil, claro y valorado por los ciudadanos. REFERENCIAS Balcı, A., Dalcı, M., Karaman, E., Kutluoğlu, U., Yücel C. Y., Medeni, T. (2010) Spreading Website Usability and Accessibility into Society: Examples from Public and Private Sectors in Turkey. International Journal on eGovernment and eBusiness Studies, 2(1) 19-29. Clancy, M. F. (2012). La guía de trámites del Estado argentino: una ventanilla de servicios accesible para todos los ciudadanos. XVII Congreso Internacional del CLAD sobre la Reforma del Estado y de la Administración Pública, Cartagena, Colombia, 30 oct. Diagnóstico de Usabilidad y Calidad de la Información de los Portales de Internet de los Entes Públicos del Distrito Federal (2007) extraído desde http://www.paot.org.mx/contenidos/doc/Informe_transparencia07.pdf Fang, Z. (2002). E-Government in Digital Era: Concept, Practice and Development. International journal of the Computer, the Internet and Management, 10(2)1-22. Flavián, C., Guinalíu, M., & Gurrea, R. (2006). The role played by perceived usability, satisfaction and consumer trust on website loyalty. Information & Management, 43(1), 1-14. doi: http://dx.doi.org/10.1016/j.im.2005.01.002 Gil-Garcia, J. R., & Martinez-Moyano, I. J. (2007). Understanding the evolution of egovernment: The influence of systems of rules on public sector dynamics. Government Information Quarterly, 24(2), 266-290. doi: http://dx.doi.org/10.1016/j.giq.2006.04.005 Gottschalk, P. (2009). Maturity levels for interoperability in digital government. Government . Information Quarterly, 26(1), 75-81. Hornbæk, K. (2006). Current practice in measuring usability: Challenges to usability studies and research. International Journal of Human-Computer Studies, 64(2), 79-102. doi: http://dx.doi.org/10.1016/j.ijhcs.2005.06.002 Hung, S., Chang, C., Yu, T. (2006). Determinants of user acceptance of the eGovernment services: The case of online tax filing and payment system. Government Information Quarterly 23(1) 97-122. Jamieson, S. 2004. Likert scales: how to (ab) use them. Medical Education, 38, 12121218 Leavitt, M. O. y Shneiderman, B. (2006). Research-Based Web Design & Usability Guidelines. In Background and Methodology. U.S. Department of Health and Human Services. Washington. Molich, R. & Nielsen, J. (1990). Improving a human-computer dialogue. Communications of the ACM , 33( 3), 338-348

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

104 Palma, M. F., Lavin, C., Rodriguez-Pe単a, N.

Moon, M.J.(2002). The evolution of e-government among municipalities: Rhetoric or reality. Public Administration Review, 62(4), 424-433 Nielsen, J. 1992. Finding usability problems through heuristic evaluation. Proceedings ACM CHI'92 Conference, Monterey, Mexico, May 3-7. Nielsen, J. and Mack, R.L. (eds) (1994). Usability Inspection Methods, John Wiley & Sons Inc. Nakamichi, N., Shima, K., Sakai, M., & Matsumoto, K.-i. (2006). Detecting low usability web pages using quantitative data of users' behavior. Paper presented at the Proceedings of the 28th international conference on Software engineering, Shanghai, China. Norris, D. F., & Moon, M. J. (2005). Advancing E-Government at the Grassroots: Tortoise or Hare? Public Administration Review, 65(1), 64-75. doi: 10.1111/j.15406210.2005.00431.x Peng, L. K., Ramaiah, C. K. and Foo, S. (2004). Heuristic-based User Interface Evaluation at Nanyang Technological University in Singapore. Program: Electronic Library and Information Systems 38 (1): 42-59. Reddick, C. G. (2005). Citizen interaction with e-government: From the streets to servers? Government Information Quarterly, 22(1), 38-57. doi: http://dx.doi.org/10.1016/j.giq.2004.10.003 Tolbert, C., & Mossberger, K. (2003). The effects of e-government on trust and confidence in government. Paper presented at the Proceedings of the 2003 annual national conference on Digital government research, Boston, MA.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 85-104

www.jistem.fea.usp.br

JISTEM - Journal of Information Systems and Technology Management Revista de Gestão da Tecnologia e Sistemas de Informação Vol. 11, No. 1, Jan/Apr., 2014 pp. 105-130 ISSN online: 1807-1775 DOI: 10.1590/S1807-17752014000100007

TECHNOLOGICAL INNOVATION PROJECTS: PROPOSAL FOR AN INTEGRATIVE MODEL BETWEEN PROJECT MANAGEMENT AND KNOWLEDGE MANAGEMENT IN A CUSTOMER-SUPPLIER PERSPECTIVE Edval da Silva Tavares Escola Superior Nacional de Seguros, Funenseg, São Paulo, SP, Brazil Marcelo Schneck de Paula Pessoa University of São Paulo, POLI/USP, São Paulo, SP, Brazil ______________________________________________________________________

ABSTRACT In face of strong competition, companies in Brazil have increased their financial investments in automation, offering new products and services and reducing operating costs. These companies are focusing their efforts on core competencies and, therefore, they often lack the internal expertise to implement new projects, especially those that bring technological innovation. For this reason, we use the processes of outsourcing or subcontracting to help implement such projects. The unit of analysis in this study is the project and the object of the study to analyze the process of knowledge transfer from a provider to a customer during the duration of the project, which involves new technologies. The main motivation of this work is to address the acquisition and retention of new knowledge related to projects designed for business customers. We have developed a study of three cases of multiple financial firms that buy new technologies and two suppliers of information technology. As a practical result, a management model of knowledge transfer and retention of knowledge in client companies is proposed and incorporated into project management. Keywords: Knowledge Management; Project Management; Innovation; Information Technology

_____________________________________________________________________________________________ Manuscript first received/Recebido em: 03/01/2013 Manuscript accepted/Aprovado em:11/02/2014 Address for correspondence / Endereço para correspondência Edval da Silva Tavares, Escola Superior Nacional de Seguros, Funenseg. Bachelor’s and Master’s in Business Administration, Feausp. Ph.D., Polytechnic School of USP, Department of Production Engineering. Adjunct Professor, Escola Superior Nacional de Seguros, Funenseg. Consultant, PricewaterhouseCoopers. GTI Researcher— Management in Information Technology, Department of Production Engineering, USP. Previously employed at Unibanco S/A, Fatec College of Technology SP, Vanzolini, Senac, Brasilprev, and Zurich Insurance. Address: Av. Paulista, 2421 – 1st floor, CEP:01311-30, Consolação, São Paulo, Brazil. Phone: +55 11 3062-2025, Email: est@terra.com.br Marcelo Schneck de Paula Pessôa, Polytechnic School of USP, Department of Production Engineering. Electronic Engineer, M.Sc. and Ph.D. Polytechnic School of USP. Associate Professor in Management of Information Technology, 2009. Professor and researcher, GTI, Management in Information Technology, Department of Production Engineering, USP. Have worked on projects related to computers, telephone exchanges, railway and industrial automation systems. Address: Av Almeida Prado travessa 2 n.128, CEP 05508-070, University City, São Paulo, Brazil. Phone: +55 11 3091-5363. Email: mpessoa@usp.br Published by/ Publicado por: TECSI FEA USP – 2014 All rights reserved.

106 Tavares, E. da S., Pessôa, M. S. de P.

INTRODUCTION

This article addresses the issue of implementing projects with technological innovation in an organization that is internalizing such new knowledge. The need for innovation is caused by the competitiveness that forces organizations to invest in product and service innovations to ensure their survival (Tidd et al, 1997; Febraban, 2003). Innovative products and services in a complex environment of Information Technology (IT) require structured projects that can be custom developed by third parties (Blechar, 1998; Cleland, 1991; Pressman, 1995). The distinction between the most innovative companies is made apparent by the right supplier-customer relationship. Those companies are the most concerned with anticipating market trends and innovative practice of substantive character, TorresFreire & Henriques (2013). The need for structured projects requires knowledge of Project Management (PM). Due to the focus on core competencies (Quinn, 1999; Prahalad, 1998 Dacorso & Silva 2013), and the internal knowledge gap, PM subcontracts to bring innovation to an organization. This gap in technical knowledge is natural in an organization before it implements new technologies in an IT project and creates two important risk factors: (Keil et al, 1998) the risk created by the project itself and the risk created by lack of knowledge, Ansari (2013). System integration projects are those that need to relate to various existing systems and are of relative complexity. These projects require new technologies, which often lack internal knowledge. This article describes a survey focused on complex projects involving new technologies unknown to the buyer. This focus is important because this context characterizes the problem that is addressed and the existence of a gap in technical knowledge on the part of the client company in relation to the supplier’s knowledge. This article is organized by the following chapters: Introduction, Conceptual Aspects (technological innovation, knowledge management and acquisition management), Problem (characterization of the problem and model of the problem), Grouping Hypotheses (proposed model of knowledge transfer), Study Cases (research planning, developing cases, step by step implementation of the study and research limitations), Results (consolidation of the research results, score-based analysis of adherence to the model, comparison of results between suppliers and customers level, interaction between customer and supplier knowledge transfer process, process of knowledge transfer, whether a process is conducive to knowledge management, and knowledge transfer), Results of Questions, Future Research, and Final Thoughts. See article summary in Figure 1, below.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Technological innovation projects: proposal for an integrative model of project management and knowledge management perspective customer-supplier

There is an increased investment in automation to face strong competition, offering new products and services and reduce operating costs.

107

Because of the concentration on core competencies and, as a consequence, there is a knowledge gap in the implementation of new projects, especially those that bring technological innovation.

For this reason, we use the subcontracting process to help implement these projects.

The unit of analysis in this study is the project and its purpose is to analyze the process of knowledge transfer from a provider to a customer at project time involving new technologies.

A minimum level of knowledge should be transferred from the seller to the customer which should be enough to permit the customer to judge its choice of new technologies and to design new products.

A study has been developed, based on multiple cases of three financial companies that buy new technologies and two suppliers of information technology. As a practical result, a management model of knowledge transfer and retention of knowledge in client companies is proposed and incorporated into project management.

Figure 1 - Article summary Source: Authors.

CONCEPTUAL ASPECTS

In this section, a literature review related to innovation, strategy, knowledge management and acquisition management is presented. 2.1.

Technological innovation

Technological innovation can be performed on the product, service or process and can be performed either incrementally or through a radical transformation, (Tidd et al, 1997:7). Thus, technological innovation addresses matters that are not part of a companyâ&#x20AC;&#x2122;s operation that must be incorporated in an economically viable manner. There is a need for a distinctive treatment of innovation projects and regular projects, Takeishi (2002:321-338). For regular projects, with known technology, the most important architectural knowledge has an overview of the components, whereas when a project involves new technologies, it is important that the organization either have a high level of knowledge of the specific component to solve unexplored engineering problems or acquire solutions using IT innovations (Ireland, 1999;

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

108 Tavares, E. da S., Pessôa, M. S. de P.

Takeishi, 2002) by subcontracting suppliers that have this knowledge domain, as shown in Table 1 (Kamel (2003); Steensma & Corley, 2000).

Project with Innovation (new technologies)

Regular project (known technologies)

Type of project

Division Effective model

Automakers should have

Clear division

Architectural knowledge

Architectural Superposition

knowledge and Specific knowledge of component

Providers should have

Specific knowledge of component

Architectural knowledge and Specific knowledge of component

Table 1 - Model suggested for an effective division of knowledge, Takeishi (2002) The strategic aspect of IT is also analyzed in Table 2, which shows McFarlan’s (1984) universe, a place that the same author, in partnership with Nolan (NOLAN,2005), recently revisited and where companies can be positioned in a strategic matrix. The horizontal axis of Table 2 shows the need to develop new applications, whether for reasons related to the market or to the rapid incorporation of new technologies or new features. The vertical axis shows the need to use a trusted IT organization, i.e., if an IT fails, the business is directly affected. This division leads to four quadrants, with the extreme right quadrant deemed Strategic, which shows the importance of IT to an organization’s strategy. The Factory quadrant shows the companies for which IT must be reliable, but do not feel pressure to develop new applications. The quadrant of IT Support does not show a need for new applications or great reliability, i.e., it is fault tolerant in the absence of a direct injury to the business. The Transition quadrant is so called because it houses those organizations that require new applications, but it is expected that over time the organizations in that quadrant evolve to the Factory or Strategic quadrants.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

IT DEFENSIVE

IT OFFENSIVE

High

Factory

Strategic

Low

Need for reliable IT (present)

Technological innovation projects: proposal for an integrative model of project management and knowledge management perspective customer-supplier

Support

Transition

Low

High

109

Need for new applications (future) Table 2 - The Four Quadrants, NOLAN, McFarlan (2005)

In the case of banks—the research focus of this paper—the framework is the strategic quadrant related to the need to continually develop new applications and the need for highly reliable IT. 2.2

Knowledge Management

During the research on the meaning of knowledge, it was identified that there are difficulties in its conceptualization.As with knowledge itself, its concept is fluid and subjective and in this paper, the concept defined by Nonaka & Takeuchi (1997), Myburgh (2000), Bollinger (2001), Beijerse (1999) and Probst (2000) is used. Knowledge covers a scope with boundaries that are not well defined. It always has a goal, a target to be reached, an action and a meaning; it requires a base of information, which can be suppressed by skills and competencies, but it mainly depends on the beliefs and attitudes of the people who possess that information. Therefore, knowledge is always connected to human actions, which are based on and correlated with a set of information, giving them meaning so as to perform some actions. Knowledge is also compared to an iceberg, in which the visible part is the knowledge that can be made explicit and the submerged part is tacit knowledge, or those concepts present in the process of knowledge creation, which is called the SECI model, (Socialization, Externalization, Combination and Internalization). Through the process of socialization, tacit knowledge is passed from one person to another. Externalization, i.e., the structure and organization of knowledge, makes it explicit that combining knowledge with knowledge leads to the internalization and consolidation of new tacit knowledge. That knowledge then goes back to the beginning of the cycle, where more knowledge can be generated (Marçula, 2001). Fagan (2001) is based on studies of IT adoption and implementation and identifies a large number of factors that may impact the process of transferring IT projects and associated knowledge. Twenty-three factors are identified and grouped into the following five categories: individual, organizational, innovation, task-related and environmental factors. Another proposal for transfer of knowledge is presented by Karhu (2002), using a framework based on the literature and case studies. The first case addresses the difficulty of explanation experienced by technical professionals, and the second case discusses a situation in which a company has a large amount of information that it does

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

110 Tavares, E. da S., Pessôa, M. S. de P.

not use properly in the decision-making process. In this case, the problem discussed relates to: • Challenges in documenting technical information; • Knowledge distributed in different countries; and • Information that is critical to a company. To address these problems, Karhu (2002) recommends the use of so-called professional knowledge administrators. These professionals describe others’ experience and knowledge. A knowledge administrator also acquires information for a company, analyzes the information in detail, formulates it and documents it for use by knowledge seekers. The technique of “mentoring”1 is a powerful method used in the knowledge transfer process, but it is only appropriate when knowledge is not distributed among many different individuals and localities. In this case, interviews and observation prove adequate for this type of knowledge transfer, with the help of knowledge administrators’ tools. Trust and personal relationships are basic to achieving positive results in the acquisition of knowledge factors. Thus, the framework follows the following steps: 1. Interviews of technical professionals and observation of their work; 2. Documentation of knowledge gained from the analysis, descriptions and written instructions; 3. Feedback using a professional process to ensure content quality; 4. Documentation that provides the basis for reading and learning from other people, who may use those readings in their own experiences and then assemble their own mental models or tacit knowledge. Another view of knowledge transfer that it is related to tactical knowledge and explicit in projects as demonstrated by Batra (2007), who has conducted a survey involving knowledge from one project applied to another project. Batra’s argument is that knowledge management for development requires a mix between knowledge that is inside the company, which in large part is tacit, and external knowledge, which in large part is explicitly local knowledge. The author explains that the contextual knowledge of a problem is more deeply understood and captured before solutions to a development project may be implemented. A survey of 46 closed projects shows that the body of project knowledge taken from the knowledge of other projects is associated with the performance of the surveyed projects. Despite the benefits of this knowledge transfer, research has shown that an increase in investments that account for this process. However, one suggestion to reduce this negative impact of a high level of effort for knowledge transfer is to assign a limited number of project members that effort. With respect to methods of knowledge transfer, it is recommended that project managers and project-based organizations use both formal methods (e.g., meetings, special teams, project reviews, mentoring, written messages, documents and observations of the results of other projects) and informal methods (e.g., interactions between project members with or without the use of multimedia resources, teleconferencing, emails and chats), Landaeta, (2008). Although innovation outsourcing has become quite widespread in field research and development, project managers always want to know what PM practices are necessary for the development of new technologies that use suppliers (Zhijian&Loch, (2009). An analysis of 24 of Siemens’s projects to develop embryonic technologies, 1

“Mentoring” is defined as tracking procedures that provide support or training.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Technological innovation projects: proposal for an integrative model of project management and knowledge management perspective customer-supplier

111

which used different sources of outsourcing, identifies five success factors common to such projects: reliability, communication, strong supplier competence, strong internal competence, a clearly defined problem, and an alignment of interests. These factors can be summarized by the importance of the level of relationship between the customer and its supplier. One study shows that according to 73.9 % of workers at software companies, there is an incentive to share knowledge, even where there is a formal and structured knowledge management program (Souza et al, 2010). To provide an example that focuses on business processes, knowledge sharing is also cited as a positive contribution by knowledge management in market-oriented companies, Ferraresi et al, (2012). The results of another study reveal the need to facilitate the integration of human resource management into managing knowledge processes, a proposal that would address the formulation of policies and strategies to strengthen human resource departments’ primary selection, training and development activities, along with their activities related to communication, design work performance evaluation and compensation, to encourage the creation, storage, sharing and application of knowledge, Gelabert et al, (2012). Medina (2012) concludes that the process of knowledge transfer in knowledgeintensive business services (KIBS) firms is facilitated by the process of socialization among experts, using the mechanism of meetings in which people share stories involving issues, training, coaching, mentoring, shadowing, practice and virtual social networking communities. The process of knowledge acquisition aims to gain knowledge from various sources internal and external to an organization. One of the fundamental principles of knowledge-transfer capacity is the location and acquisition of knowledge through collaboration between individuals and partners. The acquisition of knowledge by suppliers as a capacity for knowledge transfer is present in all of the reference models of the software process, Galvis-Lista & Sánchez-Torres (2013). In summary, the process of knowledge transfer, as in Karhu (2002) Marçula (2001), Fagan (2001), Tsang (2002), Batra (2007), Landaeta (2008), Zhijian & Loch (2009), Souza et al, (2010), Medina (2012), Ferraresi (2012), Galvis-Lista & SánchezTorres (2013), may rely on various methods of transfer and acquisition of knowledge, but it always involves human interference. The complexity of transforming tacit knowledge into explicit knowledge—or even transforming tacit knowledge into tacit knowledge—is the major challenge. 2.3. Management Acquisition The management of acquisition is a knowledge area of PM that includes goods and services that must be purchased from another business to complement the scope of a company’s project processes. This management’s job “is to plan, conduct, administer and close acquisitions,” as in PMBOK (2008). There are several levels of contemplated subcontracting, from subcontracting the project as a whole to demanding simple programming of previously defined components. This research focuses on complex projects in which the fundamental

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

112 Tavares, E. da S., Pessôa, M. S. de P.

knowledge belongs to the supplier(s). This focus is important because it is this context that characterizes both the problem addressed and the existence of a gap in technical knowledge on the part of a client company in relation to its supplier’s knowledge. Thus, the scope of this study does not involve only cases of subcontracting at the project implementation stage. This phase should be part of the case, but we do not view it in isolation. 3.

PROBLEM

In this section, the problem focus of the research is characterized and two models are presented: the problem’s characteristics and knowledge transfer. 3.1. Characterization of the problem This research aims to contribute to the processes of PM that incorporate aspects of knowledge management (KM), both during project execution and after project completion. PM processes should be complemented by KM, for example, through improved interaction with suppliers, knowledge transfer and the provision of an enabling environment for KM. Therefore, the central problem is how to use the set of KM practices within the context of PM to improve the processes that develop IT projects at various stages. This process of integration among areas of knowledge, where new knowledge is based on knowledge gained in other areas, is cited by Lovejoy (1996:1-22), who shows how operations management depends on the relationship among various disciplines or areas of expertise. Innovation projects that include at least partially new knowledge are considered strategic and, therefore, must be internalized by transferring a company’s supplier(s) to the acquiring organization. This transfer is justified because a company needs this internal knowledge, at least to judge the competence of service providers and enable the development of new products or services, because the time periods between such releases have become increasingly smaller. Thus, it is critical to structure additional processes to address this knowledge transfer. The acquisition of knowledge through a vendor also depends on a good customer-supplier relationship (Karlsson, 1998). The pursuit and implementation of IT solutions are not sufficient in themselves, without the transfer of knowledge from the supplier(s) to the client company. The process of the internalization of knowledge is essential as a basis for both the construction of new projects and the evaluation of suppliers (experts) (PMBOK, 2008; Probst, 2000; Byrd & Turner, 2001). The argument for internalization is particularly strong when a company is positioned in the strategic quadrant of the McFarlan model and there is a strategic impact on its application development portfolio (Nolan, 2005; Cordenonsi, 2001). An informal and sometimes conflicting relationship with suppliers can lead to failure to achieve project objectives and a consequent failure to absorb knowledge. Additionally, it is important to realize the importance and feasibility of internalizing internally nonexistent knowledge and transferring new knowledge.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Technological innovation projects: proposal for an integrative model of project management and knowledge management perspective customer-supplier

113

From the above, the implementation of KM practices is very important because companies use their suppliersâ&#x20AC;&#x2122; expertise to cope with competitiveness and innovation (Lim et al, 1999). 3.2. Model of the problem To better explain the context of the survey, the conceptual model presented in Figure 2 was developed. New products and services often require the use of new technologies. The complexity of technology, urgency of implementation and demand for low-cost project financing are assumed. Projects to generate products and services with innovative components require new knowledge (see the right side of Figure 2), which must be internalized at a level that provides the power of judgment in relation to the suppliers of new technologies and the construction of new products and services. Acquisition is the area of PM that addresses this search for solutions. In this area, subcontracting and outsourcing are justified by a companyâ&#x20AC;&#x2122;s need to focus on its core competencies. Importantly, the term subcontracting is used as a way to describe hiring workers to develop custom IT applications in complex projects with a high degree of integration. Thus, when firms seek new knowledge through suppliers, they should be concerned about and alert to the proper transfer of knowledge. This model analyzes the difficulties and shortcomings of this process and investigates the following questions: 1. How is it possible to enhance the model of acquiring these solutions within the context of PM and knowledge transfer? 2.

What is the best way to transfer knowledge at the time of the project?

Figure 2 - Model of the problem to be analyzed Source: Authors.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

114 Tavares, E. da S., Pess么a, M. S. de P.

GROUPING HYPOTHESES

The explanation of the concept that underlies both this article and the problem to be treated, along with the questions to be answered, is organized following the grouping of our hypotheses. Figure 3 summarizes the elements of this research with a description of the problem (P1), the main questions (Q1, Q2, Q3) and the hypotheses H1 to H6). The hypotheses were developed through the literature review. The H1, H2, and H3 hypotheses were used to confirm the sample of the cases, and the H4, H5, and H6 hypotheses were used to analyze forms of knowledge transfer.

Figure 3 - Grouping of hypotheses. Source: Authors.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Technological innovation projects: proposal for an integrative model of project management and knowledge management perspective customer-supplier

115

Once the problem to be solved and the points to be investigated are clearly understood, the model developed to represent the transfer of knowledge described in Figure 4 can be presented. So far, the study shows that the literature has provided no specific model that solves the problem posed in Figure 2 or specifically answers the questions proposed in this paper. Thus, we propose a model based on other models that partially respond to our proposal. The above model has a core that is intended to guide the actions and interactions between a client company and its supplier. The client company engages in a number of internal activities, represented on the left. The client company has IT projects that incorporate technological innovation that remains focused on its â&#x20AC;&#x153;core businessâ&#x20AC;? and provides subcontracting suppliers with knowledge necessary for technological innovations. With regard to creating an environment that is conducive to KM, there is a need to create a capture process, an organization, access to and the use of new knowledge. The company should also seek to use IT tools for the storage and dissemination of new knowledge. The PM process must contemplate the planning of this process. The supplier, represented on the right, possesses the technology, expertise and knowledge that are to be transferred. Regarding interactions between firms, clear rules for the qualification and delivery of knowledge must be established. One should seek a higher degree of formalization and focus on the processes of transfer of new knowledge. At the core of the model is the process of knowledge transfer. This process consists of mapping activity related to gaps in knowledge, new knowledge documentation, previous training and mentoring process. At its core, this model basically consists of the model proposed by Kamara et al, (2002), which presents a framework for selecting a KM strategy that can be used as a basic model to identify knowledge needs, the target knowledge to be transferred and finally, the selection of the method of knowledge transfer. This model can also be complemented with the observations of Landaeta (2008), Medina (2012), Ferraresi (2012), Galvis-Lista & SĂĄnchez-Torres (2013), which discuss the process of knowledge transfer.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

116 Tavares, E. da S., Pessôa, M. S. de P.

Figure 4 - Proposed model of knowledge transfer Source: Authors.

It is also suggested that this transfer would not be effective without a good relationship between the client company and its suppliers, who are subcontractors for creating IT projects. This assertion is supported by Tserng & Lin (2002), Kern & Willcocks (2000), Zhijian & Loch (2009) and Torres-Freire & Henriques (2013), who claim that the improvement of relationships is a factor in the success of the subcontracting process. The process of interactions between a client company and its supplier is also critical in this model, more precisely at the moment of a company’s JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Technological innovation projects: proposal for an integrative model of project management and knowledge management perspective customer-supplier

117

choice of supplier and/or technical solution. A client company that needs an innovation solution has not yet learned to judge the best or most appropriate solution for its business problem. In such a case, one can rely on a model of interaction among companies synthesized in the model as interactions (Frezatti & Tavares,1999). Finally, as Abell (2000), Tocan (2012) and Badal (2013) explain, it is essential to create or maintain an environment conducive to KM. Therefore, it is suggested here that some processes of PM are increasingly streamlined and effective, and it is also suggested that companies improve their relationships with suppliers and create a conducive environment for KM. 5.

CASE STUDIES

5.1

Planning Research

To test the model shown in Figure 4, the case study research method was considered appropriate because it enabled the analysis of several companies, suppliers and customers with an appropriate degree of depth and a specific focus on new technologies, as described below and as recommended by Yin (2001). Data collection was carried out through closed-question interviews, which provided the researcher with a more accurate picture of the environment to be searched, although there were also open-ended questions to complement the information obtained. This format was based on a questionnaire interview that previously prepared and was answered by “informants” and presented the interview as “structured” or as “a formal survey”. A personal interview enabled the project managers to present their interpretations of certain events. Furthermore, the interviews relied on an important complementary factor: direct observation through a field visit. To facilitate the proper chain of evidence and to obtain a proper comparison between the respondents’ information, or even comparing the cases together, we used the Likert scale process as a way to measure and quantify the results related to the blocks referenced in the questionnaire. Because the object of the research is the flow of knowledge between suppliers and their customers, we selected three customer companies represented by three major banks characterized by their volume of assets and two major suppliers characterized by their volume of sales. For reasons of confidentiality, the names and identifying organizations where the case studies were conducted were omitted. Table 3 presents a summary table containing information about the main characteristics of the studied organizations.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

118 Tavares, E. da S., Pessôa, M. S. de P.

Key Numbers

Bank A

Volume of Assets

US$ 65.1 billion

US$ 109.5 billion

US$ 41.6 billion

29,600

40,000

23,000

Number of employees

Bank B

Bank C

Annual revenues

Supplier A

Supplier B

US$ 32 billion

US$ 86,4 billion

Table 3: Numerical information about surveyed companies Source: Surveyed companies.

The research procedure was as follows: 1. We pretested the questionnaire with project managers who did not participate in the final survey to ensure clarity of the questions and coverage of the items to be searched; 2. The questionnaires were usually administered to two managers in each company (i.e., the customer and the supplier) who had participated in projects with innovative components; 3.

We received and analyzed the questionnaires;

4. We conducted a personal interview with each manager that responded; because two managers were interviewed at each company, we were able to check the similarity of business information; 5. We made sporadic phone calls and visits to clarify specific points that were not clear, or divergent points between two managers at the same company; 6. All of these steps together took approximately 2 months, reflecting 200 hours of total work, which shows the depth of our study in this research field. 5.2. Development of cases The cases were developed through visits to the studied companies, where we conducted interviews with several people. The questionnaire organized support for the hypotheses through a series of questions to be answered by the respondents. Figure 5 shows the organization of the survey and the case study process: • Blocks 1 and 2 - Information about the company • Block 3 - Characterization of the process of innovation • Block 4 - Characterization of IT projects • Block 5 - Level of interaction(s) with the supplier (s) • Block 6 - Process of knowledge transfer • Block 7 - Environment conducive to G.C. • Block 8 - Numerical information to give the dimension of the importance of the cases

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Technological innovation projects: proposal for an integrative model of project management and knowledge management perspective customer-supplier

119

• Process analysis was used to consolidate the results in a comparable way, using the theoretical models as a parameter.

Figure 5 - Walkthrough of the study Source: Authors.

The consolidation was performed by comparing the results with theory and the cases among themselves, i.e., for each row of the table, the results of each client’s case were compared with B and C and with vendors A and B. 5.3. Research Limitations Because the case study analyzes specific situations, such as the example in this study that analyzes the process of knowledge transfer over time from to a technology provider to its client (in the case of financial industry firms) during a project, adopting a degree of depth, one cannot generalize this study’s results to other situations or companies. For replication to occur for other companies, it is critical to present similarities to the cases surveyed. The following presents the results obtained in each of the points mentioned in the analysis and organization of results as shown in Figure 5. 6.

RESULTS

6.1

Consolidation of the results of the search

Blocks 3 and 4 (sample confirmation) check whether the cases made assumptions about IT projects with technological innovation, and blocks 5 through 7

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

120 Tavares, E. da S., PessĂ´a, M. S. de P.

relate to hypotheses H4, H5 and H6. The correlations among the hypotheses and the answers to the posed questions are described in the next subsection of item 6 results. 6.2

Analysis of adherence to the model using the score

Scoring the result of the closed questions, the analysis of the open questions, the interviewersâ&#x20AC;&#x2122; observations and the examination of other evidence during the interviews allowed for the general analysis of this item. However, it is also important to explain the numerical result of what is quantifiable, such as Table 4, in line with the assumptions presented in Figure 3. Analyzing the rows in Table 4, we observe that the only block that is integrally bonded to the model for all cases is the characterization of IT projects. For others, there are within-case variations. The remaining blocks vary in adherence according to the company searched. Observing the score for the event, the company that comes closest to the model, restricted to the quantitative view, is supplier B. The results of the research versus the assumptions of H4, H5 and H6 are detailed in sections 6.3 through 6.6 of this chapter.

Table 4 - Score resulting from field research: cases of corporate customers and suppliers Source: Authors.

6.3 Analysis of the level of interaction between customer and supplier versus hypothesis H4 The interpretations of this item are based on the premise that there must be an improvement in the level of interaction between suppliers and customers. The research result shows that companies have a level of interaction with their suppliers and customers that evolves and does not fully adhere to the model, as in

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Technological innovation projects: proposal for an integrative model of project management and knowledge management perspective customer-supplier

121

Tserng & Lin (2002, p. 107-108), Kern & Willcocks (2000, p. 328-329), Zhijian & Loch (2009) and Torres-Freire & Henriques (2013), who claim an improvement in the relationship success factor benefits the subcontracting process. Although most managers interviewed stated that their level of interaction is appropriate, they also recognized that there were problems such as a lack of clear rules of engagement and informality in processes involving new suppliers or subcontracting. The client company that is closest to the model is Company C, and the supplier company that is closest to the model is Company B. The highlight of the findings related to client company C is that there are clear rules for approval and competition among providers and principles of social responsibility that influence the company’s relationship and interaction with suppliers. From a supplier’s point of view, one respondent commented that there are situations in which customers were technically unprepared, that the relationships were not always good and often that problems with projects were blamed on suppliers, which in his view was incorrect. Another supplier took the position that clients should have an organizational focus on projects with innovative components, particularly with regard to planning for the time and resources necessary to absorb new knowledge. According to supplier companies’ project managers, projects with innovative components cannot be managed in the same way as projects that use technologies known by the customers. The other highlight related to the supplier B, which showed total adherence in the search for appropriate interaction with its customers, using many of the processes of PM and the benefits of KM in projects for its clients. Although supplier B did not completely adhere to the model in the general context of all cases, as observed in the case of client company C, in the case of supplier B and the reports of managers from other companies, this need and trend in the model search is proposed as a success factor in subcontracting and achieving project objectives. 6.4

Analysis of the result of knowledge transfer versus the Hypothesis H5 process

The interpretations of this item were based on the premise that the transfer of knowledge at project time can be composed of the following instruments: training, preparation of documentation, interviews, and mentoring. The first analysis was performed comparing the results with the proposed model and then comparing the cases, i.e., compliance and coincidence of these cases to one another. Other aspects considered in this interpretation were relevant evidence and the cases’ most significant aspects. The trace model of knowledge transfer is based on the research of Nonaka&Takeuchi (1997, p. 68-69), Karhu (2002, p. 436), Fagan (2001, p. 5-26), Tsang (2002, p.835-854), Kamara et al, (2002, p. 205-211), Landaeta (2008), Medina (2013), Ferraresi (2012), and Galvis-Lista & Sánchez-Torres (2013). In general, these authors address the process of form and tools for knowledge transfer. The evidence considered relevant to the surveyed projects, along with the items that were the most significant to those projects involved technologies that were new to the client companies at the time they were developed. In the case that follows, d and e represented projects that were innovative to the suppliers’ business customers. Companies where these projects were executed, given the “heavy reliance on technology” and development activities, are located in the strategic quadrant, according to Nolan & McFarlan (2005), of Table 2. These projects were analyzed based on Blechar (1998, p. 6) and involved changes in the installed and/or technical and/or application layer

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

122 Tavares, E. da S., Pessôa, M. S. de P.

architecture technology environments. To different degrees, these changes represented a technological breakthrough for the companies involved. The model of knowledge transfer was observed in all cases studying corporate customers and suppliers, although to various degrees and shapes. The following points were observed when considering the items analyzed, i.e., mapping, conversion forms, instruments of transfer, storage forms, modalities and level of knowledge transferred within the project: 1. The process of knowledge transfer must evolve if search results are compared with the proposed model, especially regarding the process of making knowledge explicit. All of the project managers that were surveyed reported that after the process of knowledge transfer, the most consistent form of knowledge was tacit. This suggests the fragility of the KM process. Although projects were documented in many cases, it was often recognized that the documentation was “just for the record.” The instrument of socialization is important, but only when it is used, the KM process becomes incomplete and businesses continue to rely solely on people to use and add to the knowledge gained. 2. Comparing the results—specifically, the model presented by Kamara et al, (2002)—it is observed that there is one group considered as a source of knowledge and another targeted to receive new knowledge. Each of these groups has three components: people, paper and software. By comparing this model with the results of the studied cases, it is observed that the major focus is the component of people who includes only isolated examples of papers and software, as in the case of supplier B. 3. Another aspect that shows the need for a development environment for KM is one not been found in the cases studied (except for vendor B): that of a formal function administrator or an integrator of knowledge, as set forth in Marçula (2001, p. 130-134) and Karhu (2002, p. 430-446). 4. Evolution can also be analyzed starting from knowledge “built by the people” and progressing to knowledge “embedded in capital” (Prahalad, 1998, p. 20). Using this vision, it can be stated, based on all of the evidence, that the knowledge of the analyzed cases tends to merge into group knowledge “embedded in people” rather than the group’s “corporate capital” knowledge. This statement is also aligned with the statement of the item that includes KM, in which it is found that in all of the projects surveyed, the most consistent, existing, post-transfer knowledge is in the form of tacit knowledge. 5. In the cases studied, we also find that companies need to differentiate themselves by emphasizing certain PM processes in projects with innovative components versus those with known technology environments. This occurs to the extent that the project managers surveyed had a dual role, i.e., the mission to internalize new knowledge and the task of implementing the project itself, see Meredith & Mantel (1995) and Tavares & Pessôa (1999a). Adherence to the model is proven to the extent that all project managers acknowledged the importance of knowledge transfer from suppliers in projects involving new technologies. There is also overlap between the processes used in knowledge transfer. However, the transfer processes used, in most cases, focus only on people using the method of socialization for converting knowledge. The evolution of the proposed model is mirrored in the process used by supplier B, which involves KM processes in its PM.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Technological innovation projects: proposal for an integrative model of project management and knowledge management perspective customer-supplier

6.5 H6

123

Analysis of the result of the enabling environment for KM versus Hypothesis

The interpretations of this item were based on the premise that there must be an enabling environment for KM. The results show that for client companies A, B and C and for supplier A, although most of the items related to the processes that allow the creation of a KM environment are practiced, these companies do not have a process explicitly for this purpose. The processes that do exist are important but informal. The only exception is supplier B because that company practices all of the items related to the processes that allow the creation of a KM environment and the company also has an explicit process for this purpose. For supplier B, the processes that exist are both important and formal. In this company, beyond those cases deserving of mention, there are storage processes for new knowledge, as well as processes to spread knowledge using IT tools, making that new knowledge available to every business and in some cases, to customers. 6.6 Knowledge Transfer: the results of the issues that consolidate Hypotheses H4, H5 and H6 This item considers the issues presented in item 3.2, which concern the presentation of the problem model. Thus, it is possible to obtain guidance on how to resolve the two questions posed. QUESTION 1: How is it possible to enhance the model for acquiring these solutions in the context of PM and knowledge transfer? Improving the procurement model for outsourcing IT projects with components of technological innovation should occur primarily through the following measures: 1. There should be an improvement in the interaction between client companies and their suppliers through bidding and approval processes that have clear rules and opportunities for all suppliers. Although this reality is not quite the same for all, as set forth in the model outlined by Tserng & Lin (2002), Kern & Willcoks (2000), Zhijian & Loch (2009) and Torres-Freire & Henriques (2013), client companies do seek to improve this model, with the full support of their suppliers. Importantly, client organization C and supplier B use very similar processes to those used in the model mentioned above. 2. The planning and execution of some processes of KM at project time can help the success of an IT project with innovative components following the proposed process: a. Companies should map the gap in new knowledge versus current internal knowledge during a projectâ&#x20AC;&#x2122;s planning phase (i.e., management skills, resource planning and acquisition of people). b. Companies should train their internal project teams on new technologies during the processes of resource planning, and team development. c. Companies should plan the transfer of new knowledge to internal staff during the process of team development and contract closing, thus indicating the importance of formalizing this activity with suppliers.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

124 Tavares, E. da S., Pess么a, M. S. de P.

d. Companies should undertake a planning process for storing and spreading new knowledge during the process of team development. e. A project with innovative components must be planned and executed with much greater focus than in other types of projects in which technology is known and dominated by members of the internal project team. This focus is critical because the project manager should be concerned with balancing knowledge-transfer activities with the actual execution and delivery of the project. 3. Improving interactions with suppliers and using KM processes at project time have an important complement in creating an environment conducive to KM. 4. This environment is not complete in all cases studied. Most cases show that there are processes that facilitate KM, but that those processes are neither complete nor informal. Again, the only exception is the case of supplier B, which features an adequate process of creation, use and dissemination of knowledge using inclusive processes and IT tools. The result in Table 5 shows that the process of storing knowledge for future dissemination should be part of the PM processes during HR team development. QUESTION 2: What is the best way to transfer knowledge at project time? According to the survey, which involved a field of three client organizations and two suppliers, a proper form of knowledge transfer requires the steps set forth below. Moreover, the provided instruments of knowledge transfer for the cases studied confirmed these steps: 1. Either before or during the planning and execution of the project, personnel must be trained. 2. The project, especially new knowledge, must be documented. 3. There must be a mentoring process in which the supplier company, through its employees, transfers knowledge at project time, showing how the project is accomplished and then watching the professionals at the client company implement the project activities.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Technological innovation projects: proposal for an integrative model of project management and knowledge management perspective customer-supplier

125

Table 5 - Results of the vision of the use of the processes of KM processes in the PM Source: Authors.

FUTURE RESEARCH

Here are other related research opportunities that could complement our work: 1. A detailed study and criteria for qualifying and selecting suppliers of new technologies could be undertaken; 2. The project managerâ&#x20AC;&#x2122;s major challenge is to balance the concentration of efforts between a knowledge transfer of new technology and the implementation and delivery of the project itself, therefore, learning how to achieve an optimal balance between the two would be very interesting; 3. It would be interesting to search among companies of the same group (e.g., Matrix and subsidiaries) the transfer of technical knowledge in projects that have components of technological innovation.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

126 Tavares, E. da S., Pessôa, M. S. de P.

FINAL THOUGHTS

This article studies projects that incorporate some type of technological innovation that will bring new information or practices into an organization’s technology. This means that projects with this feature have a “hidden” objective: the transfer of knowledge from supplier to customer. A project’s success can be jeopardized if the client company does not appropriate this knowledge. The contribution of this research is to create a process of knowledge transfer embedded into the acquisition process at project time. This process is summarized in Figure 4, and Table 5 shows its grip in the cases studied. Although these results cannot be generalized due to their application to only some cases studied, the model can be adopted in situations similar to those shown. The model has three pillars: good customer and supplier relationships, knowledge transfer and an environment conducive to KM. REFERENCES Abell, Angela. Skills for knowledge environments. The Information Management Journal. July, 2000, p.33-41. Ansari, Mohammed Saleh Al, Open and Closed R&D Processes: Internal Versus External Knowledge, European Journal of Sustainable Development (2013), 2, 1, 1-18. Badal, Alen, Organizational Knowledge Management Movement Strategies, Journal of Knowledge Management, Economics and Information Technology, Vol. III, Issue 3 June 2013. Batra, Surinder Knowledge Management in Development Projects: A Case Study of HIDECOR Project in India Global Journal of Flexible Systems Management. Delhi: Jan-Jun 2007. Vol. 8, Iss.1/2; p. 55 (10 pages). Beijerse, Roelof P. Questions In Knowledge Management: Defining And Conceptualizing a Phenomenon. Journal of Knowledge Management. V. 3, p. 94-109, 1999. Blechar, M.; Loureiro, K.; Wallace L. Enterprise Information Architecture 1998.Gartner Group, Strategic Analysis, Report, p. 1-36, 1998. Bollinger, Andrey S.; Smith, Roberto D. Managing organizational knowledge as a strategic asset. Journal of Knowledge Management, p. 1-10, 2001. Byrd, Terry Anthony; Turner, Douglas E. An Exploratory Analysis of the Value of the Skills of IT personnel: Their Relationship to IS Infrastructure and Competitive Advantage. Decision Sciences, Vol. 12, Number 1, Winter, p. 21-54, 2001. Cleland, David I. The age of Project Management. Project Management Journal, p. 1924, March 1991. Cordenonsi, Jorge Luiz Um modelo de Administração da Tecnologia da Informação: Um estudo no setor Bancário Privado Brasileiro. São Paulo. Tese de Doutorado, Fundação Getúlio (não tem acento no nome da fundação) Vargas, São Paulo, 2001, 630p.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Technological innovation projects: proposal for an integrative model of project management and knowledge management perspective customer-supplier

127

Fagan, Mary Helen. Global information technology transfer: A framework for analysis. Journalof Global Information Technology Management; Marietta; 2001. FEBRABAN - Federação Brasileira dos Bancos. Dados sobre Tecnologia. São Paulo, 2003. Disponível na Internet emhttp://www.febraban.org.br/Arquivo/Servicos/Dadosdosetor/investimentos.asp. Acesso em:21 junho2003. Ferraresi, Alex Antonio; Santos, Silvio Aparecido dos; Frega, José Roberto; Pereira, Heitor José, Knowledge management, market orientation, innovativeness and organizational outcomes: a study on companies operating in brazil, JISTEM - Journal of Information Systems and Technology Management, Vol. 9, No. 1, Jan/Apr. 2012, pp.89-108. Frezatti, Fábio; Tavares, Edval da Silva. Análise da Decisão de Investimento em Sistemas Integrados de Informações: Possíveis Modelos e suas Influências no Processo Decisório. VI Congresso Brasileiro de Custos, São Paulo, 1999. Galvis-Lista, Ernesto, Sánches-Torres, Jenny Marcela, A critical review of knowledge management in software process reference models, JISTEM – Journal of Information Systems and Technology Management Revista de Gestão da Tecnologia e Sistemas de Informação Vol. 10, No. 2, May/Aug., 2013 pp.323-338. Gelabert, Carlos Macías, MSc; MARTINEZ, Allan Aguilera, PhD.,Contribución de la gestión de recursos humanos a la gestión del conocimiento, Estudos Gerenciales 28. 123 (Apr-Jun 2012) 133-148. Ireland, Paul. Satisficing dependent customers: on the power of suppliers in IT systems integration supply chains. Supply Chain Management, Bradford, 1999. Kamara, John M.; Anumba, Chimay J.; Carrillo, Patrícia M.A Clever approach to selecting a knowledge management strategy.International Journal of Project Management. 20, p. 205-211, 2002. Kamel, Sherif. Managing Globally With Infomation Technology. IRM Press, Cairo, 2003. Karhu, Katja. Expertise cycle - an advanced method for sharing expertise, Journal of Intellectual Capital. Vol. 3, n. 4, p. 40-446, 2002. Karlsson, Christer; Rajesh Nellore; KlasSöderquist. Black Box Engineering: Redefining the Role of Product Specifications. J. Prod. Innov.Manag, p. 534-549, 1998. Keil, Mark; Cule, Paul E.; Lyytinen, Kalle; Schimidt, Roy C. A Framework for Identifying Software Project Risks, Communications of the ACM, November 1998, v. 41 n 11, p. 76-83. Kern, T.; Willcocks, L. Exploring information technology outsourcing relationships: theory and practice. Journal of Strategic Information Systems, 9, p. 321-350, 2000. Landaeta, Rafael E Evaluating Benefits and Challenges of Knowledge Transfer Across Projects. Engineering Management Journal. Rolla: Mar 2008. Vol. 20, Iss.1; p. 29 (10 pages). Lim, Kwang K.; Pervaiz, K. Ahmed; ZAIRI, Mohamed. Managing for Quality through Knowledge Management, Total Quality Management, Vol. 10, Abingdon; Jul 1999.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

128 Tavares, E. da S., Pessôa, M. S. de P.

Marçula, Marcelo. Metodologia para a Gestão do Conhecimento em Pequenas e Médias Empresas, Apoiada pela Tecnologia da Informação. São Paulo. Dissertação (Mestrado) - Universidade Paulista. Departamento de Engenharia de Produção. São Paulo, 2001, 160p. Mcfarlan, W.E. Information Technology Changes The Way You Compete. Harvard Business Review, v.62, n.3, p.98-103, May/June 1984. Medina, José Manoel Cárdenas, Processos De Socialização E Sistemas De Gestão De Conhecimento Em Empresas Kibs, Tese de Doutorado, Universidade de São Paulo, Depto de Engenharia de Produção, 2013, 154p. Meredith, Jack R; Mantel, Samuel J. Project Management a Managerial Approach. John Wiley, New York, 1995. Myburgh, Sue. The Convergence of Information Technology Information Management, The Information Management Journal, p 4-16, April, 2000. NOLAN, R.; Mcfarlan, F.W. Information Technology and the Board of Directors. Harvard Business Review, October,2005. Nonaka, Ikujiro; TAKEUCHI, Hirotaka. Criação de Conhecimento na Empresa Como as Empresas Japonesas Geram a Dinâmica da Inovação. Rio de Janeiro: Campus, 1997. Pmbok – Um Guia do Conjunto de Conhecimento em Gerenciamento de Projetos. PMIProject Management Institute. PA, USA. 4ªedição. 2008. Prahalad, C.K. Managing Discontinuities: The Emerging Challenges Industrial Research Intitute, May-Jun 1998. Pressman, Roger S. Engenharia de Software. São Paulo: Makron Books, 1995. Probst, Gilbert; RAUB, Steffen; Romhardt, Kai. Managing Knowledge: building blocks for success. John Willer and Sons, Chichester, 2000. Quinn, James Brian. Strategic Outsourcing: Leveraging Knowledge Capabilities, Sloan Management Review, p. 9-21, Summer 1999. Silva, Glessia, Dacorso, Antonio Luiz Rocha, Inovação Aberta Como Uma Vantagem Competitiva Para A Micro E Pequena Empresa, Revista de Administração e Inovação, São Paulo, v. 10, n.3, p.251-268, jul./set. 2013. Souza, Yóris Linhares; VASCONCELOS, Maria Celeste Reis Lobo; JUDICE, Valéria Maria Martins; JAMIL, George Leal A Contribuição Do Compartilhamento Do Conhecimento Para O Gerenciamento De Riscos De Projetos: Um Estudo Da Indústria De Software, Revista da Gestão de Tecnologia da Informação, Vol7, No 1, 2010, p. 183-204. Steensma, H. Kevin; Corley, Kevin G. On the performance of technology-sourcing partnerships: The interaction between partner interdependence and technology attributes, Academy of Management Journal; Mississippi State; Dec 2000. Takeishi, Akira. Knowledge Partitioning in the Interfirm Division of Labor: The Case of Automotive Product Development, Organization Science, vol. 13, n.3, p. 321-338, MayJune, 2002. Tavares, Edval da Silva; Pessôa, Marcelo Schneck de Paula; Plonsky, Guilherme Ary. Implantação de Internet na Área Financeira: Um Estudo de Caso de Inovação. XIX Enegep, Rio de Janeiro, p. 1-19, 1999a.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Technological innovation projects: proposal for an integrative model of project management and knowledge management perspective customer-supplier

129

Tavares, Edval da Silva; Pessôa, Marcelo Schneck de Paula. Os Processos de Gerenciamento do Projeto: Implementação de um Sistema ERP. VI Simpep, Bauru, p. 1-8, Nov. 1999. Tidd, Joe; Bessant, John, Pavitt, Keith. Managing Innovation Integrating Technological, Market and Organizational Change, Wiley, New York, 1997. Tocan, Madalina Cristina, Knowledge Based Strategies for Knowledge Based Organizations, Journal of Knowledge Management, Economics and Information Technology, Vol 2, Iss 6, Pp 167-177 (2012). Torres-Freire, Carlos, Henriques, Frederico As Empresas olham além de seus muros para inovar?, Revista de Administração e Inovação, São Paulo, v. 10, n.3, p.143-164, jul./set. 2013. Tsang, Eric W.K. Acquiring Knowledge by Foreign Partners from International Economy: Learning-By-Doing and Learning Myopia. Strategic Management Journal, p. 835-854, May 2002. TSERNG, H. Ping; Lin, Pao H. An accelerated subcontracting and procuring Model for construction projects, Automation in Construction 11, p. 105-125, 2002. Yin, Roberto K. Estudo de Caso. Planejamento e Métodos. 2a.ed., Porto Alegre: Bookman, 2001. Zhijin Cui, Christoph H Loch, Bernd Grossmann, Ru He. Research Technology Management. Arlington: Nov/Dec 2009. Vol. 52, Iss.6; pg. 54, 10 pgs.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

130 Tavares, E. da S., Pess么a, M. S. de P.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

JISTEM – Journal of Information Systems and Technology Management Revista de Gestão da Tecnologia e Sistemas de Informação Vol. 11, No. 1, Jan/Apr., 2014 pp. 105-130 ISSN online: 1807-1775 DOI: 10.1590/S1807-17752014000100007

PROJETOS COM INOVAÇÃO TECNOLÓGICA: PROPOSTA DE UM MODELO INTEGRATIVO ENTRE GERÊNCIA DE PROJETOS E GERÊNCIA DO CONHECIMENTO NUMA PERSPECTIVA CLIENTE-FORNECEDOR Edval da Silva Tavares Escola Superior Nacional de Seguros, Funenseg, São Paulo, SP, Brazil Marcelo Schneck de Paula Pessoa University of São Paulo, POLI/USP, São Paulo, SP, Brazil ______________________________________________________________________ RESUMO Na área financeira, as empresas no Brasil têm aumentado seus investimentos em automação para enfrentar a forte concorrência, oferecer novos produtos e serviços e reduzir custos operacionais. Essas empresas estão concentrando seus esforços nas competências essenciais e, por isso, muitas vezes não têm o conhecimento interno para a implementação de novos projetos, especialmente aqueles que trazem inovação tecnológica. Por esta razão, é utilizado o processo de terceirização ou subcontratação para ajudar na implementação desses projetos. A unidade de análise deste estudo é o projeto e o objeto do estudo analisar o processo de transferência de conhecimento de um fornecedor para um cliente durante o período de duração do projeto envolvendo novas tecnologias. A principal motivação deste trabalho foi a de tratar da obtenção e retenção de novos conhecimentos relacionados a projetos que são elaborados pelas empresas clientes. Foi elaborado um estudo de casos múltiplos de três empresas do setor financeiro que compram novas tecnologias e dois fornecedores de Tecnologia da Informação. Como resultado prático um modelo de gestão de transferência de conhecimentos e de manutenção do conhecimento nas empresas clientes foi proposto e incorporado na gestão de projetos. Palavras-Chave: Gerência do Conhecimento, Gerência de Projetos, Inovação, Tecnologia da Informação.

_____________________________________________________________________________________________ Manuscript first received/Recebido em: 03/01/2013 Manuscript accepted/Aprovado em:11/02/2014 Address for correspondence / Endereço para correspondência Edval da Silva Tavares, Escola Superior Nacional de Seguros,Funenseg. Bachelor’s and Master’s in Business Administration, Feausp. Ph.D., Polytechnic School of USP, Department of Production Engineering. Adjunct Professor, Escola Superior Nacional de Seguros, Funenseg. Consultant, PricewaterhouseCoopers. GTI Researcher— Management in Information Technology, Department of Production Engineering, USP. Previously employed at Unibanco S/A, Fatec College of Technology SP, Vanzolini, Senac, Brasilprev, and Zurich Insurance. Address: Av Paulista, 2421 - 1st Floor, CEP 01311-30, Consolação, São Paulo, Brazil. Phone: +55 11 3062-2025. Email: est@terra.com.br Marcelo Schneck de Paula Pessoa, Polytechnic School of USP, Department of Production Engineering. Electronic Engineer, M.Sc. and Ph.D. Polytechnic School of USP. Associate Professor in Management of Information Technology, 2009. Professor and researcher, GTI, Management in Information Technology, Department of Production Engineering, USP. Have worked on projects related to computers, telephone exchanges, railway and industrial automation systems. Address: Av Almeida Prado travessa 2 n.128, CEP 05508-070, University City, São Paulo, Brazil. Phone: +55 11 3091-5363. Email: mpessoa@usp.br Published by/Publicadopor: TECSI FEA USP – 2014 All rights reserved.

106 Tavares, E. da S., Pessoa, M. S. de P.

INTRODUÇÃO

Este artigo aborda a questão da realização de projetos com inovação tecnológica em uma organização que está internalizando esses novos conhecimentos. A necessidade de inovação é decorrente da competitividade que obriga as organizações a investir em produtos e serviços com inovações para garantir sua sobrevivência (Tiddet al, 1997; Febraban, 2003). Produtos e serviços com inovações em um ambiente complexo de Tecnologia da Informação (TI) demandam projetos estruturados que podem ser desenvolvidos sob encomenda por terceiros (Blechar, 1998; Cleland, 1991; Pressman, 1995). O adequado relacionamento cliente-fornecedor tornou evidente a distinção entre as empresas mais inovadoras e dentre elas aquelas mais preocupadas em antecipar tendências do mercado e fazer inovação de caráter substantivo(Torres-Freire& Henriques, 2013). A necessidade de projetos estruturados demanda conhecimento de Gerência de Projetos (GP). Em função do foco nas competências essenciais (Quinn, 1999; Prahalad, 1998, Silva&Dacorso 2013) e da lacuna de conhecimento interno, a GP utiliza-se da subcontratação para trazer as inovações para a organização. Essa lacuna de conhecimento técnico é natural na organização antes da implementação das novas tecnologias em um projeto de TI e possui dois importantes fatores de risco no projeto: o risco do projeto em si e o risco da falta de conhecimento (Keil et al, 1998; Ansari,2013). Projetos de Integração de Sistemas são aqueles que necessitam se relacionar com diversos sistemas já existentes e possuem relativa complexidade. Esses projetos mostram-se importantes demandadores de novas tecnologias que, muitas vezes, carecem de conhecimento interno. Este artigo descreve uma pesquisa que focou projetos complexos que envolviam novas tecnologias desconhecidas pelo adquirente. Este foco é importante porque é nesse contexto que se caracteriza o problema abordado e a existência de uma lacuna de conhecimento técnico por parte da empresa cliente em relação ao conhecimento do fornecedor. A contribuição deste artigo é oferecer um modelo de transferência de tecnologia para resolver essa questão. Este artigo está organizado nos seguintes capítulos: Introdução, Aspectos Conceituais (inovação tecnológica, gerência do conhecimento e gerência de aquisição), Problema (caracterização do problema e modelo do problema), Agrupamento das Hipóteses (modelo proposto de transferência do conhecimento), Estudo de Casos (planejamento da pesquisa, desenvolvimento dos casos, passo a passo da execução do estudo e limitações da pesquisa),Resultados (consolidação dos resultados da pesquisa, análise de aderência ao modelo através da pontuação, a comparação de resultados entre empresas clientes e fornecedores, nível de interação entre o cliente e o processo de transferência de conhecimento do fornecedor, processo de transferência de conhecimento, ambiente propício à gerência do conhecimento e transferência do conhecimento: resultado das questões, Pesquisas Futuras e finalmente Considerações Finais. Vide resumo do artigo na Figura 1, abaixo.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Projetos com inovação tecnológica: proposta de um modelo integrativo entre gerência de projetos e gerência do conhecimento numa perspectiva cliente-fornecedor.

107

Figura 1 - Resumo do artigo Fonte: os autores. 2.

ASPECTOS CONCEITUAIS

Neste item é apresentada uma revisão bibliográfica referente à inovação, estratégia, gerência do conhecimento e gerência de aquisição. 2.1. Inovação tecnológica Inovação tecnológica pode ser realizada no produto, no serviço ou no processo e pode ser realizada de forma incremental, radical ou de transformação, (Tiddet al, 1997:7). Dessa forma, a inovação tecnológica trata de aspectos que não fazem parte da operação da empresa e precisa ser incorporada de forma economicamente viável. Existe uma necessidade de tratamento diferenciado entre projetos com inovação e projetos regulares (Takeishi, 2002:321-338). Para projetos regulares, com tecnologia conhecida, é mais importante o conhecimento arquitetural, ou seja, a visão geral dos componentes, ao passo que, quando o projeto envolve novas tecnologias, é importante que a organização tenha um alto nível de conhecimento do componente específico para resolver problemas de engenharia ainda inexplorados ou através da aquisição de soluções com inovações de TI (Ireland, 1999; Takeishi, 2002), subcontratando fornecedores com domínio desse conhecimento, conforme representado no Quadro 1 (Kamel, 2003; Steensma&Corley, 2000).

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

108 Tavares, E. da S., Pessoa, M. S. de P.

Tipo de

Divisão do

Montadoras

Fornecedores

projeto

modelo

deveriam ter

conhecidas)

(tecnologias

regurlares

Projetos

efetivo

Clara divisão

Conhecimento

Arquitetural

Específico do Componente

tecnologias)

(novas

inovação

Projeto com

Conhecimento

Superposição Arquitetural e

Arquitetural e

Específico do

Componente

Quadro 1 - Modelo sugerido para uma efetiva divisão do conhecimento (Takeishi,2002). O aspecto estratégico da TI também é analisado pelo Quadro 2, do universo de McFarlan (1984), que foi revisitado mais recentemente pelo mesmo autor em parceria com Nolan (NOLAN, 2005) em que as empresas podem ser posicionadas na matriz estratégica. No eixo horizontal do Quadro 2 está a necessidade de desenvolvimento de novas aplicações, seja por questões de mercado, seja por razões de incorporação rápida de novas tecnologias ou novas funcionalidades. No eixo vertical está a necessidade da organização utilizar uma TI confiável, ou seja, uma TI que, se falhar, prejudica diretamente os negócios. Esta divisão leva a quatro quadrantes, sendo o da extrema direita e acima denominado Estratégico onde se pode concluir que a importância da TI para a organização é estratégica. O quadrante Fábrica é para as empresas nas quais a TI precisa ser confiável, mas não há pressão para o desenvolvimento de novas aplicações.

Alto Baixo (presente)

confiável

Necessidade de TI

O quadrante Suporte trata de uma TI que não tem necessidade de novas aplicações nem precisa ser muito confiável, ou seja, tolera falhas sem prejuízo direto aos negócios. O quadrante Transição possui esse nome porque abriga as organizações que exigem novas aplicações, mas espera-se que, ao longo do tempo, evolua para o quadrante Fábrica ou Estratégico. TI defensiva

TI ofensiva

Fábrica

Estratégico

Suporte

Transição

Baixa

Alta

Necessidade de novas aplicações (futuro)

Quadro 2 - Os Quatro Quadrantes(NOLAN; McFARLAN,2005)

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Projetos com inovação tecnológica: proposta de um modelo integrativo entre gerência de projetos e gerência do conhecimento numa perspectiva cliente-fornecedor.

109

No caso dos bancos, foco da pesquisa deste trabalho, o enquadramento é no quadrante estratégico por causa da necessidade de continuamente desenvolver novas aplicações e da necessidade de uma TI altamente confiável. 2.2

Gerência do Conhecimento

Ao pesquisar sobre o que é conhecimento foi identificado que há dificuldades na sua conceituação. Assim como o próprio conhecimento, sua conceituação é fluida e subjetiva e neste trabalho será utilizada a conceituação definida por Nonaka&Takeuchi (1997), Myburgh (2000), Bollinger (2001), Beijerse (1999) e Probst (2000). O conhecimento abrange um escopo cujas fronteiras não são bem delimitadas. Tem sempre um objetivo, um alvo a ser atingido, uma ação, um significado, baseia-se em informações, está calcado em habilidades e competências, mas principalmente depende de crenças e atitudes das pessoas que o detêm. Portanto, está sempre ligado à ação humana, que se baseia e se correlaciona com um conjunto de informações, dando significado a elas com o objetivo de executar alguma ação. O conhecimento também é comparado a um iceberg, no qual a parte visível é o conhecimento que foi possível ser explicitado e a parte submersa o conhecimento tácito, conceitos estes presentes no processo de criação do conhecimento, denominado modelo SECI. Através do processo de socialização, o conhecimento tácito é passado de uma pessoa para outra. A externalização, ou seja, a estruturação e organização do conhecimento, o tornam explícito, que, em combinação com outros conhecimentos que a pessoa possui, leva a uma internalização e à consolidação de novos conhecimentos tácitos. Isso volta ao início do ciclo e pode gerar novamente mais conhecimentos (Marçula, 2001). Fagan (2001) baseou-se em estudos de adoção e implementação de TI que identificaram um grande número de fatores que podem impactar o processo de transferência de projetos de TI e os conhecimentos associados. Foram identificados e agrupados 23 fatores em cinco categorias: individual, organizacional, inovação, relacionadas a tarefas e fatores ambientais. Outra proposta de transferência de conhecimento é apresentada por Karhu (2002) através de um framework baseado na literatura e estudo de casos. O primeiro caso aborda a dificuldade de explicitação que os profissionais técnicos possuem e o segundo caso discute uma situação na qual a empresa possui um número elevado de informações, mas não as utiliza adequadamente no processo de tomada de decisão. Neste caso, o problema abordado diz respeito a: • Desafios na documentação de informações técnicas; • Conhecimento distribuído em diferentes países; e • Informação que é fundamental para a empresa. Para estes problemas, Karhu (2002) recomenda o uso do profissional chamado administrador do conhecimento. Esse profissional tem a função de descrever a experiência e o conhecimento de outras pessoas. O administrador do conhecimento também se dedica a adquirir informação na empresa, analisando a informação detalhadamente, formulando-a e documentando-a de forma que seja usada pelos pesquisadores de conhecimento. No processo de transferência do conhecimento, a técnica do “mentoring” é um método poderoso, mas é somente adequado quando o conhecimento não está distribuído entre muitos indivíduos e localidades diferentes. Para esse caso, as entrevistas e a observação mostram-se instrumentos adequados para este

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

110 Tavares, E. da S., Pessoa, M. S. de P.

tipo de transferência do conhecimento, com a ajuda do administrador do conhecimento. A confiança e o relacionamento pessoal são fatores básicos para resultados positivos na aquisição do conhecimento. Desta forma, o framework, obedece às seguintes etapas: 1. 2. escritas;

Entrevistas e observação do trabalho dos profissionais técnicos; Documentação do conhecimento obtido após análise, descrições e instruções

3. Processo de feedback com os profissionais para assegurar a qualidade do conteúdo; 4. Estes documentos são base para leitura e aprendizagem de outras pessoas, as quais poderão usar o que foi lido em suas experiências. Depois disto montarão seu próprio modelo mental ou seu conhecimento tácito. Outra visão de transferência de conhecimento relacionada ao conhecimento tático e explícito em projetos é demonstrada por Batra (2007) através de uma pesquisa utilizando conhecimento de um projeto para outro projeto. O argumento é que a Gerência do Conhecimento para o desenvolvimento exige uma mistura entre o conhecimento local, interno à empresa, o qual é em grande parte tácito, com o conhecimento externo, que é em grande parte explícito. O autor coloca que desta forma o conhecimento contextual do problema é mais profundamente compreendido e capturado antes das soluções de desenvolvimento do projeto serem implementadas. Uma pesquisa realizada com 46 projetos fechados mostrou que o corpo do conhecimento de projetos realizados a partir do conhecimento de outros projetos estava associada à performance dos projetos pesquisados. Apesar dos benefícios desta transferência de conhecimento, a pesquisa mostrou que existiu um aumento de investimentos por conta deste processo. No entanto, uma prática sugerida para limitar este impacto negativo do alto esforço de transferência de conhecimento é o de atribuir um número limitado de membros do projeto envolvidos neste esforço. Quanto aos métodos de transferência de conhecimento, os gerentes de projetos e organizações baseadas em projetos são recomendados a usar métodos formais, como reuniões, equipes especiais, revisões de projetos, mentoring, mensagens escritas, documentos e observações dos resultados de outros projetos, e métodos informais, por exemplo, interações entre os membros dos projetos, com ou sem utilização de recursos de multimídia, teleconferência, e-mails e bate-papos (Landaeta, 2008). Embora a inovação do outsourcing tenha se tornado bem disseminada no campo da pesquisa e desenvolvimento, os gerentes de projetos sempre querem saber quais práticas de gerência de projetos são necessárias para o desenvolvimento de novas tecnologias utilizando fornecedores (Zhijian & Loch, 2009). Uma análise de 24 projetos de desenvolvimento de tecnologias embrionárias da Siemens, utilizando diferentes fontes de outsourcing, identificou cinco fatores de sucesso em comum para estes projetos: confiabilidade, comunicação, forte competência do fornecedor, forte competência interna, definição clara do problema e alinhamento de interesses. Esses fatores podem ser resumidos na importância do nível de relacionamento entre a empresa cliente e o fornecedor. Um estudo mostra que em empresas de software existe um incentivo ao compartilhamento do conhecimento segundo 73,9% dos trabalhadores entrevistados, embora não exista um programa formal e estruturado de Gerência do Conhecimento (Souza et al, 2010).

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Projetos com inovação tecnológica: proposta de um modelo integrativo entre gerência de projetos e gerência do conhecimento numa perspectiva cliente-fornecedor.

111

O compartilhamento do conhecimento também é citado como uma das contribuições positivas da Gerência do Conhecimento em empresas orientadas ao mercado, um exemplo que foca em processos de negócio (Ferraresi et al, 2012). Os resultados de um estudo revelam a necessidade de facilitar a integração da gerência de recursos humanos para o gerenciamento de processos de conhecimento. Uma proposta seria tratar da formulação de políticas e estratégias destinadas a reforçar os recursos humanos, as principais atividades de seleção, treinamento e desenvolvimento, comunicação, trabalho de design de avaliação de desempenho e remuneração, de forma a encorajar a criação, o armazenamento, o compartilhamento e a aplicação de conhecimento (Gelabertet al, 2012). Medina (2012) conclui que o processo de transferência de conhecimento em empresas de serviços empresariais de conhecimento intensivo (KIBS) é facilitado pelo processo de socialização entre os especialistas utilizando os mecanismos de reuniões, nas quais são compartilhados relatos e histórias com problemas, treinamento, coaching, mentoring, shadowing, comunidade de prática e redes sociais virtuais. O processo de aquisição de conhecimento visa o ganho de conhecimento de várias fontes internas e externas à organização. Um dos princípios fundamentais das capacidades de transferência de conhecimento é a localização e aquisição de conhecimentos através da colaboração entre indivíduos e parceiros. A aquisição de conhecimento através de fornecedores como um dos itens das capacidades de transferência de conhecimento esteve presente em todos os modelos de referência do processo de software, segundo Galvis-Lista & Sánchez-Torres (2013). Resumindo, o processo de transferência do conhecimento, conforme os autores Karhu (2002), Marçula (2001), Fagan (2001), Tsang (2002), Batra (2007), Landaeta (2008), Zhijian&Loch (2009), Souza et al (2010), Medina (2012), Ferraresi (2012), Galvis-Lista & Sánchez-Torres (2013) pode contar com vários métodos de transferência e aquisição do conhecimento, sempre envolvendo a interferência humana. A complexidade em transformar o conhecimento tácito em explícito ou mesmo o conhecimento tácito em tácito é o grande desafio. 2.3Gerência de Aquisição A Gerência de Aquisição é uma das áreas de conhecimento da Gerência de Projetos que inclui os processos necessários para adquirir bens e serviços de outra empresa para complementar o escopo do projeto. Esta gerência “consiste em planejar, conduzir, administrar e encerrar as aquisições”, conforme PMBOK (2008). Existem vários níveis de subcontratação e eles podem contemplar desde a subcontratação do projeto como um todo até a simples demanda de programação dos componentes previamente definidos. Nesta pesquisa estão sendo focados projetos complexos nos quais o conhecimento fundamental pertence ao(s) fornecedor(es). Este foco é importante porque é dentro deste contexto que se caracteriza o problema abordado e a existência de uma lacuna de conhecimento técnico por parte da empresa cliente em relação ao conhecimento do fornecedor. Desta forma, o escopo deste estudo não prevê este caso de subcontratação simplesmente na etapa de implantação do projeto. Esta fase deve fazer parte também, mas não de forma isolada.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

112 Tavares, E. da S., Pessoa, M. S. de P.

PROBLEMA

Neste item o problema-foco da pesquisa é caracterizado e são apresentados dois modelos: o que caracteriza o problema e o de transferência de conhecimento. 3.1. Caracterização do problema Esta pesquisa teve por objetivo contribuir para os processos da Gerência de Projetos (GP) e incorporou aspectos da Gerência do Conhecimento (GC) tanto durante a execução do projeto como após seu término. Os processos da GP deverão ser complementados através da GC, tais como a melhoria de interação com os fornecedores, transferência de conhecimento e o ambiente propício à Gerência do Conhecimento. Portanto, o problema central está em definir como utilizar as práticas da GC dentro do contexto da GP e aprimorar os processos do desenvolvimento dos projetos de TI nas suas várias fases. Este processo de integração entre as áreas de conhecimento em que o novo conhecimento está baseado no conhecimento acumulado em outras áreas é citado por Lovejoy (1996:1-22), que mostra como a Gerência de Operações depende do relacionamento de várias disciplinas ou áreas de conhecimento. Nos projetos que contemplam inovação, pelo menos parte do novo conhecimento é considerado estratégico e, desta forma, precisa ser internalizado através da transferência da(s) empresa(s) fornecedora(s) para a organização adquirente. Esta transferência se justifica, pois as empresas precisam deste conhecimento interno, no mínimo para julgar a competência dos fornecedores e permitir o desenvolvimento de novos produtos ou serviços, uma vez que o tempo destes lançamentos tem se tornado cada vez menor. Desta forma, é fundamental estruturar processos adicionais para tratar desta transferência de conhecimento. A obtenção do conhecimento através do fornecedor também depende do bom relacionamento cliente-fornecedor (Karlsson, 1998). A busca e a implementação das soluções de TI não são suficientes em si mesmas, sem que ocorra a transferência do conhecimento do(s) fornecedor(es) para a empresa cliente. O processo da internalização do conhecimento é fundamental como base para a construção de novos projetos, assim como para julgamento dos fornecedores (experts) (PMBOK, 2008; Probst, 2000; Byrd & Turner, 2001. O argumento da internalização é reforçado principalmente quando se enquadra a empresa no quadrante estratégico do modelo de McFarlan no Quadro 2 ,(NOLAN, McFARLAN 2005; Cordenonsi, 2001). O relacionamento informal e, por vezes, conflituoso com os fornecedores pode levar ao não cumprimento dos objetivos do projeto, assim como à consequente falha na absorção de conhecimento. Constatamse também a importância e a viabilidade da internalização do conhecimento inexistente internamente e a transferência de novos conhecimentos. Pelo acima exposto, a implementação de práticas da Gerência do Conhecimento (GC) é muito importante, um vez que que as empresas se utilizam do conhecimento dos fornecedores para fazer frente à competitividade e inovação (Lim et al, 1999). 3.2. Modelo do problema Para melhor compreender o contexto da pesquisa realizada, foi desenvolvido um modelo conceitual apresentado na Figura 2.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Projetos com inovação tecnológica: proposta de um modelo integrativo entre gerência de projetos e gerência do conhecimento numa perspectiva cliente-fornecedor.

113

Novos produtos e serviços muitas vezes exigem o uso de novas tecnologias. A complexidade da tecnologia, a urgência de implementação e a necessidade de custos reduzidos demandam projetos estruturados. Estes projetos para a geração de produtos e serviços com componentes de inovação demandam novos conhecimentos (lado direito da Figura 2), que precisam ser internalizados em um patamar que proporcione o poder de julgamento em relação aos fornecedores de novas tecnologias e à construção de novos produtos e serviços. A área da GP que trata desta busca de soluções é a Aquisição. Nesta área, a opção pela subcontratação ou outsourcing justifica-se pela necessidade de a empresa focar-se nas suas competências essenciais. É importante ressaltar que o termo subcontratação estará sendo usado como forma de contratação de desenvolvimento de aplicações de TI em projetos complexos, com alto grau de integração, feitos sob medida. Dessa forma, quando as empresas buscam os novos conhecimentos através dos fornecedores, deveriam preocupar-se e atentar para a adequada transferência desses conhecimentos. Esse modelo analisa as dificuldades e as lacunas existentes neste processo e propicia investigar as seguintes questões: 1. Como aprimorar o modelo de aquisição dessas soluções dentro do contexto da Gerência de Projetos e transferência do conhecimento? 2. projeto?

Qual a melhor forma de transferência do conhecimento em tempo de

Figura 2 - Modelo do problema a ser analisado Fonte: os autores.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

114 Tavares, E. da S., Pessoa, M. S. de P.

AGRUPAMENTO DAS HIPÓTESES

Através da explicitação da conceituação que embasa o artigo e do problema a ser tratado, assim como das perguntas a serem respondidas, organizou-se a seguir o agrupamento das hipóteses. A Figura 3 resume os elementos da pesquisa realizada com a descrição do Problema (P1), as principais perguntas (Q1, Q2, Q3) e as hipóteses (H1 a H6). As hipóteses foram elaboradas através da revisão bibliográfica, sendo que as hipóteses H1, H2, H3 foram utilizadas para confirmar a amostra dos casos e as hipóteses H4, H5, H6 foram usadas para analisar a forma de transferência do conhecimento.

Figura 3 - Agrupamento das hipóteses Fonte: autores.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Projetos com inovação tecnológica: proposta de um modelo integrativo entre gerência de projetos e gerência do conhecimento numa perspectiva cliente-fornecedor.

115

Uma vez compreendido com clareza o problema a ser resolvido e quais os pontos a serem investigados, pode ser apresentado o modelo elaborado para representar a transferência de conhecimento descrito na Figura 4. O estudo mostrou, até o presente momento, que não foi encontrado na literatura um modelo específico que atenda ao problema proposto na Figura 2 ou que responda às perguntas propostas neste trabalho de forma específica. Desta forma, propôs-se um modelo baseado em outros modelos que atendiam parcialmente à proposta realizada. O modelo citado possui um núcleo que visa orientar a ação e as interações entre a empresa cliente e a empresa fornecedora. A empresa cliente possui uma série de atividades internas, representadas à esquerda. A empresa cliente possui projetos de TI que incorporam inovação tecnológica que, por estar procurando se manter focada no seu “core business”, realiza a subcontratação de fornecedores com o conhecimento necessário das inovações tecnológicas. Com relação à criação de ambiente propício à gerência do conhecimento, há a necessidade da criação de um processo de captura, organização, acesso e o uso do novo conhecimento. Também se deve buscar o uso de ferramentas de TI para o armazenamento e a disseminação do novo conhecimento. O planejamento deste processo deve estar contemplado no processo de gerenciamento do projeto. A empresa fornecedora, representada à direita, possui o domínio dessa tecnologia, o conhecimento técnico e o conhecimento a ser transferido. Com relação às interações entre as empresas, devem ser estabelecidas regras claras para a habilitação e o fornecimento desses conhecimentos e deve-se procurar um maior grau de formalização, além de existir um foco nos processos de transferência do novo conhecimento. No núcleo do modelo encontra-se o processo de transferência do conhecimento. Este processo é composto por atividade de mapeamento das lacunas de conhecimento, documentação do novo conhecimento, treinamentos prévios e processo de mentoring. Este modelo é composto, no seu núcleo, basicamente da proposta de Kamaraet al (2002), que apresenta um framework para selecionar a estratégia de GC que pode ser usada como modelo básico para identificar as necessidades de conhecimento, o conhecimento-alvo a ser transferido e finalmente a seleção do método de transferência do conhecimento. Este modelo também pode ser complementado com as visões de Landaeta (2008), Medina (2012), Ferraresi (2012), Galvis-Lista & SánchezTorres (2013), que abordam o processo de transferência do conhecimento.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

116 Tavares, E. da S., Pessoa, M. S. de P.

Figura 4 - Modelo proposto de transferência do conhecimento . Fonte: os autores. Sugere-se também que esta transferência não será efetiva sem que haja um bom relacionamento da empresa cliente com as empresas fornecedoras, que são

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Projetos com inovação tecnológica: proposta de um modelo integrativo entre gerência de projetos e gerência do conhecimento numa perspectiva cliente-fornecedor.

117

subcontratadas para elaboração dos projetos de TI. Esta afirmação é suportada pelos autores Tserng&Lin (2002), Kern&Willcocks (2000), Zhijian&Loch (2009) e TorresFreire & Henriques (2013), que afirmam ser a melhoria de relacionamento fator de sucesso para o processo de subcontratação. O processo de interações entre a empresa cliente e a empresa fornecedora também é fundamental neste modelo, mais precisamente no momento da escolha do fornecedor e/ou da solução técnica. Ele é importante para a empresa cliente que necessita de uma solução com inovação e ainda não aprendeu a julgar qual é a melhor ou mais adequada solução para seu problema de negócio. Nesse caso pode-se contar com um modelo de interação entre as empresas sintetizado por Frezatti& Tavares(1999). E, finalmente, conforme Abell (2000), Tocan (2012) e Badal (2013),é fundamental que se crie ou mantenha um ambiente propício para a GC. Portanto, sugere-se aqui que alguns processos da GP sejam cada vez mais agilizados e efetivos, além da melhoria do relacionamento com os fornecedores e a criação de um ambiente propício para a GC. 5.

ESTUDO DE CASOS

5.1

Planejamento da pesquisa

Para testar o modelo apresentado na Figura 4 foi considerado como método de pesquisa mais adequado para este tipo de investigação o estudo de casos, pois possibilitou a análise de diversas empresas, fornecedoras e clientes, com um grau de profundidade adequado e foco específico no fornecimento de novas tecnologias, conforme descrito a seguir e recomendado por Yin (2001). A coleta de dados foi desenvolvida através de entrevistas, o que possibilitou ao pesquisador ter um retrato mais preciso do ambiente a ser pesquisa do Foram realizadas perguntas fechadas, embora existissem também perguntas de caráter aberto para complementar as informações sobre os casos. Este formato de entrevista baseada em questionário, previamente elaborado e respondido pelos “informantes”, caracteriza a entrevista como “estruturada” ou como “um levantamento formal”. A entrevista pessoal possibilitou que os gerentes de projeto apresentassem suas interpretações de certos acontecimentos. Além disso, as entrevistas contaram com um fator complementar importante: a observação direta através da visita de campo. Para facilitar o correto encadeamento de evidências, buscar comparação adequada entre as informações dos entrevistados, ou mesmo comparar os casos entre si, utilizou-se o processo da escala Likert, que possibilitou medir e quantificar os resultados relacionados aos blocos mencionados no questionário. Dado que o objeto da pesquisa é o fluxo do conhecimento entre o fornecedor e o cliente, foram selecionadas três empresas clientes representadas por três grandes bancos, caracterizados pelo seu volume de ativos; e duas grandes empresas fornecedoras, caracterizadas pelo seu volume de faturamento. Por questões de sigilo, os nomes e a identificação das organizações em que foram realizados os estudos de caso foram omitidos. O Quadro 3 apresenta uma tabela-resumo contendo informações sobre as principais características dessas organizações.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

118 Tavares, E. da S., Pessoa, M. S. de P.

Principais números

Banco A

Banco B

Banco C

Fornecedor A

Volume de ativos

Número funcionários

US$ 65,1 bilhões

US$ 109,5 bilhões

US$ 41,6 bilhões

29.600

40.000

23.000

Faturamento anual

US$ 32 bilhões

Fornecedor B

US$ 86,4 bilhões

Quadro 3 - Informações numéricas das empresas Fonte: as empresas pesquisadas. A sequência da pesquisa foi a seguinte: • Pré-teste do questionário com gerentes de projeto que não participaram da pesquisa final para garantir a clareza das perguntas e a cobertura dos itens a serem pesquisados; • Envio do respectivo questionário para dois gerentes de cada empresa (cliente e fornecedor) que tivessem participado de projetos com componentes de inovação para a empresa; • Recepção e análise dos questionários; • Entrevista pessoal com cada um dos gerentes que responderam às pesquisas; Como para cada empresa foram entrevistados dois gerentes pode-se fazer uma verificação da similaridade de informações da empresa; • Telefonemas e visitas esporádicas para eventuais esclarecimentos de pontos específicos que não ficaram claros ou de pontos divergentes entre os dois gerentes da mesma empresa; • Todos estes passos levaram cerca de dois meses, computando-se 200 horas de trabalho total, mostrando-se desta forma o nível de profundidade dedicado ao estudo nesta pesquisa de campo. 5.2. Desenvolvimento dos casos Os casos foram desenvolvidos através de visitas a estas empresas e realizadas entrevistas com diversas pessoas em cada uma delas. Foi utilizado um questionário de apoio organizado a partir das hipóteses com uma série de perguntas a serem respondidas pelos entrevistados. A Figura 5 mostra a organização do questionário e do processo do estudo de caso: • Blocos 1 e 2 – informativo sobre a empresa pesquisada • Bloco 3 – caracterização dos processos de inovação • Bloco 4 – caracterização dos projetos de TI • Bloco 5 – nível de interação com o(s) fornecedor(es) • Bloco 6 – processo de transferência do conhecimento

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Projetos com inovação tecnológica: proposta de um modelo integrativo entre gerência de projetos e gerência do conhecimento numa perspectiva cliente-fornecedor.

119

• Bloco 7– ambiente propício à GC • Bloco 8– informações numéricas para dar a dimensão da importância dos casos • Processo de análise utilizado para consolidar os resultados de uma forma comparável, usando como parâmetro os modelos teóricos.

Figura 5 -Passo a passo da execução do estudo Fonte autores. A consolidação foi realizada comparando os resultados com a teoria e os casos entre si, ou seja, para cada linha do quadro, estão sendo comparados os resultados do caso cliente A, com o B e C, assim como com os fornecedores A e B. 5.3. Limitações da pesquisa Em razão do estudo de caso analisar situações específicas – no caso deste trabalho o processo de transferência de conhecimento em tempo de projeto do fornecedor de tecnologia para o cliente (a abordagem aqui focou a indústria financeira) – e adotar um grau de profundidade, não se pode generalizar seus resultados para qualquer outra situação ou empresas. Para que ocorra a replicação para outras empresas, é fundamental que elas apresentem similaridades com os casos pesquisados. Os itens a seguir apresentam o resultado obtido em cada um dos pontos mencionados na análise e organização dos resultados, conforme Figura 5.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

120 Tavares, E. da S., Pessoa, M. S. de P.

RESULTADOS

6.1

Consolidação dos resultados da pesquisa

Os blocos 3 e 4 (confirmação da amostra) dizem respeito à verificação se os casos tinham os pressupostos de projetos de TI com inovação tecnológica; e os blocos 5º ao 7º dizem respeito às hipóteses H4, H5 e H6. As correlações entre as hipóteses e as respostas às questões propostas são descritas nos próximo subitem do item 6, intitulado Resultados. 6.2

Análise de aderência ao modelo através da pontuação

O resultado da pontuação das perguntas fechadas, a análise das perguntas abertas, a observação do entrevistador e a análise das demais evidências durante as entrevistas permitiram a análise geral deste item. No entanto, é importante explicitar também o resultado numérico daquilo que se pôde quantificar, conforme Quadro 4, alinhado com as hipóteses mencionadas na Figura 3. Analisando as linhas do Quadro 4, observa-se que o único bloco totalmente aderente ao modelo para todos os casos é o da Caracterização dos Projetos de TI. Para os demais, existe uma variação de acordo com o caso. Os demais blocos variam na aderência de acordo com a empresa pesquisada. Observando-se a pontuação por caso, a empresa que mais se aproxima do modelo, restringindo-se a uma visão quantitativa, é a empresa fornecedora B. A análise dos resultados da pesquisa versus as hipóteses H4, H5 e H6 está detalhada dos itens 6.3 ao 6.6 deste capítulo.

Quadro 4 - Pontuação resultante da pesquisa de campo: casos de empresas clientes e das empresas fornecedoras Fonte: autores.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Projetos com inovação tecnológica: proposta de um modelo integrativo entre gerência de projetos e gerência do conhecimento numa perspectiva cliente-fornecedor.

121

6.3 Análise do resultado do nível de interação entre cliente e fornecedor versus Hipótese H4 A interpretação realizada neste item foi baseada na premissa de que deve haver uma melhoria do nível de interação entre as empresas fornecedoras e as empresas clientes. O resultado da pesquisa mostrou que as empresas possuem um nível de interação com seus fornecedores ou clientes em evolução e não totalmente aderente ao modelo, conforme os autores Tserng&Lin (2002, p. 107-108), Kern&Willcocks (2000, p. 328329), Zhijian&Loch (2009) e Torres-Freire & Henriques (2013), que afirmam ser a melhoria do relacionamento fator de sucesso para o processo de subcontratação. Embora a maioria dos gerentes entrevistados afirme que o nível de interação seja adequado, também reconhece que existem problemas como falta de regras claras de contratação e informalidade nos processos de habilitação de novos fornecedores ou subcontratação. A empresa cliente que mais se aproximou do modelo foi a empresa cliente C e a empresa fornecedora foi a empresa B. O destaque da empresa cliente C é que, além de existirem regras claras para homologação e concorrência entre os fornecedores, existem princípios de responsabilidade social que influenciam no relacionamento e na interação com os fornecedores. Do ponto de vista dos fornecedores foi comentado que há situações nas quais seus clientes estão despreparados tecnicamente, nem sempre o relacionamento é bom e muitas vezes problemas de projeto creditados aos fornecedores, em sua visão, não estão corretos. Outra colocação dos fornecedores é que as organizações clientes deveriam dar foco em projetos com componentes de inovação, principalmente no que se refere ao planejamento de tempo e recursos para absorver o novo conhecimento. Segundo os gerentes de projeto das empresas fornecedoras, os projetos com componentes de inovação não poderiam ser gerenciados da mesma forma que os projetos que utilizam tecnologias conhecidas pelas empresas clientes. Outro destaque foi o da empresa fornecedora B, que mostrou total aderência na busca da interação adequada com seus clientes, utilizando-se muito dos benefícios da GC nos projetos demandados por seus clientes, assim como através dos processos da GP. Embora sem uma aderência completa ao modelo no âmbito geral de todos os casos, observa-se, no caso da organização cliente C, no caso da empresa fornecedora B e no relato dos gerentes das demais empresas uma necessidade e uma tendência de busca do modelo proposto como fator de sucesso para a subcontratação e para o atingimento dos objetivos dos projetos. 6.4 Análise do resultado do processo de transferência do conhecimento versus Hipótese H5 A interpretação realizada neste item foi baseada na premissa de que a transferência do conhecimento em tempo de projeto pode ser composta dos seguintes instrumentos: treinamento, elaboração de documentação, mentoring e entrevistas. A primeira análise foi feita comparando-se os resultados com o modelo proposto e, em seguida, a comparação entre os casos, ou seja, conformidade e coincidência desses casos entre si. Outro aspecto considerado nesta interpretação foram as evidências relevantes e os aspectos mais significativos dos casos. O modelo traçado de transferência de conhecimento baseia-se nas pesquisas dos autores Nonaka&Takeuchi (1997, p. 68-69), Karhu (2002, p. 436), Fagan (2001, p. 5-26), Tsang (2002, p. 835854), Kamara et al, (2002, p. 205-211), Landaeta (2008), Medina (2013), Ferraresi

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

122 Tavares, E. da S., Pessoa, M. S. de P.

(2012) e Galvis-Lista & Sánchez-Torres (2013). De modo geral, estes autores tratam do processo, da forma e dos instrumentos para a transferência do conhecimento. As evidências consideradas relevantes e os itens mais significativos relacionam-se aos projetos pesquisados, os quais envolveram novas tecnologias para as respectivas empresas clientes no momento em que foram desenvolvidos. No caso dos itens a seguir, de e representaram projetos com inovação para as empresas clientes destes fornecedores. As empresas onde estes projetos foram executados, dada a “grande dependência de tecnologia” e ações de desenvolvimento, localizam-se no quadrante estratégico, segundo Nolan&Mcfarlan (2005), no Quadro 2. Estes projetos, analisados com base na citação de Blechar (1998, p. 6), envolveram mudanças nos ambientes da tecnologia instalada e/ou da arquitetura técnica e/ou da camada de aplicação. Mudanças estas que, em graus diferentes representaram uma inovação tecnológica para a empresa que a demandou. Em todos os casos pesquisados de empresas clientes e fornecedoras, o modelo de transferência de conhecimento foi constatado, embora em graus e formas diversas. Os seguintes pontos foram observados quando se consideraram os itens analisados de mapeamento, formas de conversão, instrumentos de transferência, formas de armazenamento, modalidades e nível de conhecimento transferido no âmbito de projeto: • O processo de transferência do conhecimento precisa evoluir se forem comparados os resultados da pesquisa com o modelo proposto, principalmente quanto ao processo da explicitação do conhecimento. Todos os gerentes de projeto pesquisados informaram que, após o processo de transferência do conhecimento, a forma mais consistente de conhecimento estava na modalidade de conhecimento tácito. Esta constatação mostra a fragilidade do processo da GC. Embora em muitos dos casos exista documentação dos projetos, há o reconhecimento de que a documentação foi feita “somente para constar”. O instrumento da socialização é importante, mas, quando somente este é utilizado, o processo da GC torna-se incompleto e as empresas continuam dependendo unicamente das pessoas para o prosseguimento e a utilização dos conhecimentos adquiridos. • Comparando-se os resultados, especificamente com o modelo apresentado por Kamara et al (2002), observa-se que existe um grupo considerado fonte de conhecimentos e outro alvo para receber os novos conhecimentos. Cada um destes grupos trabalha com três componentes: as pessoas, papéis e software. Ao comparar este modelo com os resultados dos casos pesquisados, observa-se que o grande foco foi o componente das pessoas, com exemplos isolados de papéis e software, como o caso da empresa fornecedora B. • Outro aspecto que mostra a necessidade de evolução do ambiente para GC é o fato de não ter sido constatado, nos casos pesquisados, exceto no fornecedor B, função formal de administrador ou integrador do conhecimento, conforme literatura de Marçula (2001, p. 130-134) e Karhu (2002, p. 430-446). • A evolução também pode ser analisada partindo-se do conhecimento “incorporado pelas pessoas” e evoluindo para o conhecimento “incorporado no capital” (Prahalad, 1998, p. 20). Usando-se esta visão pode-se afirmar, com base em todas as evidências, que no conhecimento dos casos analisados esta incorporação tende mais para o grupo do conhecimento “incorporado pelas pessoas” do que para o grupo do conhecimento “incorporado no capital”. Esta afirmação também está alinhada com a afirmação do item referente à GC, na qual se constata que o conhecimento mais

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Projetos com inovação tecnológica: proposta de um modelo integrativo entre gerência de projetos e gerência do conhecimento numa perspectiva cliente-fornecedor.

123

consistente, existente após as transferências de conhecimento, encontrava-se na modalidade do conhecimento tácito em todos os projetos pesquisados. • Nos casos pesquisados também foi constatada a necessidade de diferenciar-se na ênfase em determinados processos da GP, em projetos com componentes de inovação versus aqueles com ambiente de tecnologia conhecida. Isto ocorre na medida em que os gerentes de projeto pesquisados possuem uma missão dupla, ou seja, missão de internalizar o novo conhecimento e a missão de implementar o projeto em si. Ver Meredith&Mantel (1995) e Tavares&Pessôa (1999a). A aderência ao modelo mostrou-se na medida em que todos os gerentes de projeto admitiram a importância da transferência do conhecimento dos fornecedores envolvidos em projetos com novas tecnologias. Existiu também coincidência entre processos utilizados na transferência de conhecimento. No entanto, o processo de transferência utilizou-se, na maioria dos casos, de processos somente focados nas pessoas, utilizando a modalidade da socialização para a conversão de conhecimento. A evolução para o modelo proposto está espelhada nos processos levantados na empresa fornecedora B, que utiliza processos da GC nos da GP em seus projetos. 6.5 Análise do resultado do ambiente propício à gerência do conhecimento versus Hipótese H6 A interpretação realizada neste item foi baseada na premissa de que deve existir um ambiente propício à Gerência do Conhecimento. O resultado mostrou que, para as empresas clientes A, B e C e para a empresa fornecedora A, embora a maioria dos itens relacionados aos processos que propiciam a criação de um ambiente de Gerência de Conhecimento seja praticada, elas não possuem um processo explícito com este objetivo. Os processos que existem são importantes, mas informais. A única exceção neste item foi a empresa fornecedora B, pois todos os itens relacionados aos processos que propiciam a criação de um ambiente de Gerência de Conhecimento eram praticados nesta empresa e ela também possui um processo explícito com este objetivo. Na empresa fornecedora B, os processos que existem são importantes e formais. Nesta empresa, além dos processos que merecem destaque, há os processos de armazenamento do novo conhecimento, assim como sua disseminação com uso de ferramentas de TI, colocando o novo conhecimento à disposição de toda empresa e, em alguns casos, também à disposição de seus clientes. 6.6 Transferência do Conhecimento: resultado das questões que consolidam as Hipóteses H4, H5 e H6 Neste item são feitas considerações sobre as questões apresentadas no item 3.2, referente à apresentação do modelo do problema. Assim, pode-se obter alguma orientação sobre como resolver as duas questões apresentadas.

QUESTÃO 1: Como aprimorar o modelo de aquisição dessas soluções dentro do contexto da Gerência de Projetos e da transferência do conhecimento?

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

124 Tavares, E. da S., Pessoa, M. S. de P.

O aprimoramento no modelo de aquisição para subcontratação de projetos de TI com componentes de inovação tecnológica deve dar-se basicamente pelas seguintes medidas: • Melhoria de interação entre as empresas clientes e as empresas fornecedoras, através de processos de homologação e concorrências com regras claras e com oportunidades para todas as empresas fornecedoras. Embora esta realidade não seja completamente igual para todos os casos como no modelo traçado por Tserng&Lin (2002), Kern&Willcoks (2000), Zhijian&Loch (2009) e Torres-Freire & Henriques (2013)), observa-se que as empresas clientes buscam aprimorar este modelo e as empresas fornecedoras o apoiam totalmente. É importante ressaltar que a organização cliente C e a empresa fornecedora B têm processos muito próximos do modelo mencionado anteriormente. • O planejamento e a execução de alguns processos da Gerência de Conhecimento em tempo de projeto podem ajudar no êxito do projeto de TI com componentes de inovação. A seguir o processo proposto: • Mapeamento da lacuna do novo conhecimento versus o conhecimento interno atual durante a fase de planejamento do projeto (Gerência de Habilidades, Planejamento de Recursos e Aquisição de Pessoas). • Treinamento da equipe interna do projeto nas novas tecnologias durante os processos de Planejamento de Recursos e Desenvolvimento da Equipe. • Planejamento da transferência do novo conhecimento para a equipe interna nos processos de Desenvolvimento da Equipe e de Fechamento do Contrato, indicando a importância da formalização desta atividade com os fornecedores. • Planejamento de processo de armazenagem do novo conhecimento e futura disseminação durante o processo de Desenvolvimento da Equipe. • O projeto com componentes de inovação deve ser planejado e executado com muito maior foco do que os demais tipos de projetos em que a tecnologia é conhecida e dominada pelos membros da equipe interna do projeto. Este foco é fundamental, pois o gerente do projeto deverá preocupar-se em balancear as atividades de transferência do conhecimento com a execução e a entrega do projeto em si. • A melhoria da interação com os fornecedores e o uso de processos da Gerência do Conhecimento em tempo de projeto têm um complemento importante na criação de um ambiente propício para a Gerência do Conhecimento. • Este ambiente não se mostrou completo em todos os casos pesquisados. A maioria deles mostrou que existem processos que facilitam a Gerência do Conhecimento, porém não completos e de maneira informal. Novamente, a única exceção foi o caso da empresa fornecedora B, que possui um processo adequado de criação, uso e disseminação do conhecimento com uso inclusive de processos e ferramentas de TI para tal. O resultado do Quadro 5 mostra que o processo de armazenagem de conhecimento, para futura disseminação, deve fazer parte dos processos da GP durante o processo de RH de desenvolvimento da equipe. QUESTÃO 2:Qual a melhor forma de transferência do conhecimento em tempo de projeto?

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Projetos com inovação tecnológica: proposta de um modelo integrativo entre gerência de projetos e gerência do conhecimento numa perspectiva cliente-fornecedor.

125

A adequada forma de transferência do conhecimento pressupõe os passos a seguir, de acordo com o que mostrou a pesquisa de campo nas três organizações clientes e nas duas empresas fornecedoras. Dos instrumentos de transferência de conhecimento previstos, confirmaram-se os seguintes, para os casos pesquisados: 1.

Treinamento prévio ou durante o planejamento e execução do projeto.

Documentação do projeto e, principalmente, do novo conhecimento.

3. Processo de mentoring, no qual a empresa fornecedora, através de seus profissionais, transfere o conhecimento em tempo de projeto, mostrando como se faz e, em seguida, acompanhando os profissionais da empresa cliente na execução das atividades.

Quadro 5 - Resultado da visão da utilização dos processos da GC nos processos da GP Fonte: autores. 7.

PESQUISAS FUTURAS

Aqui estão relacionadas outras oportunidades de pesquisa que podem complementar o trabalho desenvolvido: • Um estudo detalhado e critérios para habilitação e escolha de fornecedores de novas tecnologias;

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

126 Tavares, E. da S., Pessoa, M. S. de P.

• O grande desafio do gerente do projeto está no balanceamento da concentração dos seus esforços entre a transferência de conhecimento de uma nova tecnologia e a implantação e entrega do projeto propriamente dito, portanto pesquisar como chegar a um nível ótimo deste balanceamento seria muito interessante; • Pesquisar a transferência de conhecimento técnico em projetos com componentes de inovação tecnológica entre empresas do mesmo grupo, por exemplo, matriz e filiais. 8.

CONSIDERAÇÕES FINAIS

Este artigo estudou a questão de projetos que incorporam algum tipo de inovação tecnológica, a qual deverá trazer práticas ou tecnologias desconhecidas no âmbito da organização. Isto significa que projetos com esta característica possuem um objetivo “escondido”: a transferência de conhecimento do fornecedor para o cliente. O sucesso do projeto pode ficar comprometido, se a empresa cliente não se apropriar deste conhecimento. A contribuição desta pesquisa foi a criação de um processo de transferência do conhecimento incorporado no processo de aquisição em tempo de projeto. Este processo está resumido na Figura 4 e o Quadro 5 mostra sua aderência nos casos estudados. Embora tais resultados não possam ser generalizados, em função de sua aplicação em alguns casos estudados, o modelo pode ser adotado em situações similares às apresentadas. Esse modelo está baseado em três vertentes: bom relacionamento cliente e fornecedor, processo de transferência de conhecimento e ambiente propício para a gerência do conhecimento. REFERENCES Abell, Angela. Skills for knowledge environments. The Information Management Journal. July, 2000, p.33-41. Ansari, Mohammed Saleh Al, Open and Closed R&D Processes: Internal Versus External Knowledge, European Journal of Sustainable Development (2013), 2, 1, 1-18. Badal, Alen, Organizational Knowledge Management Movement Strategies, Journal of Knowledge Management, Economics and Information Technology, Vol. III, Issue 3 June 2013. Batra, Surinder Knowledge Management in Development Projects: A Case Study of HIDECOR Project in India Global Journal of Flexible Systems Management. Delhi: Jan-Jun 2007. Vol. 8, Iss.1/2; p. 55 (10 pages). Beijerse, Roelof P. Questions In Knowledge Management: Defining And Conceptualizing a Phenomenon. Journal of Knowledge Management. V. 3, p. 94-109, 1999. Blechar, M.; Loureiro, K.; Wallace L. Enterprise Information Architecture 1998.Gartner Group, Strategic Analysis, Report, p. 1-36, 1998.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Projetos com inovação tecnológica: proposta de um modelo integrativo entre gerência de projetos e gerência do conhecimento numa perspectiva cliente-fornecedor.

127

Bollinger, Andrey S.; Smith, Roberto D. Managing organizational knowledge as a strategic asset. Journal of Knowledge Management, p. 1-10, 2001. Byrd, Terry Anthony; Turner, Douglas E. An Exploratory Analysis of the Value of the Skills of IT personnel: Their Relationship to IS Infrastructure and Competitive Advantage. Decision Sciences, Vol. 12, Number 1, Winter, p. 21-54, 2001. Cleland, David I. The age of Project Management. Project Management Journal, p. 1924, March 1991. Cordenonsi, Jorge Luiz Um modelo de Administração da Tecnologia da Informação: Um estudo no setor Bancário Privado Brasileiro. São Paulo. Tese de Doutorado, Fundação Getúlio (não tem acento no nome da fundação) Vargas, São Paulo, 2001, 630p. Fagan, Mary Helen. Global information technology transfer: A framework for analysis. Journalof Global Information Technology Management; Marietta; 2001. FEBRABAN - Federação Brasileira dos Bancos. Dados sobre Tecnologia. São Paulo, 2003. Disponível na Internet emhttp://www.febraban.org.br/Arquivo/Servicos/Dadosdosetor/investimentos.asp. Acesso em:21 junho2003. Ferraresi, Alex Antonio; Santos, Silvio Aparecido dos; Frega, José Roberto; Pereira, Heitor José, Knowledge management, market orientation, innovativeness and organizational outcomes: a study on companies operating in brazil, JISTEM - Journal of Information Systems and Technology Management, Vol. 9, No. 1, Jan/Apr. 2012, pp.89-108. Frezatti, Fábio; Tavares, Edval da Silva. Análise da Decisão de Investimento em Sistemas Integrados de Informações: Possíveis Modelos e suas Influências no Processo Decisório. VI Congresso Brasileiro de Custos, São Paulo, 1999. Galvis-Lista, Ernesto, Sánches-Torres, Jenny Marcela, A critical review of knowledge management in software process reference models, JISTEM – Journal of Information Systems and Technology Management Revista de Gestão da Tecnologia e Sistemas de Informação Vol. 10, No. 2, May/Aug., 2013 pp.323-338. Gelabert, Carlos Macías, MSc; MARTINEZ, Allan Aguilera, PhD., Contribución de la gestión de recursos humanos a la gestión del conocimiento, Estudos Gerenciales 28. 123 (Apr-Jun 2012) 133-148. Ireland, Paul. Satisficing dependent customers: on the power of suppliers in IT systems integration supply chains. Supply Chain Management, Bradford, 1999. Kamara, John M.; Anumba, Chimay J.; Carrillo, Patrícia M.A Clever approach to selecting a knowledge management strategy. International Journal of Project Management. 20, p. 205-211, 2002. Kamel, Sherif. Managing Globally With Infomation Technology. IRM Press, Cairo, 2003. Karhu, Katja. Expertise cycle - an advanced method for sharing expertise, Journal of Intellectual Capital. Vol. 3, n. 4, p. 40-446, 2002. Karlsson, Christer; Rajesh Nellore; KlasSöderquist. Black Box Engineering: Redefining the Role of Product Specifications. J. Prod. Innov.Manag, p. 534-549, 1998.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

128 Tavares, E. da S., Pessoa, M. S. de P.

Keil, Mark; Cule, Paul E.; Lyytinen, Kalle; Schimidt, Roy C. A Framework for Identifying Software Project Risks, Communications of the ACM, November 1998, v. 41 n 11, p. 76-83. Kern, T.; Willcocks, L. Exploring information technology outsourcing relationships: theory and practice. Journal of Strategic Information Systems, 9, p. 321-350, 2000. Landaeta, Rafael E Evaluating Benefits and Challenges of Knowledge Transfer Across Projects. Engineering Management Journal. Rolla: Mar 2008. Vol. 20, Iss.1; p. 29 (10 pages). Lim, Kwang K.; Pervaiz, K. Ahmed; ZAIRI, Mohamed. Managing for Quality through Knowledge Management, Total Quality Management, Vol. 10, Abingdon; Jul 1999. Marçula, Marcelo. Metodologia para a Gestão do Conhecimento em Pequenas e Médias Empresas, Apoiada pela Tecnologia da Informação. São Paulo. Dissertação (Mestrado) - Universidade Paulista. Departamento de Engenharia de Produção. São Paulo, 2001, 160p. Mcfarlan, W.E. Information Technology Changes The Way You Compete. Harvard Business Review, v.62, n.3, p.98-103, May/June 1984. Medina, José Manoel Cárdenas, Processos De Socialização E Sistemas De Gestão De Conhecimento Em Empresas Kibs, Tese de Doutorado, Universidade de São Paulo, Depto de Engenharia de Produção, 2013, 154p. Meredith, Jack R; Mantel, Samuel J. Project Management a Managerial Approach. John Wiley, New York, 1995. Myburgh, Sue. The Convergence of Information Technology Information Management, The Information Management Journal, p 4-16, April, 2000. NOLAN, R.; Mcfarlan, F.W. Information Technology and the Board of Directors. Harvard Business Review, October,2005. Nonaka, Ikujiro; TAKEUCHI, Hirotaka. Criação de Conhecimento na Empresa Como as Empresas Japonesas Geram a Dinâmica da Inovação. Rio de Janeiro: Campus, 1997. Pmbok – Um Guia do Conjunto de Conhecimento em Gerenciamento de Projetos. PMIProject Management Institute. PA, USA. 4ªedição. 2008. Prahalad, C.K. Managing Discontinuities: The Emerging Challenges Industrial Research Intitute, May-Jun 1998. Pressman, Roger S. Engenharia de Software. São Paulo: Makron Books, 1995. Probst, Gilbert; RAUB, Steffen; Romhardt, Kai. Managing Knowledge: building blocks for success. John Willer and Sons, Chichester, 2000. Quinn, James Brian. Strategic Outsourcing: Leveraging Knowledge Capabilities, Sloan Management Review, p. 9-21, Summer 1999. Silva, Glessia, Dacorso, Antonio Luiz Rocha, Inovação Aberta Como Uma Vantagem Competitiva Para A Micro E Pequena Empresa, Revista de Administração e Inovação, São Paulo, v. 10, n.3, p.251-268, jul./set. 2013. Souza, Yóris Linhares; VASCONCELOS, Maria Celeste Reis Lobo; JUDICE, Valéria Maria Martins; JAMIL, George Leal A Contribuição Do Compartilhamento Do Conhecimento Para O Gerenciamento De Riscos De Projetos: Um Estudo Da Indústria

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

Projetos com inovação tecnológica: proposta de um modelo integrativo entre gerência de projetos e gerência do conhecimento numa perspectiva cliente-fornecedor.

129

De Software, Revista da Gestão de Tecnologia da Informação, Vol7, No 1, 2010, p. 183-204. Steensma, H. Kevin; Corley, Kevin G. On the performance of technology-sourcing partnerships: The interaction between partner interdependence and technology attributes, Academy of Management Journal; Mississippi State; Dec 2000. Takeishi, Akira. Knowledge Partitioning in the Interfirm Division of Labor: The Case of Automotive Product Development, Organization Science, vol. 13, n.3, p. 321-338, May-June, 2002. Tavares, Edval da Silva; Pessôa, Marcelo Schneck de Paula; Plonsky, Guilherme Ary. Implantação de Internet na Área Financeira: Um Estudo de Caso de Inovação. XIX Enegep, Rio de Janeiro, p. 1-19, 1999a. Tavares, Edval da Silva; Pessôa, Marcelo Schneck de Paula. Os Processos de Gerenciamento do Projeto: Implementação de um Sistema ERP. VI Simpep, Bauru, p. 1-8, Nov. 1999. Tidd, Joe; Bessant, John, Pavitt, Keith. Managing Innovation Integrating Technological, Market and Organizational Change, Wiley, New York, 1997. Tocan, Madalina Cristina, Knowledge Based Strategies for Knowledge Based Organizations, Journal of Knowledge Management, Economics and Information Technology, Vol 2, Iss 6, Pp 167-177 (2012). Torres-Freire, Carlos, Henriques, Frederico As Empresas olham além de seus muros para inovar?, Revista de Administração e Inovação, São Paulo, v. 10, n.3, p.143-164, jul./set. 2013. Tsang, Eric W.K. Acquiring Knowledge by Foreign Partners from International Economy: Learning-By-Doing and Learning Myopia. Strategic Management Journal, p. 835-854, May 2002. TSERNG, H. Ping; Lin, Pao H. An accelerated subcontracting and procuring Model for construction projects, Automation in Construction 11, p. 105-125, 2002. Yin, Roberto K. Estudo de Caso. Planejamento e Métodos. 2a.ed., Porto Alegre: Bookman, 2001. Zhijin Cui, Christoph H Loch, Bernd Grossmann, Ru He. Research Technology Management. Arlington: Nov/Dec 2009. Vol. 52, Iss.6; pg. 54, 10 pgs.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

130 Tavares, E. da S., Pessoa, M. S. de P.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 105-130

www.jistem.fea.usp.br

JISTEM - Journal of Information Systems and Technology Management Revista de Gestão da Tecnologia e Sistemas de Informação Vol. 11, No. 1, Jan/Apr., 2014 pp. 131-152 ISSN online: 1807-1775 DOI: 10.1590/S1807-17752014000100008

METHOD FOR MEASURING THE ALIGNMENT BETWEEN INFORMATION TECHNOLOGY STRATEGIC PLANNING AND ACTIONS OF INFORMATION TECHNOLOGY GOVERNANCE Lúcio Melre da Silva João Souza Neto Catholic University of Brasilia, Brasília, Distrito Federal, Brazil ______________________________________________________________________ ABSTRACT The purpose of this research is to present a method for measuring the degree of alignment between Strategic Planning and Information Technology Management practices and Information Technology Governance. A survey of IT governance maturity at the High Courts and the Supreme Court was carried out in order to reach this aim. The Attribute Table of the COBIT 4.1 was used both as a model for maturity analysis as for the degree of alignment of IT strategic plans of these bodies with the IT Strategic Planning established by the National Judiciary Council (CNJ). It was assessed the maturity of thirty four processes, according to six attributes, in the four COBIT domains. The proposed method, named COMPLAN-GTI, allows the linking of the guidelines of the strategic planning to the COBIT processes. The field research above mentioned shows that the alignment between the planning established by the CNJ and those established by the High Courts and Supreme Court is around 68%, leading to the conclusion that the policies and actions established by the National Council of Justice for the Judiciary are being followed. The application of the method is also used to confirm whether the management practices and the IT Governance are consistent with the strategic plan established by the organization. It was observed in the research carried out in the Courts that the average convergence between PETIs and management practices and Governance lies around 70%, leading to the conclusion that the strategic plans exerted influence on the action planning of these organizations. Keywords: Information Technology Governance; Strategic Planning; COBIT.

_____________________________________________________________________________________________ Manuscript first received/Recebido em: 11/06/2012 Manuscript accepted/Aprovado em:09/07/2013 Address for correspondence / Endereço para correspondência Lúcio Melre da Silva. Master in Information Technology and Knowledge Management – MGCGI/UCB, Catholic University of Brasilia, Researcher at Catholic University of Brasilia in Metamodels of IT frameworks, Campus Avançado, SGAN 916 Asa Norte - Modulo B - Sala A111 - CEP: 70.790-160 Brasília – DF, Brasil - Telephone: (61) 8109-4140 - E-mail: lucio.melre@gmail.com. João Souza Neto, Doctor of Science in Electrical Engineering, University of Brasilia – UNB, Professor at Catholic University of Brasilia, on the Master’s degree Program in Information Technology and Knowledge Management, Campus Avançado, SGAN 916 Asa Norte - Modulo B - Sala A121 - CEP:70.790-160 Brasília – DF, Brasil Telephone: (61) 3448-6534 - E-mail: joaon@ucb.br. Published by/ Publicado por: TECSI FEA USP – 2014 All rights reserved.

132 Silva, L. M. Da, Neto, J. S.

INTRODUCTION

The Brazilian Constitution, in its Second Article, states that the Legislative, the Executive and the Judiciary are branches of the government, independent and harmonious among themselves. Chapter III deals specifically with the Judiciary, and determines its composition, principles, the responsibilities of the several agencies that comprise it, as well as the guarantees of the Judiciary members and the Judiciary administrative and financial autonomy. The Judiciary is constituted by the Supreme Court, the Superior Court, the Federal Regional Courts and their Federal Judges, Labor, Electoral and Military Courts and their respective Judges and the Courts and Judges of the States. The Constitutional Amendment No. 45, dated December 30th, 2004, laid the foundations for the accomplishment of a comprehensive reform in the Brazilian Judiciary. One of the innovations introduced by the reform of the Brazilian Constitution was the creation of the National Council of Justice – CNJ (Conselho Nacional de Justiça), whose mission is to contribute to a decision made with morality, efficiency and effectiveness, for the benefit of justice. The CNJ is an agency focused on the reformulation of the Judiciary’s officials and the judicial procedures, especially regarding the administrative and procedural control and transparency, and it was set up in conformance with the Federal Constitution, in particular pursuant to article 103-B. It has a Standing Committee of Information Technology and Infrastructure, which proposes actions to:  Implement adequate infrastructure for the intended operation of the Judiciary;  Create the Information Technology Strategic Planning to ensure the appropriate technology for the proper performance of the activities of the Courts, the interoperability between different systems, and the improvement and implementation of the electronic judicial process, and 

Deploy the electronic judicial process.

In order to assist the CNJ in the Information Technology (IT) management and Governance activities, it was created a committee which has done relevant work on the Information Technology Strategic Planning for the Judiciary. This committee was approved by the Resolution No. 99 of November 24 th, 2009, which established its mission, vision and attributes as well as the strategic objectives to be achieved by the Judiciary. This research was conducted at the Superior Courts and at the Supreme Court and, although the Supreme Court it is not obliged to follow the recommendations of the CNJ, it has adopted all the practices related to the IT management and Governance. The Supreme Court is also a member of the National Committee of Information Technology and Communications of the Judiciary.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

Method for Measuring the Alignment between Information Technology Strategic Planning and Actions of Information Technology Governance

133

THEORETICAL REFERENCE

2.1. Strategic Planning In 2011 a research called Global Status Report on the Governance of Enterprise IT was conducted by ITGI. The research aimed to identify trends regarding the importance of the Information Technology and more than 800 IT executives from organizations located in twenty-one countries participated. The same survey was conducted in 2004, 2006 and 2008, in order to identify the growth or decline in the trends. This research showed that the vast majority of respondents see IT as an issue to the business strategies, thus confirming that IT increases the competitiveness of enterprises. (ITGI, 2011). Strategic planning is critical to the survival of public and private organizations, since it establishes a guideline for the actions that should be followed by all units of the organization, aiming at the achievement of enterprise targets. It is a dynamic, systemic, participatory and collective process used to determine the goals, strategies and actions of the organization. The process starts with the identification of the problems within the organization. (REZENDE, 2011). In order to meet its goals and objectives it is crucial for an organization to have its Information Technology projects and activities aligned with the demands and needs of the business. Some authors, as Henderson and Venkatraman (1993), have discussed the importance of the Information Technology for streamlining the activities of the organization, stating that this role requires the deployment of an efficient IT platform (including hardware, software and communication systems) for the management and control of all processes. However, three scenarios are still observed in the relationship between IT and the business: 1. Focus on operational services and infrastructure of the organization; 2. Delivery of the IT solutions and support to enterprise strategies, without participation in strategy making; 3. Full integration of the IT activities with the business objectives and strategic goals of the company. Scenario 3, which is considered the most suitable one for the delivery of the desired business value, can be achieved with the adoption of an IT strategic planning (PETI â&#x20AC;&#x201C; Plano EstratĂŠgico de TI), which should be aligned and integrated with the institutional strategic planning. PETI is a dynamic and iterative process that defines, in a strategic level, the organizational information, the IT resources (hardware, software, data and information management, and information systems), the people involved in the process and the infrastructure necessary to meet all the goals and objectives established by the organization. (REZENDE, 2011).

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

134 Silva, L. M. Da, Neto, J. S.

2.2. Information Technology Governance The Governance of Information Technology is part of the Corporate Governance and it consists of the leadership, the organizational structures and the processes that ensure the IT organization to sustain and extend the organization's strategies and goals, based on the guidelines of the strategic planning. It is necessary to discuss the Corporate Governance, since all definitions of the Information Technology Governance are directly or indirectly related to it. The Code of Best Practices for Corporate Governance, published by the Instituto Brasileiro de Governanรงa Corporativa, (Brazilian Institute of Corporate Governance) states that Corporate Governance is the system by which organizations are directed, monitored and encouraged, involving the relationships among the owners, the board of directors, the management and the control bodies. Good Corporate Governance practices translate principles into objective recommendations, aligning interests in order to enhance and preserve the value of the organization, facilitating its access to the resources and contributing to its longevity. (IBGC, 2009). This is not a new subject, but it deserved special mention in the press after the financial scandals at the beginning of this century, when important U.S. companies such as Enron, WorldCom and Tyco led thousands of customers into bankruptcy due to accounting manipulations and financial disruptions. Weill and Ross (2006) proposed a framework to link corporate governance to IT governance. There are two groups in this framework. The first group describes the relationships of the board with the shareholders and other stakeholders. The senior executive team acting as an agent of the board is responsible for articulating strategies and behaviors to carry out the directions of the board. The other group encompasses the seven main assets (human, financial, physical, intellectual property, relationship, information, and IT), and, through them, the companies accomplish their strategies and generate business value. According to these authors, companies with common mechanisms for various assets present a better performance. IT governance involves many aspects related to the practices established and consolidated in the market. IT governance development strategies should take into account aspects related to the available resources, the structure and the business of each organization. The proper goal setting and decision making should result in benefits for the organization. In order to be considered effective, and according to Weill and Ross (2006), IT governance must answer three questions: 1. Which IT decisions must be made to ensure the effective management and use of IT? 2. Who should make such decisions? 3. How will these decisions be made and monitored? To answer the first two questions, the authors proposed an array of governance arrangements, that relates five IT key decisions (IT principles, IT architecture, IT infrastructure, needs for business applications and IT investment and prioritization) to

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

Method for Measuring the Alignment between Information Technology Strategic Planning and Actions of Information Technology Governance

135

seven organizational archetypes (business monarchy, IT monarchy, feudalism, federalism, duopoly, and anarchy). The IT key decisions concern to the major decisions to be taken in the domain of the IT governance, while the archetypes typify the decision makers. The IT principles clarify the business role of IT; the architecture defines the requirements for integration and standardization; the infrastructure determines the shared services and the support services; the need for business applications specify the business needs for IT applications, which were acquired or developed internally, and the investments and prioritization of IT indicate which initiatives to fund and how much to spend on them. Regarding the archetypes, the responsibilities of those who make the decisions are passed on to the senior management in business monarchy to IT managers in IT monarchy, to the managers of the business units in feudalism, to the headquarters and branches managers in federalism. In the case of IT duopoly, IT managers and some other group are the decision makers and, finally, in anarchy, the decisions are made individually or in small groups. The ITGI (2007) defines the following focus areas in the IT Governance: 1. Strategic Alignment: align the IT operations with the organizationâ&#x20AC;&#x2122;s strategic objectives. It is responsible for ensuring alignment and prioritization of projects based on the strategic goals of the organization; 2. Value Delivery: ensures that IT delivers to the business the benefits foreseen in the IT strategy. It is responsible for the cost optimization and the provision of the IT intrinsic value; 3. Resource Management: focuses on the better use of investments and on the appropriate management of the critical IT resources: applications, information, infrastructure and people; 4. Risk Management: emphasizes a clear understanding of the organizationâ&#x20AC;&#x2122;s appetite for risk as well as the compliance requirements, the transparency about the significant risks to the organization and the inclusion of risk management into the routine activities; 5. Performance Measurement: tracks and follows up the implementation of the strategies, the progress of projects, the use of the resources, the delivery and the support services performance. In the public sector some difficulties arise when it adopts the frameworks developed specifically for private companies. Therefore, Weill and Ross (2006) proposed a specific framework for nonprofits organizations, being categorized as government organizations which include defense, immigration, public services, police, education and health, as well as NGOs. According to the authors, four major challenges of nonprofit organizations were identified: measurement of value and performance, investments in IT infrastructure, coproduction and architectures, and citizens, clients and buyers. The measurement of value is difficult to implement because some factors like profit or cost reduction are not involved. Thus, the measurement of value should be

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

136 Silva, L. M. Da, Neto, J. S.

made taking into account other parameters such as the customer satisfaction, the quality of the product delivered and the quality of the support to the customer in the public sector, the customer is the citizen). According to the authors, the investments in infrastructure can be justified in three ways: by holding office, i.e., without the need to submit justification; by expense reduction, that can be quantified and assessed; and by the enablement of new capabilities. Co-production refers to the capacity of nonprofit organizations to encourage or compel the co-producers to commit to creating public value to a wider audience. Such capacity can bring direct benefits to the citizens as much as it allows reducing deadlines and anticipating the delivery of services to the society. Citizens, customers and buyers should be identified in the provision of services by nonprofit organizations, since the treatment devoted to each one of them may be different due to the goals to be achieved. The identification of these four major challenges, in association with others, influences how the organization implements its IT governance, thus justifying the importance of this identification. Research developed by Xavier (2010) in the Federal Public Administration agencies concluded that the COBIT 速 (Control Objectives for Information and related Technology) may serve as reference for the implementation of improvements aiming at the establishment of IT goals and indicators. Also, it allows monitoring the evolution of the IT governance maturity level in the agencies of the Brazilian Public Sector. 2.2.1 The COBIT Framework The COBIT framework was developed by ITGI and the current version is number 5. This version was published in mid-April 2012 and it is a significant update to COBIT. However, the previous version, the 4.1, used in this study, still has wide acceptance due to its large knowledge base application. COBIT 4.1 provides best practices for IT Governance using a model which consists of domains, processes and activities presented in a manageable and logical structure (ITGI, 2007). It provides a framework to manage and control IT activities and presents five key characteristics: focus on business, process orientation, overall acceptability, compliance requirements and common language. The COBIT framework is based on the premise that IT has to deliver the information required by the business to help it achieve its goals. It provides a framework and a guide to implement IT governance, allowing the prioritization of IT processes that should be improved. The model combines the business requirements for information with the objectives of the IT function. Thus, the basic principle of the COBIT framework can be summarized as IT resources which are managed by IT processes to achieve IT goals which, in turn, respond to business requirements (ITGI, 2007). This principle is illustrated in Figure 1, in the COBIT cube. In each dimension of the cube, in this figure, are shown the IT processes (thirty-four, divided into four domains), the IT resources (applications, information, infrastructure and people) and the business requirements (effectiveness,

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

Method for Measuring the Alignment between Information Technology Strategic Planning and Actions of Information Technology Governance

137

efficiency, confidentiality, integrity, availability, compliance and reliability of information).

Figure 1 â&#x20AC;&#x201C; The COBIT Cube Source: (ITGI, 2007, p. 27)

IT processes are organized into four areas and divided into two hundred and ten activities. Each domain has its own set of control objectives and scope. They are the following: Plan and Organize (PO), Acquire and Implement (AI), Deliver and Support (DS) and Monitor and Evaluate (ME). The maturity analysis of each COBIT process allows the organization to identify its current stage, the current state of the market (benchmarking), the maturity desired and the path to be traversed to go from the current situation to the future one. Each COBIT process is rated on a generic and complex scale, requiring a complete and systemic view of the organization (XAVIER, 2010). The scale used is shown in Table 1 that describes the general characteristics of the degrees of maturity. Table 1 - Maturity Scale of COBIT 4.1 0 - Non-Existent

Complete lack of a recognized process. The organization does not even acknowledge that there is an issue to be worked on.

1 - Initial / Ad hoc

There is evidence that the organization has recognized that there are issues which need to be addressed. However, there is no standardized process, but rather, an ad hoc approach which tends to be applied individually or on a case-by-case basis. The general approach to management is disorganized.

2 - Repeatable but Intuitive

Processes have evolved to a stage where similar procedures are followed by different people doing the same task. There is no formal training or communication of standard procedures, and the responsibility is left to the individuals. There is a high degree of confidence in the individualsâ&#x20AC;&#x2122; knowledge , and, consequently, errors may occur.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

138 Silva, L. M. Da, Neto, J. S.

3 - Defined

Procedures were standardized, documented and communicated through training. It is mandatory that these processes are followed up. However, there is a possibility that deviations will not be detected. The procedures are not sophisticated, but there is a formalization of existing practices.

4 - Managed and Measurable

The management monitors and measures adherence to the procedures and takes action when the processes are not working well. The processes are constantly improved and, thus, they provide good practices. The use of automation and tools is limited or fragmented.

5 - Optimized

The processes have been refined to a level of good practice and are continually improved. IT is used as an option to automate the workflow, providing tools to improve quality and effectiveness, thus helping the organization to adapt quickly.

Source: (ITGI, 2007, p.21). Adapted by the authors.

The proposed measure of maturity, presented here, is supported by the study entitled IT Governance and Process Maturity of ITGI. This proposal is linked to COBIT, and it shows a simpler maturity evaluation process, compared to the maturity assessment mechanisms commonly employed (ITGI, 2008). The IT Governance and Process Maturity publication shows the details of a study involving fifty-one institutions in North America, Asia and Europe, which are organized, among other criteria, by area. These areas of expertise include capital-intensive industries (high cost of capital assets), utilities (infrastructure for public services), service industries, financial institutions, and finally, government and non-profit organizations. The use of attributes attached to processes allowed the design of a methodology to assess the maturity in a simpler and more straightforward way. The original approach of the COBIT presents a specific model that provides a maturity scale for each of the thirty-four processes. Thus, each process has its own maturity model, which has been generated from a generic model. This generic model provides for the identification of referential attributes (Awareness and Communication; Policies, Plans and Procedures; Tools and Automation; Skills and Expertise; Responsibility and Accountability and Goal Setting and Measurement). Such attributes are organized in a 0 to 5 scale (0 = non-existent, 1 = initial or ad hoc, 2 = repeatable but intuitive, 3 = defined, 4 = managed and measured and 5 = optimized) that allows its application to any process, i.e., there is no need, when evaluating the maturity of a given process, to use a specific maturity model linked to it, but to use only the Attributes Maturity Table (Table 2), where the generic model is presented.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

Method for Measuring the Alignment between Information Technology Strategic Planning and Actions of Information Technology Governance

Some tools Recognition of There are ad may exist; the need for the hoc approaches usage is based process is to processes on standard emerging and practices desktop tools There is There is no sporadic The process planned communication and policies are approach to the s about the undefined tool usage issues Source: (XAVIER, 2010)

139

There is no Skills required definition of for the process accountability are not and Goals are not identified. responsibility. clear and no People take measurement A training plan ownership of takes place does not exist issues based on and no formal their own training occurs initiative on a reactive basis

Each maturity level, in this generic model, presents what is expected for each attribute. Indeed, there is a range of maturity levels for each one of the six attributes. According to this scale, it becomes easy to identify the level of maturity of each process simply locating the situation that best fits the current situation of the process in the Attributes Maturity Table. 3. METHOD TO COMPARE STRATEGIC PLANS AND IT GOVERNANCE ACTIONS - COMPLAN â&#x20AC;&#x201C; GTI The COMPLAN - GTI method was created to check the alignment between the strategic plans of the Courts and the plan established by the National Judicial Council for the Judiciary, as well as the alignment of IT governance and management practices with the related strategic planning. This method proposes an objective analysis of the strategic planning of an organization by comparing and listing of all objectives, actions and goals of the strategic planning to the COBIT processes. It also evaluates whether management and IT governance practices are performed by the organizations in accordance with its strategic planning. The step of the method dedicated to comparing the actions and goals of the strategic planning to the COBIT processes is justified by the fact that the Courts which have been evaluated had recently done a COBIT maturity evaluation that pointed out their strengths and weaknesses. The outcome of this evaluation casta doubt on whether the Courts were planning objective actions to overcome their shortcomings. Therefore, this is the purpose of this step of the COMPLAN-GTI method, i.e., to assess whether the planned strategic actions have support in the reality exposed by the COBIT maturity assessment. A strategic plan aims to establish a guideline for the actions that should be followed by all units of the organization in order to reach its targets. The National

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

140 Silva, L. M. Da, Neto, J. S.

Council of Justice established, pursuant to Resolution No. 99 of November 24 th 2009, the IT Strategic Planning for the Judiciary (BRAZIL, 2009), but it did not indicate, among the established objectives, which ones had the highest priority. Consequently, all actions must be undertaken with the same degree of importance. The application of this method requires a previous analysis of the IT Strategic Planning of the organization in order to identify the themes (or perspectives) and the strategic objectives related to the lines of action to be carried out. The method seeks to relate every proposed action to accomplish a strategic planning goal to the COBIT processes. To achieve this, the following actions must be taken: 1. Identify the keywords of the proposed action; 2. Search the occurrence of the keywords in the whole set of processes (thirtyfour) and in the detailed control objectives (two-hundred and ten) of the COBIT. If there is a match, it has to be assessed whether the process or the detailed control objective relates to the proposed action. This assessment, although subjective, should be performed by an appraiser with the following competencies, skills and characteristics: 2.1. Professional experience: he/she should be a technical professional or a participant of the managerial staff of the Court; this person should have practical experience in the area of the evaluated process and he/she should know about the IT management processes practiced in the Court; 2.2. Knowledge of the COBIT 4.1 framework, processes and detailed control objectives; 2.3. Ability to relate the actions taken by the Court to the indicated COBIT processes; 2.4. Ability to work in a team, if the assessment is carried out by more than one professional. 3. The identified process should comprise the column Related COBIT Process â&#x20AC;&#x201C; Priority COBIT Process; 4. In case there is no match between one of the keywords and the processes and the detailed control objectives of COBIT, synonyms should be used in the conducted search in order to exhaust all the possibilities of relationship with those COBIT components. Steps 1 to 4 must be repeated for all actions related to the strategic planning in order to identify all the COBIT processes related to the specific actions of the IT Strategic Plan (PETI). This method was also applied to the IT strategic planning of each one of the Courts under study, therefore allowing the identification of the COBIT processes related to each theme or strategic objective listed in the IT strategic planning of the organization. After the identification of the COBIT processes related to the actions set out in the IT strategic planning of both CNJ and the Court, it is possible to create a map to indicate the presence of these processes in the two plans. This study identified four scenarios in the relationship between the strategic planning of the CNJ and the nth Court being analysed (TRIBUNALn):

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

Method for Measuring the Alignment between Information Technology Strategic Planning and Actions of Information Technology Governance

141

i) COBIT process present in PETI - CNJ and present in PETI - TRIBUNALn (Scenario 1), ii) COBIT process present in PETI - CNJ and absent in PETI - TRIBUNALn (Scenario 2), iii) COBIT process present in PETI - TRIBUNALn and absent in PETI - CNJ (Scenario 3) and iv) COBIT process absent in both strategic planning (Scenario 4). The percentage of alignment is obtained by relating the quantity of processes belonging to scenarios 1 and 2. These are the scenarios where the COBIT process is present in the IT strategic planning established by the CNJ for the Judiciary. Then, it was decided that if the percentage of alignment exceeds 50%, the plans should be considered aligned. In order to obtain this percentage, the following formula was used:

The application of the research questionnaire is also part of the method. The questionnaire identifies the maturity of each one of the COBIT processes in the Court which are analyzed according to six attributes (Awareness and Communication; Policies, Plans and Procedures; Tools and Automation; Skills and Expertise; Responsibility and Accountability and Goal Setting and Measurement). The Process Maturity (MatProc) is the integer value obtained by truncation of the arithmetic mean of the values of each of the attributes related to that process. XAVIER (2010, p. 59). Thus,

Every process was evaluated according to its current status at the time and what is expected two years from now (future status). All thirty-four COBIT processes were evaluated. To check the alignment between the strategic planning of the Court and its IT governance and management practices, a table was created. It shows the current and future maturities of all COBIT processes and the improvement of the maturity necessary for the achievement of the future situation. To calculate this increase in maturity, the current maturity is subtracted from the future maturity. The application of this criterion aims to identify which processes will be subject to greater attention by the Court in the actions to be undertaken in the next two years. Such efforts are related to the status of the COBIT processes. The actions to be taken concerning the processes of scenarios 1 and 2 are those related to the strategic planning of the CNJ. Thus, these efforts should be prioritized as

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

142 Silva, L. M. Da, Neto, J. S.

they aim to enforce the provisions of the CNJ. Processes related to scenarios 3 and 4 are not part of the PETI-CNJ because the processes of scenario 3 are listed only in the PETI of the Court and the processes of scenario 4 are not present in any of the strategic plans. Thus, the percentage of alignment of IT governance and management practices with strategic planning is obtained by applying the formula:

where: EsfSit n = Increase of Maturity for Scenario n If the alignment percentage exceeds 50%, it was decided that the IT governance and management practices should be considered aligned with the strategic planning of the Court, since more than half of the efforts are directed to the guidelines set out in the IT strategic planning. 4.

APPLICATION OF THE COMPLANâ&#x20AC;&#x201C;GTI METHOD

The method was applied to the IT strategic planning of the Brazilian Superior Courts and the Supreme Court as well as to the PETI established by the CNJ for the Judiciary. It was also used to evaluate the IT governance and management practices of the Courts through the assessment of the priorities given to the COBIT processes. To protect the information provided by the Courts, they were identified only as TRIBUNALn , where n ranges from 1 to 5, since five Courts were surveyed. The following values were set by the CNJ on the IT Strategic Planning for the Judiciary: speed, modernity, accessibility, transparency, social and environmental responsibility, fairness, ethics and probity. Thirteen strategic objectives were grouped into eight themes, which are presented in Table 3, with the lines of action established by the CNJ for its implementation. For each of the actions listed in PETI-CNJ the COMPLAN-GTI method was applied in order to identify the COBIT related processes. The analysis of the PETI-CNJ identified major strategic objectives and actions to be undertaken for the achievement of themes. The themes identified by CNJ were: efficiency, access to the main Judicial information system; social responsibility; alignment and integration, institutional performance, people management, infrastructure and technology budget. Each theme had one or more related strategic objectives, with their respective actions. For every action, keywords with their synonyms were identified and they were called search expressions. These expressions have been searched in the process table and in the detailed control objectives of the COBIT. On the selected processes, a subjective analysis was performed by the assessor to evaluate whether the process or the detailed control objective was actually related to the proposed action. The method COMPLAN-GTI was applied to the IT strategic planning of the Courts. According to studies performed in the strategic planning of the Court, the main

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

Method for Measuring the Alignment between Information Technology Strategic Planning and Actions of Information Technology Governance

143

themes/perspectives, the strategic objectives and the lines of action to be undertaken by the Court were identified. After collecting the actions and the lines of action, a method to identify the COBIT processes related to each action was applied, as shown in Table 3. The planning established by CNJ (BRAZIL, 2009) stated in its 2 nd Article that: "The National Council of Justice and the Courts indicated in sections 11 to 92 of Article VII of the Constitution will establish their respective information technology and communications (ITC) strategic plans aligned with the National ICT Strategic Plan, with a minimum coverage of five years and it shall be approved in its plenary and special organs until March 31, 2009." Thus, all the goals and targets set in the strategic planning of the Court under analysis must be aligned to the goals and targets set by the National Council of Justice. 5.

ANALYSIS OF THE RESULTS

The results were organized according to three criteria, as follows: qualitative data analysis of the Courts; alignment between the IT strategic planning of the Courts and that same alignment concerning the National Judicial Council for the Judiciary and, finally, the analysis of the IT actions undertaken by the Courts in light of their IT strategic planning. 5.1

Alignment between PETI-CNJ and PETI-TRIBUNAL

To check the alignment between the strategic planning of the Courts and the one established by the CNJ for the Judiciary, a table was created. In this table, called Table 3, all the COBIT processes were listed and their presence or absence in the strategic planning of CNJ and the Courts was marked. The COBIT processes were selected using the COMPLAN-GTI method as described in Chapters 3 and 4. The columns "CNJ" and "TRIBUNAL N" show the â&#x20AC;&#x153;yesâ&#x20AC;? status if the related process is present in their strategic planning according to the four scenarios that may occur in the relationship between the strategic planning of the CNJ and the COURT.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

144 Silva, L. M. Da, Neto, J. S.

PROCES SO

CNJ

PO01

yes

PO02

yes

PO03

yes

PO04

yes

PO05

yes

PO06

yes

PO07

yes

PO08

yes

PO09

yes

PO10

yes

AI01 AI02

yes

AI03

yes

AI04

yes

AI05

yes

AI06 AI07

yes

DS01

yes

DS02 DS03

yes

DS04

yes

DS05

yes

DS06 DS07

yes

Table 3 â&#x20AC;&#x201C; Alignment of the PETI of the Courts with the PETI-CNJ Sce. Trib Sce. Trib Sce. Trib Sce. Trib Sce. Trib TRIB.1 TRIB.2 TRIB.3 TRIB.4 TRIB.5 1 2 3 4 5 Scenario Scenario Scenario Scenario Scenario yes yes yes yes yes 1 1 1 1 1 Scenario Scenario Scenario Scenario Scenario yes 2 2 1 2 2 Scenario Scenario Scenario Scenario Scenario yes yes yes yes yes 1 1 1 1 1 Scenario Scenario Scenario Scenario Scenario yes yes yes yes yes 1 1 1 1 1 Scenario Scenario Scenario Scenario Scenario yes yes yes yes yes 1 1 1 1 1 Scenario Scenario Scenario Scenario Scenario yes yes yes 2 1 1 2 1 Scenario Scenario Scenario Scenario Scenario yes yes yes yes 1 1 1 2 1 Scenario Scenario Scenario Scenario Scenario yes yes yes yes yes 1 1 1 1 1 Scenario Scenario Scenario Scenario Scenario yes yes yes 1 1 2 1 2 Scenario Scenario Scenario Scenario Scenario yes yes yes yes yes 1 1 1 1 1 Scenario Scenario Scenario Scenario Scenario yes yes 4 4 3 4 3 Scenario Scenario Scenario Scenario Scenario yes 2 2 1 2 2 Scenario Scenario Scenario Scenario Scenario yes yes yes yes 1 2 1 1 1 Scenario Scenario Scenario Scenario Scenario yes yes yes yes yes 1 1 1 1 1 Scenario Scenario Scenario Scenario Scenario yes 2 2 1 2 2 Scenario Scenario Scenario Scenario Scenario yes yes yes 3 4 3 3 4 Scenario Scenario Scenario Scenario Scenario yes yes yes 1 2 1 1 2 Scenario Scenario Scenario Scenario Scenario yes yes yes yes 1 2 1 1 1 Scenario Scenario Scenario Scenario Scenario yes 4 4 3 4 4 Scenario Scenario Scenario Scenario Scenario yes yes 2 2 1 1 2 Scenario Scenario Scenario Scenario Scenario yes 0 yes yes 1 2 1 2 1 Scenario Scenario Scenario Scenario Scenario yes yes yes yes 1 1 1 2 1 Scenario Scenario Scenario Scenario Scenario yes yes yes yes 3 3 4 3 3 Scenario Scenario Scenario Scenario Scenario yes yes yes yes 1 1 1 2 1

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

Method for Measuring the Alignment between Information Technology Strategic Planning and Actions of Information Technology Governance

PROCES SO

CNJ

TRIB.1

DS08

yes

DS09

yes

DS10

yes

DS11 DS12 yes

DS13 ME01

yes

yes yes

ME02 ME03

yes

ME04

yes

145

Sce. Trib Sce. Trib Sce. Trib Sce. Trib Sce. Trib TRIB.2 TRIB.3 TRIB.4 TRIB.5 1 2 3 4 5 Scenario Scenario Scenario Scenario Scenario yes yes 1 2 2 1 1 Scenario Scenario Scenario Scenario Scenario yes 3 4 3 4 4 Scenario Scenario Scenario Scenario Scenario yes 3 4 4 3 4 Scenario Scenario Scenario Scenario Scenario 4 4 4 4 4 Scenario Scenario Scenario Scenario Scenario 4 4 4 4 4 Scenario Scenario Scenario Scenario Scenario yes 3 4 3 4 4 Scenario Scenario Scenario Scenario Scenario yes yes yes 1 2 1 1 1 Scenario Scenario Scenario Scenario Scenario yes yes yes 3 3 4 3 3 Scenario Scenario Scenario Scenario Scenario 2 2 2 2 2 Scenario Scenario Scenario Scenario Scenario yes yes yes yes 1 1 1 1 1

Graph 1 portraits the result of the analysis of the alignment between the PETICNJ and the IT strategic planning of the Courts (formula % alignment). Graph 1 – Alignment between PETI-CNJ and PETI of the Courts

Graph 1 points out that all Courts had alignment percentages exceeding 50%, what indicates, as defined, that the strategic plans of these Courts are aligned with the strategic plans of the CNJ. These results answer “yes” to the first research question which is: Are the guidelines and recommendations of the National Council of Justice being met? In order to answer this question, it is necessary to have in mind that those

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

146 Silva, L. M. Da, Neto, J. S.

guidelines and recommendations aim to meet the guidelines of the controlling agencies, and, also, they especially aim to make the agencies of the Judiciary act with greater effectiveness and efficiency. This result was expected because Resolution No. 99, in its 2nd Article, imposed that the Courts specified in sections 11 to 92 of Article VII of the Constitution (including the High Courts) would have "to develop their information technology and communications strategic planning aligned with the National ICT Strategic Plan, with a minimum coverage of five years [ ... ]" (BRAZIL, 2009, authors' emphasis) . On average, the High Courts and the Supreme Court had a percentage of 70.16% of alignment of their IT strategic planning with the planning established by the CNJ for the Judiciary. Nevertheless, this alignment is not complete due to the diversity of priorities set by the management of the Courts, and such management lasts only two years. Thus, all Courts undertake information technology actions not only to achieve the goals set by the CNJ but, also, to meet the guidelines of their respective administrations. 5.2

Application of the survey questionnaire of IT governance

The survey questionnaire was applied to the five Courts in order to assess the maturity of the IT governance using the COBIT attribute table, as explained in the description of the data collection phase. Each process was evaluated according to the six attributes of the COBIT (Awareness and Communication; Policies, Plans and Procedures; Tools and Automation; Skills and Expertise; Responsibility and Accountability, and Goal Setting and Measurement) on current and future perspectives. The maturity of the process was calculated with basis on the values obtained. This research also aims to provide some information to the Courts in order to enable them to work out benchmarking. Graphs 2 and 3 show the maturities of all Courts, as well as the global average maturity. Thus, this data can be used by managers and administrators to conduct analysis to identify the position of one Court compared to the position of other Courts, and to search for creative solutions to solve common problems. The average maturity, per process, is also shown in the graphs in order to facilitate their identification. Graph 2 â&#x20AC;&#x201C; Current Average Maturity. Processes PO1 to AI7

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

Method for Measuring the Alignment between Information Technology Strategic Planning and Actions of Information Technology Governance

147

Graph 3 â&#x20AC;&#x201C; Current Average Maturity. Processes DS1 to ME4

The global average maturities, the average maturity of each of the High Courts and the average maturity of the Supreme Court were calculated in the current and future scenario. The maturity level values were calculated to one decimal place to allow a better understanding of the differences found between the Courts, although for the COBIT 4.1, these values should be presented in integer format without decimal places. The maturities shown are the integer values obtained for each process, according to the set of attributes of the COBIT. The arithmetic mean was calculated for each process. The following formula was used to calculate the average maturity, the current and future average maturity of each Court.

Graph 4 was plotted with the values obtained above.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

148 Silva, L. M. Da, Neto, J. S.

Graph 4 â&#x20AC;&#x201C; Current and Future Maturities of the Courts

The average maturity of the Courts was obtained applying the formula:

The current and future average maturity of the High Courts and the Supreme Court was: Current average maturity Future average maturity

= 2.0 = 3.1

The current maturity value, 2.0, shows that the management and the IT governance processes of the Superior Courts, although repeatable, are not documented and are just intuitive. Processes have evolved to a stage where similar procedures are followed and executed by different people. However, there are no formal training standard procedures and the official communication is not yet institutionalized. There is a tendency to focus the responsibilities on the individual, with the corresponding increase risk of errors. The fact that the courts wish to reach a level of maturity in the next two or three years shows that the upper administration of the Courts is confident in sponsoring the IT projects. To reach this level of maturity, it is necessary that the IT governance practices are effectively sponsored by the high authorities of the Courts who should prioritize the creation of IT governance Committees, and, also, they should prioritize the actions related to the strategic alignment of IT with business and the delivery of value. It is expected that in two years from now the procedures be standardized, documented and

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

Method for Measuring the Alignment between Information Technology Strategic Planning and Actions of Information Technology Governance

149

formally communicated as well as the monitoring of the adherence to the standard procedures be in effect. 5.3

Alignment between PETI-TRIBUNAL and IT governance

The application of the method to survey the alignment among the IT governance (ITG), the management practices and the strategic planning have identified which processes should be object of greater attention by the Court in the actions to be undertaken in the next two years. Such efforts are related to the situation of the COBIT process. Graph 5 shows the percentage of effort dedicated to the actions related to scenarios 1 and 2, above 50%. Graph 5 - %Alignment ITG and PETIâ&#x20AC;&#x201C;TRIBUNAIS

The average value found is 70%, what justifies the statement that the IT governance and the management practices of the Courts are in line with their strategic planning. These results answer the second research question: Are the actions prioritized by the Courts aligned with their strategic planning? It is important to remark that effort calculations were performed in absolute terms, with each unity of maturity increase (future status - current status) corresponding to one unity of effort, regardless of the complexity of the process evaluated. Some COBIT processes require greater efforts than others in order to increase one unit into the maturity level. The framework COBIT 4.1 provides an annex, named Linking IT Processes to IT Goals, where the thirty-four COBIT processes are related to twentyeight IT Goals. It is possible to confirm that some processes influence a larger number of goals (ITGI, 2007). For example, when analyzing IT Goals achieved by the process ME4 - Provide IT governance (five goals), it seems that these goals are more complex than the process AI1 - Identify Automated Solutions (only two goals). However, for the purposes of this research, these efforts were equally carried out.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

150 Silva, L. M. Da, Neto, J. S.

CONCLUSION

The aim of this work is to propose a method to assess whether the guidelines of an information technology strategic planning (PETI) are observed on the IT governance and management practices, and to verify the alignment between IT strategic plans. In order to evaluate the applicability of the method the IT strategic planning of the Superior Courts and the Supreme Court were analyzed, and also the PETI established by the National Judicial Council for the Judiciary. As a model for maturity analysis, it was undertaken a survey of IT governance maturity of the Superior Courts and the Supreme Court using the Attribute Table of COBIT, which is composed of six attributes: Awareness and Communication; Policies, Plans and Procedures; Tools and Automation, Skills and Expertise, Responsibility and Accountability and Goal Setting and Measurement. This research was carried out due to the low incidence of academic studies (thesis, articles and books) that deal with both IT Governance and Strategic Planning in public organizations, as well as the repeated interventions of internal and external control agencies. The research in the Court of Audit, specially, pointed out the weaknesses on IT Governance in the agencies and entities of the Brazilian Federal Public Administration. The huge financial resources that have been invested by the public sector in activities related to information technology were also considered. In the last ten years, for example, 12.5 billion dollars have been spent on IT resources, according to the Court of Audit. The proposed method, called COMPLAN-GTI, allows linking the guidelines of an IT strategic planning to COBIT processes. In this study, the version 4.1 of the COBIT framework was used. Initially, it was carried out an evaluation of the relation of the IT strategic planning established by the National Judicial Council for the Judiciary concerning the COBIT processes. The same analysis was done with the PETI of the High Courts and the Supreme Court, making it possible to determine the degree of alignment between the various strategic plans. The Courts are, on average, 68% aligned with the guidelines and actions established by the National Council of Justice for the Judiciary. The application of the method permitted to check if the IT governance and management practices are consistent with the strategic plan of the organization. It was observed on the survey carried out in the Courts that the average convergence between PETI and IT governance and management practices lies at 70%, therefore concluding that the strategic plans have influenced the action plans in these organizations. The results obtained will allow the Federal Public Administration agencies, in particular the Brazilian Supreme Courts, to conduct benchmarking aiming to identify the strengths and weaknesses and, with the support of those who implemented best practices in the Courts, to contribute to the improvement of the Brazilian public sector. It is also possible to perform an analysis to identify the position of the Court in relation to other Courts, in order to enable the search for creative solutions to solve common problems.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

Method for Measuring the Alignment between Information Technology Strategic Planning and Actions of Information Technology Governance

151

It is worth mentioning that the proposed method can, in principle, be applied to public or private organizations, as a general tool for the analysis of IT governance management practices. As for the limitations of this research, it can be mentioned that the method requires minimum skills for the assessor, such as knowledge of the maturity measurement model based on attributes of the COBIT 4.1, knowledge of the actions of IT governance and management practiced on the agency, and ability to relate the actions taken by the Court and the related COBIT processes. In addition, as the survey did not have the characteristics of an audit, there was no evidence of responses, what requires caution on the analysis of the results. Finally, it is important to point out that this research has not undergone a validation process, given the difficulty of reproducing the conditions necessary for its implementation, such as the existence of an overall strategic plan of the sector, a strategic plan for each agency evaluated and a recent IT Governance maturity evaluation for each one of them. Nowadays, this situation is found only in the Judiciary, particularly in the CNJ and the High Courts, being unique on the Federal Public Administration. It is suggested, as a future work, to undertake a focus group meeting with managers of the Federal Public Administration for model validation. In this focus group meeting the principles, guidelines and procedures of the COMPLAN - GTI method will be presented and the perceptions of the managers about the proposed model will be collected through semi-structured interviews. The contributions of these experts will be included in a new version of the COMPLAN-GTI method. Moreover, the proposed model can be enhanced to implement its guidelines on the new COBIT 5 framework, published by ISACA in April 2012. The evolution of the current model 4.1 to the version 5 model will bring benefits due to the separation of the governance practices from the management practices, simplifying the application of the method. Overall, this work may serve as a reference for other research studies related to the same general themes: IT governance, strategic planning, and IT management in public organizations REFERENCES Brasil. Conselho Nacional de Justiça. Resolução nº 99, de 24 de novembro de 2009. Planejamento Estratégico de TI do Poder Judiciário. http://www.cnj.jus.br/images/gestao-planejamento-poder-judiciario/petic_nacional.pdf Henderson, John; Venkatraman, N. Strategic alignment: Leveraging information technology for transforming organizations. IBM Systems Journal, Vol. 32, No. 1, 1993. IBGC. Código das melhores práticas de governança corporativa. 4ª Edição. São Paulo: IBGC, 2009. ITGI (Information Technology Governance Institute). Cobit 4.1. USA: 2007. <http://www.isaca.org/Knowledge-Center/cobit/Documents/cobit41-portuguese.pdf>. ______. Global Status Report on the Governance of Enterprise IT. USA, 2011. <http://www.isaca.org/Knowledge-Center>. ______. IT Governance and Process Maturity. USA, 2008.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

152 Silva, L. M. Da, Neto, J. S.

Rezende, Denis Alcides. Planejamento de Sistemas de Informação e Informática. Guia Prático para Planejar a Tecnologia da Informação Integrada ao Planejamento Estratégico das Organizações. 4º Edição. São Paulo: Editora Atlas, 2011. Weill, Peter e ROSS, Jeanne. W. Governança de Tecnologia da Informação. São Paulo: M. Books do Brasil Editora Ltda., 2006. Xavier, M. B. G. Mensuração da Governança de TI na Administração Direta Federal Brasileira. 2010. 123p. Dissertação de Mestrado. Universidade Católica de Brasília, Brasília, 2010. http://www.bdtd.ucb.br/tede/tde_busca/arquivo.php?codArquivo=1333&PHPSESSID= 21faebff671d65448560a0b967731d20

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 , pp. 131-152

www.jistem.fea.usp.br

JISTEM - Journal of Information Systems and Technology Management Revista de Gestão da Tecnologia e Sistemas de Informação Vol. 11, No. 1, Jan/Apr., 2014 pp. 153-168 ISSN online: 1807-1775 DOI: 10.1590/S1807-17752014000100009

BENEFITS OF THE MARKETING INFORMATION SYSTEM IN THE CLOTHING RETAIL BUSINESS Josimeire Pessoa de Queiroz Braulio Oliveira Centro Universitário da FEI, São Paulo, SP, Brazil ______________________________________________________________________ ABSTRACT This paper has the purpose of being familiarized with the benefits of the marketing information system in companies that operate in the retail business. A field research was done in a major clothing retailer, by means of a case study that consisted of interviews with two employees of the company, one from the top management and one from the marketing area, based on a semi-structured script , in addition to a documental analysis. The results indicate that information and its systematization possess great importance for the development of competitiveness, contributing to the development of the operations and to the promotion of new business opportunities, as provided by the pertinent literature. Keywords: Marketing information system; Environmental analysis; Marketing decision-making; Clothing retail business; Competitiveness. 1.

INTRODUCTION

Due to high competitiveness, many companies have sought resources and management tools which enable them to quickly and flexibly respond to market demands. This is how information technology tools have gained a prominent role within competitive companies, as they aim to promote reliability and fast information, contributing to the mitigation of risks in decision-making processes (Kubiak & Kowalik, 2010). _____________________________________________________________________________________________ Manuscript first received/Recebido em 25/11/2010 Manuscript accepted/Aprovado em: 11/01/2014 Address for correspondence / Endereço para correspondência Josimeire Pessoa de Queiroz. Business Administrator with a Master´s degree from Centro Universitário da FEI and Doctorate student in Business Administration at Centro Universitário da FEI. Address: Travessa Indiavaí, 69. CEP 03526-030 – São Paulo/SP. Telephone: (55.11) 98161-8139 E-mail: josimeirepessoa@ig.com.br. Braulio Oliveira. Business Administrator with a Master´s degree from Universidade Prebisteriana Mackenzie, a doctorate degree from Faculdade de Economia, Administração e Contabilidade da Universidade de São Paulo (FEA/USP). Professor at the Administration Post-graduation Programa at FEI. Address: Rua Tamandaré, 688. Liberdade. CEP: 01525-000 – São Paulo/SP. Telephone: (55.11) 3207-6800. E-mail: braulio@fei.edu.br. Published by/ Publicado por: TECSI FEA USP – 2014 All rights reserved.

154 Queiroz, J. P., Oliveira, B.

Hooley, Saunders and Piercy (2005, p. 133) highlight the importance of information in the decision-making processes, as its importance is related to the spotting of opportunities and to the best business management. Particularly to the extent that the marketing process is concerned, its strategic dimension is in charge of providing guidance to companies as to market opportunities, aiming at the development and maintenance of competitive advantages and profits in the long run, based on the systematic and continuous analysis of the needs and wishes of current and potential clients as well as the needs and wishes of competitors (Lambin, 2000). Therefore, strategic marketing is the one that monitors the market and indentifies market opportunities and segments based on the expectations of the target audience (Minciotti, 1992). Notwithstanding, Hooper, Huff and Thirkell (2010) argue that organizational functions around business strategies are increasingly fed by marketing strategies, interpreted by marketing orientation and by its elements (consumer orientation, competitor orientation or interfunctional coordination). Likewise, Cravens and Piercy (2006) state that information is an input which can be used as a base to solve problems, assess potential actions, improve operational performance and mitigate risks in decision-making processes, which, according to Gupta (2012), enable new opportunities to take place as well as changes in the industry. As such, it is necessary to develop a formal and stable structure to collect information, which freely flows within the organization so that it contributes to proper decision-making processes by managers (Jaworski, Macinnis & Kohli, 2002). Such authors as Kotler and Keller (2006) and Lambin (2000) state the need for structuring and systematizing information so that marketing decisions are continuously supported. According to Berhan, Paul and Jan (2012), it is impossible to consider marketing without an information system. In this context, Pinto et al. (2006) argue that many companies combine marketing policies and techniques with information technologies, enabling managers to offer distinctive services and products, create value for the clients and generate profits for the organization. Thus, the companyÂ´s structure must be learning-process oriented, capable of obtaining relevant information from the data related to the decisions to be made; such decisions pertain to the strategies and actions which enable the defined goals to be met. In addition, Levy and Weitz (2000) argue that information has a particular value in the retail business, as it offers new possibilities to identify opportunities for competitive advantages through the familiarity with the target audience, with the products and services offered and with the competition. By considering all the information stated above, this paper has the purpose of being familiarized with the benefits of the marketing information system (MIS) for marketing decisions in the clothing retail business. In order to meet the defined goal, in addition to approaching related subject matters, a field research was done by means of a case study method, in a major company in the clothing retail business, which is an expressive segment in the Brazilian economy. This paper is divided into five parts, including this introduction. The second part addresses the MIS concept and models; the third part shows the field research methodology; the fourth part shows the results of the field research, from the companyÂ´s JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 153-168

www.jistem.fea.usp.br

Benefits of the Marketing Information System in the Clothing Retail Business

155

characterization to the presentation and analysis of obtained data; the fifth part presents this paper´s conclusion. 2.

MIS CONCEPT AND MODELS

In general, an information system can be defined as a set of interconnected components which work in concert in order to collect, process, store and recover data and distribute information for the purpose of making it easy to plan, control, coordinate and analyze the decision-making process within organizations (Laudon & Laudon, 2001; Swartz & Iacobucci, 1999; Deshpande, 2013). Levy and Weitz (2000) also state that the use of an information system must associate and store data from the whole organization in order to identify and understand current and potential clients, which makes it crucial in marketing management and planning. For Karamarko (2010), measuring and assessing business results are possible thanks to the reports and forecasts provided by the MIS. New information management technologies available offer possibilities for improving the competitive advantage of the companies, such as information management systems, database systems, decision-making support systems, customer relationship and value chain management systems, and smart competitiveness systems (Cravens & Piercy, 2006). Regarding marketing decision making in particular, Kotler and Keller (2006) explain that the information system allows classifying, analyzing, assessing and distributing relevant information in a precise and timely fashion for the decision makers. In agreement with this view, Mattar et al. (2009) state that the MIS allows monitoring the company´s results and the external environment, disseminating the data obtained by the company and contributing to reducing uncertainties in the decision-making process. Therefore, the use of the marketing information system for decision making saves time and prevents rework, as it provides information to all marketing activities regarding planning, promotion, sales of goods and services for both customer satisfaction and organizational goals (Ismail, 2011). Thus, the MIS “supports the marketing plan and decisions, helping the management of information” (Campomar & Ikeda, 2006, p. 37). In addition, Chatzipanagiotou and Coritos (2010) state that the MIS is vital for decision-making processes and for the development of marketing planning. However, Gounaris, Panigyrakis and Chatzipanagiotou (2007) believe that, based on their studies, despite the recognition and importance of information in many academic papers, this subject matter is complex and always associated with business success. Nevertheless, there is no agreement among many scholars regarding the definition, efficiency and operationalization of the system, as this subject matter becomes even more complicated when the marketing information system is only limited to such aspects as sales, profitability, and market share for strategic decision-making processes. However, many MIS definitions refer to it as an important element in the preparation and in the organizational strategic development, as per Figure 1.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 153-168

www.jistem.fea.usp.br

156 Queiroz, J. P., Oliveira, B.

Figure 1 â&#x20AC;&#x201C; Definitions of the Marketing Information System MIS DEFINITIONS

AUTHORS

Computerized system to provide, in an organized fashion, the Harmon, 2003. flow of information, analytically and operationally, capable of supporting decision-making processes, marketing activities of an organization and all the strategic marketing elements. Interacting and continuous structure of people, equipment and Kotler procedures to obtain, classify, analyze, assess and distribute 2006 necessary and accurate information in a timely manner to the marketing decision makers, whether the information is internal, external or from other markets, which is made necessary, useful and sufficient to serve users.

Keller,

A complex interacting structure of people, machines, and James,1998 procedures to generate, in an orderly and pertinent manner, internal and external information to the organization, used as a base for the decision-making process. Computer planned in a data system whose purpose is to provide Boone a task manager with a continuous flow of information which is 2007. relevant to specific decisions in its scope of responsibilities.

Kurtz,

Collection of relevant data within a significant goal, whose data King, 2010. composes recommendations which can be stored for future use. Quality project to help in the decision-making process, Jobber composed of four elements: database (internal reports); 2006. marketing intelligence (economy, market, business environment), market research (regular and special occasion/ goal specific); analysis of marketing information (Decision Support System).

Fahy,

Source: Authors, adapted from Ismail (2011, p. 175-178). Mattar et al. (2009, p. 100) regard the definition of the MIS by Cox and Good as the most complete, in 1967.

It is a structured and interacting complex of people, machines and procedures whose purpose is to generate an orderly and continuous data flow, collected from internal and external sources of the company, to be used as a base for decision making in areas specifically in charge of marketing. With no intention of exhausting this subject matter or presenting a broad review of the MIS models, some of them are presented with the purpose of enabling the discussion of this subject matter and obtaining more support for the field research. As for the operational model, Kotlere and Keller (2006) believe that the MIS must feature the following elements: a) the subsystem of internal records, which allows monitoring and analysis of the organizationÂ´s performance; b) the subsystem of marketing intelligence, which monitors the many environmental variables and marketing events that influence the business, in which the competitors are included; c) the subsystem of marketing research, which is aimed at the development, collection, analysis and edition of systematic data reports and relevant findings for the resolution of specific marketing problems; d) a subsystem of analysis and support to market decisions â&#x20AC;&#x201C; a coordinated JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 153-168

www.jistem.fea.usp.br

Benefits of the Marketing Information System in the Clothing Retail Business

157

set of tools and techniques, software and hardware to assist in the analysis and interpretation of the information. The suggested model is represented by Figure 2 (Kotler, 1998, p. 12)1: Figure 2 â&#x20AC;&#x201C; MIS Model by Kotler Marketing Information System

Marketing environment

Internal Reports Subsystem

Marketing Research Subsystem

Target markets Marketing channels Competitors Audience Macro environmental forces

Marketing Managers Analysis Planning Implementation Control

Marketing Intelligence Subsystem

Market Analytical Subsystem

Source: Kotler (1998, p. 12).

Mattar et al. (2009) highlight that fact that a generic MIS model must be applied and adapted to the reality of any company as long as it comprehends such activities as joining, processing, disseminating and storing relevant data and information, internal and external to the organization, for marketing decision-making processes. Thus, the proposed model, as seen in Figure 4, is made up of four subsystems: (1) environmental monitoring system; (2) competitive information systems; (3) internal information systems; (4) marketing research system. Nevertheless, Mattar et al. (2009) add that there is not a standardized MIS model; each company has a particular need for information which corresponds to its expectations, also respecting the management style and/or the culture of the group which manages it.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 153-168

www.jistem.fea.usp.br

158 Queiroz, J. P., Oliveira, B.

Figure 3 – MIS Model by Mattar et al Data Sources Macro environment

Company

Competitors

Market

Data Data Collection Environmental Monitoring System

Competitive information System

Internal information systems

Marketing Research System

Processing Monitors

Condenses

Stores

Analyzes

Assesses

Treats

Updates

Interprets

Selects

Classifies

Recovers

Disseminates

Information Users Analyze

Plan

Organize

Decide

Execute

Control

Source: Mattar et al. (2009, p.105).

FIELD RESEARCH METHODOLOGY

Allowing for the relevance presented by the literature reviewed regarding the relationship of the MIS with the marketing decision making, the purpose of the field research was defined as the verification of the benefits obtained with the use of the MIS. For this purpose, the descriptive research was chosen, which was performed by means of a case study, as it refers to a contemporary phenomenon, within a real context, (Vergara, 2004; Yin, 2005). The chosen company for the empirical contribution operates in the clothing retail business, which is viewed as an expressive sector in the Brazilian economy. Data collection took place by means of personal, individual interviews, with two of the company´s executives, based on a semi-structured script, which consisted of questions related to the variables: “use of the MIS” and “benefits provided by the MIS”. Such questions were written based on the literature review performed, for the purpose of enabling a confronting and a corresponding analysis. The information obtained was verified and crossed with the information available in the management documents provided by those being researched. The research was done in February 2009, with two decision makers, in accordance with their roles: one of them from the top management and the other one from the marketing department. Two employees were interviewed for the purpose of

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 153-168

www.jistem.fea.usp.br

Benefits of the Marketing Information System in the Clothing Retail Business

159

reducing biased opinions and personal interests, as it was possible to triangulate the data obtained as well as with the documents provided by the company (Yin, 2005). Such documents contained specific data on the research variables and data related to the results and their projections, which served as an important indicator for the purposes of the research problem in question, especially regarding the existence of a MIS and the scope and flow of the information generated. The script was based on the theoretical background described in this paper. The first part of the script attempted to become more familiarized with the company whereas the second part addressed the purpose of the research itself. The analysis, from the transcription of the answers, which were recorded with the consent of the interviewees, and from the documents provided by them, was descriptive and interpretative, and it attempted to find a correspondence with the theory presented so that the information obtained in the literature review could be reinforced or questioned. (Vergara, 2004; Yin, 2005). 4.

PRESENTATION OF THE RESULTS OF THE FIELD RESEARCH

4.1

Characterization of the business sector of the researched company

According to the Ministry of Development, Industry and Foreign Trade (MDIFT) (2008), by comparing January 2008 with February 2007, the sectors which grew the most in terms of productivity were the ones in transportation equipment (8.1%), machines and equipment (8.3%), leather and shoes (10.7%) and clothing (11.6%). According to Renner (2009), the fashion retail sector is quite fragmented which allowed the successful development of major department store chains, which increasingly receive investments from foreign groups. The fragmented market and the search for positioning have made it clear that when it comes to brand consumption, according to Luz (2008), the clothing market is mainly composed of classes A and B, with class A spending 19 times more than the other classes; classes A and B account for the consumption of 57% of the fashion products in Brazil, whereas class C accounts for 31%. Regarding brand consumption, considering the economic expressiveness, Luz (2008) also reveals that the S達o Paulo Fashion Week [SPFW] event sets in motion around 1.5 billion Brazilian reals. However, according to a report released by Renner (2009), the increase in the purchasing power of the families in classes C and D have drawn the attention of the retail sector interested in attracting the low-income population. Also, the behavior of the population has been the target of analysis, with the search of a single point of sales, leading to the expansion of the shopping centers and, consequently the presence of department stores. For the production of items offered for consumption in the fashion market, according to the Ministry of Labor and Employment (MLE), most of the employees who compose the sector are women and the States that hire the most employees in the sector are S達o Paulo, Santa Catarina and Minas Gerais.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 153-168

www.jistem.fea.usp.br

160 Queiroz, J. P., Oliveira, B.

4.2

Characterization of the company

The researched company was founded in the late 40´s and its main activity was the sale of textiles. Acquired by a domestic group at the end of the 70´s, after restructuring, it started to offer ready-to-wear clothes. Currently, it is among the major fashion retail chains in the country. The quest for better positioning made it, in the 90´s, turn its attention to fashion, in international and national trends (the SPFW, for instance). In the same period of time, they introduced quality improvements to its products and the launching of its own brands. Currently, the company is the second largest chain of department stores in Brazil and it competes with companies that receive foreign investment. Among the domestic ones, it occupies the first position. The group to which it belongs is regarded as the largest group in the making of clothes in Latin America. The company keeps employees in fashion centers in Europe and in the United States, which enables its collections to be based on the main fashion trends. On average, 120,000 pieces a day are made in their factories, delivered at their own stores by the carrier company that belongs to the group. Advertising campaigns are aimed at the main target audiences of the stores: women aged 25 to 40 years, through television advertisements and specific in-store marketing activities, using information from its customer database. With the purpose of strengthening the relationship with the active customer, the company offers the store's own card to make financial services available, such as cash withdrawals, personal loans, insurance, home and vehicle assistance, theft and robbery protection, unemployment and home insurance. In Figure 4, it is possible to see other information on the researched company. Figure 4 – Structure of the Company Distribution Centers Operating in

Natal (State of Rio Grande do Norte) and Guarulhos (State of São Paulo), with 55,000 square meters and 86,000 square meters of living space, respectively. 21 Brazilian States and in the Federal District São Paulo

Head office Shopping center

Northeastern Region

Number of stores

135 in 2011

Financial services company

17.6 million members in 2010

Total number of meters of the stores

395,636 m2 square meters

Factories

Northeastern Region - one for the production of plain textile (jeans and shirts), the other for the production of knitwear and shirts.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 153-168

www.jistem.fea.usp.br

Benefits of the Marketing Information System in the Clothing Retail Business

161

Women´s fashion wear; men´s wear, children´s and teenagers´ wear, shoes and home fashion.

Departments Target Audiences

Classes C and D

Number of Employees

38,700 in 2011 R$ 2,6 billion Brazilian reals

Revenue in 2010

200 thousand pieces a day

Average production in 2009

Source: Authors, based on the data collected in the research. 4.2

Description and Analysis of the Researched Variables

4.2.1 Use of the MIS The company has an Information System (IS) based on the software called SAP (Systems Applications and Products in Data Processing), developed by the German company with the same name. It enables to obtain information and disseminate them throughout the company so that it can be used in its activities. Regarded as slightly complex by the interviewees, it sometimes requires that some information, after being extracted and analyzed, be transferred to the Intranet, which tends to make its use easy by the users. The company´s SAP system is made up of two bases called BW (Business Warehouse); one of them serves the company´s commercial area and the other one serves the remaining areas. The model can be divided into two blocks: (1) one defined from the user´s profile, who has access only to their department´s information, in order to perform their tasks, as seen in Figure 6, and (2) the other one in which the information presents itself in a disaggregated manner, more complex, and which is worked on with statistical modeling tools. Due to the merging of the Business Intelligence and Information Technology, the Intranet, based on the user´s profile, at the moment, shows some “screens” or accesses under construction, as seen in Figure 5, by the ellipse in white. Figure 5 – IS model based on the user´s profile.

Intranet

HR Purchase Sales

Source: Authors, based upon the company´s Intranet output.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 153-168

www.jistem.fea.usp.br

162 Queiroz, J. P., Oliveira, B.

As for the flow and distribution of information, the access is restricted to each department within the company. Each store has only access to its information. The purpose of the restrictions, mentioned by both interviewees, is to keep the information from being passed on to the competition. Due to strategic reasons, the Internal audit alone has full access to all the information and it grants access authorization. However, according to the interviewees, these measures are seen by many users as something unsuitable and complex, which causes delays in the decision-making process, especially in the Marketing area, which needs data that matches the customerÂ´s behavior, regional cultures, sales and other aspects. As for the MIS model, both interviewees stated that the company developed its own model, which meets their needs and features a system of decision-making support and of relationship management with customers and the value chain. In detail, the marketing professional explained that the MIS is an SAP module, accessible to the marketing department, and it has the subsystems of records, marketing research and decision-making support, which enables the use of the statistical modeling. Therefore, the model developed by the company is the closest to what is proposed by Kotler (1998). Also, as seen in the pertinent literature, the company has an information system structure interconnected with the use of software and the Intranet, which operates in an integrated manner with the MIS for the development of strategies and action plans (Laudon & Laudon, 2004; Cravens & Piercy, 2006; Swartz & Iacobucci, 1999; Hooper, Huff & Thirkell, 2010; Deshpande, 2013). Nevertheless, by taking into account the fact that the pertinent literature highlights that information must be accessible to all decision makers, the abovementioned restriction results in certain dissatisfaction from the users, which might hamper the achievement of the benefits suggested by the pertinent literature (Laudon & Laudon, 2004; Hooley, Saunders, & Piercy, 2005; Cravens & Piercy, 2006; Minciotti, 1992; Campomar & Ikeda, 2006; Mattar et al., 2009; Swartz & Iacobucci, 1999; Deshpande, 2013). 4.2.2 Benefits provided by the MIS The two interviewed managers, when questioned about the benefits the company is provided by the MIS, recognized that there are many of them, but in fact the company only allows to achieve them if they are used with other IS modules, which is not always feasible, due to the above-mentioned access restrictions. The statement is made clear when the interviewees classify in terms of importance the benefits which are solely provided the MIS (Figure 6), whose items were listed based on the literature review.

Figure 6 â&#x20AC;&#x201C; Benefits generated from the use of the companyÂ´s MIS Item

BENEFIT OBTAINED

RESULTS

High

Average

Reduce operational costs

Prepare more accurate and timely reports

Improve productivity

Control Marketing costs

Improve internal services performed and offered

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 153-168

www.jistem.fea.usp.br

Low

163

Benefits of the Marketing Information System in the Clothing Retail Business

Item

BENEFIT OBTAINED

RESULTS

High

Average

Low

Improve decision-making process

Foresee and develop plans

Improve the organizational structure

Promote price and advertising strategies

Improve the flexibilization of the changes in environmental factors

Optimize the provision of services offered to customers.

Improve the interaction with their suppliers

Improve the sales force performance

Develop staff training programs

Build a distinctive customer service

Identify the target market and concentrate efforts

Become familiarized with the competitorÂ´s activities

Monitor and identify new segments

Develop new products/services

Manage and reduce inventory costs

Find new sources of goods

Strengthen the relationship with consumers

Improve the distribution channel control*

Reduce the risk perceived by the customer *

Build advantages in the long run over the competition

Identify the nature of the services and products offered to the market *

Improvement of the control

Improvement in the planning

X X X X

X X X X X X

*N/A / unknown. Source: Authors, developed based upon the data collected from the research.

Both believe that the MIS allows the establishment of the foundation of retail strategies. They also speak highly of the possibility of reducing operational costs, making more accurate reports, improving productivity, controlling marketing costs, improving internal services, improving decision making, developing plans, promoting price and communication strategies, optimizing the services offered to customers, indentifying target markets and new segments and managing inventories.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 153-168

www.jistem.fea.usp.br

164 Queiroz, J. P., Oliveira, B.

However, such benefits as improving distribution control, reducing the risks perceived by customers and indentifying the nature of the services and products offered to the market are analyzed in the IS by other sectors and involve Logistics (factory/carrier), Purchasing and Business Intelligence. Even the contribution from the MIS to marketing results needs the monitoring of the responses from campaigns and promotions by the Business Intelligence, particularly due to information access restrictions. Regarding the development of the main retail strategies, the information obtained contributes to strategic planning and decisions on the lines of products, price setting and store expansion and location. As to the marketing planning, there is the concern for monitoring needs, satisfaction and customer purchase processes, performed by the Business Intelligence department, which handles the strategic function of supply and provision of information to the managers for decision-making purposes. With a focus on the customers and maintaining its positioning within the segment in which it operates, the company invests in the integration with the factory in order to gain market and offer fashion with quality and fair prices. This integration is seen as a distinctive element over the competition, a sort of an innovative service added to the customer service. Nevertheless, for the interviewees, the integration store/factory is seen as a distinctive element which tends to enable cost reduction and improvement in price strategies over the competitors. According to the top management interviewee, based on the analysis of market share: “Currently, we are the second largest company in this sector, but we have been working to reach the first position soon, and in order to do so the company has worked on the integration between the stores and the group, opening of new stores and refurbishing of the existing ones, consolidation of market share in the Northern and Northeastern regions, increase in the market share in São Paulo and Rio de Janeiro, creation of a financial services company, strengthening of the relationship with customers by means of store cards and focus on innovation, launching of trends and products by means of market research and marketing intelligence, modernization of the manufacturing complex and implementation of information technology for operational and financial management.” Regarding the interaction of the company with competitors, there is some difficulty obtaining information and, often times, the store´s managers are the ones who visit their competitors and pass information on the Business Intelligence, Commercial, Marketing, and to other operational areas. With the purpose of meeting market needs and maintaining its positioning, the company believes that observing and analyzing the main events and the demographic trends demand efforts, but which tend to offer more opportunities than difficulties if they are properly explored. Considering the fact that the company has many stores in other States of the country, constant studies are performed in relation to the cultures and customs of each region served, as well as in relation to the main fashion events in each region. This type of analysis is seen as an opportunity to increase sales when associated with meeting seasonal sales demand. 5.

FINAL CONSIDERATIONS

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 153-168

www.jistem.fea.usp.br

Benefits of the Marketing Information System in the Clothing Retail Business

165

As seen in the statements from the interviewees, the acquisition of new technologies, the acquisition and use of other software and the merging of the IT and BI areas under the same management show the concern of the researched company to significantly invest in IS and in its use in an integrated manner with the MIS and with other companyÂ´s databases, which allow analyzing customers, the market and, though poorly, the competition as well. The close-knit relationship between Marketing, Business Intelligence and Information Technology shows the need for adaptation of the reality and the culture of the company to the use of information. By analyzing the benefits provided by the MIS, it is possible to see the dedication of the company to obtain information about its customers, in order to strengthen their relationship, as well as the search for meeting their customers' needs. The company uses information more as a support to the risk reduction management in decision making than to the competitive advantage creation, although it is moving in the direction of the use of information as a strategic resource, for the purpose of providing a better performance of the operations and new business opportunities, as it has recently integrated Business Intelligence and Information Technology under the same management. The search for and creation of a competitive advantage can be maximized if the company uses the MIS effectively, increasing the possibility of profiting from the implemented innovations and from identifying other demands that are not met by its competitors, demands which the company would have conditions to explore (Kotler & Keller, 2006; Gounaris, Panigyrakis & Chatzipanagiotou, 2007; Mattar et al., 2009; Gupta, 2012). The information obtained about customers and the market trends prompted the group to opt for the production of their own products, and, consequently, their distribution by their own stores. In the companyÂ´s business sector, the products have a short life cycle, which challenges it to make speedy and flexible market plannings, where the use of information is essential, in accordance with what the pertinent literature presents (Swartz & Iacobucci, 1999; Sisodia, 1992; Lovelock & Whright, 2005; Levy & Weitz, 2000; Pinto et al., 2006; Minciotti, 1992; Cravens & Piercy, 2006; Kotler & Keller, 2006; Mattar et al., 2009). Although the information access restrictions, based on the creation of profiles, cause delays in the development of projects and in the definition of plannings, the research permitted to see that the MIS provides the benefits presented in the pertinent literature; however, in order to further enjoy such benefits, there is still the need of its integrated use with the companyÂ´s IS. Within the limitations of this work, the adopted method to perform the field research can be mentioned, as it does not allow for generalizations. However, it is important to highlight that, despite this limitation, the objective established, which is the verification of the benefits obtained with the use of the MIS, was achieved. Another evident limitation is in regards to performing a single case study; it is believed that the possible comparison with other case studies could further clarify the treatment of the flow and distribution of information, which, in this research, was limited to the access based on the creation of user profiles.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 153-168

www.jistem.fea.usp.br

166 Queiroz, J. P., Oliveira, B.

As a manner for contributing to further research, the exploration of other variables is recommended, such as the necessary information for decision-making processes and environmental analysis, regarding this research´s problem. Also, it is recommended that similar research be done, and also of a quantitative nature in the business sector in question and in other relevant ones, for better assessment of the management of information that the marketing information system provides for creating and maintaining competitiveness, whether due to the scarcity of empirical studies or due to economic representativeness.

REFERENCES Berhan, E.; Paul, I.; Jan, H. G. (2012). Marketing information systems and price change decision making: the case of Ethiopian industries. Germany: LAP Lambert Academic Publishing. Campomar, M. C., & Ikeda, A. A. (2006). O planejamento de marketing e a confecção de planos: dos conceitos a um novo modelo. São Paulo: Saraiva. Chatzipanagiotou, K.C.; & Coritos, C. D. (2010). A suggested typology of Greek upscale hotels based on their MrKIS: implications for hotels’ overall effectiveness. European Journal of Marketing, 44 (11/12), 1576-1611. Cravens, D. W., & Piercy, N. F. (2006). Strategic marketing. Columbus: McGraw-Hill. Deshpande, A. S. (2013). Marketing information system for industrial products. International Journal of Management & Information Technology, 3 (1), 71-76. Gounaris, S. P.; Panigyrakis, G. G.; & Chatzipanagiotou, K. C. (2007). Measuring the effectiveness of marketing information systems: an empirically validated instrument. Marketing Intelligence & Planning, 25 (6), 612-631. Gupta, A. (2012). An overview of information technology in tourism industry. International Journal Applied Services Marketing, 1 (1). Hooley, G. J., Saunders, J. A., & Piercy, N. F. (2005). Estratégia de marketing e posicionamento competitivo. São Paulo: Pearson Prentice Hall. Hooper, V. A.; Huff, S. L.; & Thirkell, C. P. (2010, feb.) The impact of IS-marketing alignment on marketing performance and business performance. The DATA BASE for Advances in Information Systems, 41 (1), 36-55. Ismail, S. T. (2011, jan.) The role of marketing information system on decision making: an applied study on Royal Jordanian Air Lines (RJA). International Journal of Business and Social Science, 2 (3), 175-185. Jaworski, B. J., Macinnis, D. J., & Kohli, A. K. (2002). Generating competitive intelligence in organizations. Journal of Marketing, 5 (4), 279- 307. Karamarko, N. (2010). System of tourist destination management as a Croatian identity. Tourism & Hospitality Management - Conference Proceedings, p. 950-966.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 153-168

www.jistem.fea.usp.br

Benefits of the Marketing Information System in the Clothing Retail Business

167

Kotler, P. (1998). Administração de Marketing. São Paulo: Prentice Hall. Kotler, P., & Keller, K. L. (2006). Administração de marketing. São Paulo: Prentice Hall. Kubiak, B. F.; & Kowalik, M. F. (2010, dec.). Marketing information systems as a driver of an organization’s competitive advantage. Journal of Internet Banking and Commerce, 15 (3), 2-10. Lambin, J. J. (2000). Marketing estratégico. Lisboa: McGraw Hill. Laudon, K. C., & Laudon, J. P. (2001). Gerenciamento de sistemas de informação. Rio de Janeiro: LTC. Laudon, K. C., & Laudon, J. P. (2004). Sistemas de informação gerenciais: administrando a empresa digital. São Paulo: Prentice Hall. Levy, M., & Weitz, B. A. (2000). Administração de varejo. São Paulo: Atlas. Lovelock, C., & Wright, L. (2005). Serviços: marketing e gestão. São Paulo: Saraiva. Luz, M. (2008). Paulo Borges fala sobre o mercado da moda. Recuperado em 18 de fevereiro, 2010 de http://simplesmenteelegante.com/?p=1416. Mattar, F. N., Oliveira B., Motta, S. L. S., & Queiroz, M. J. (2009). Gestão de produtos, serviços, marcas e mercados: estratégias e ações para alcançar e manter-se “top of Market”. São Paulo: Atlas. Minciotti, S. A. (1992). O sistema de informações de marketing como suporte para a adoção do marketing estratégico: o desenvolvimento de um modelo. Tese de doutorado, Universidade de São Paulo, São Paulo, SP, Brasil. Ministério do Desenvolvimento, Indústria e Comércio Exterior. (2008). Encontro nacional de comércio e serviços. Recuperado em 04 de novembro, 2010, de http://www.desenvolvimento.gov.br/sitio/interna/noticia.php?area=4&noticia=8606. Ministério do Trabalho e Emprego. (2008). Revista do trabalho. Recuperado em 18 de fevereiro, 2010, de http://www.mte.gov.br/revista/edicao2/txt_complementar1.asp. Pinto F., Marques A., Gago P., & Santos, M. F. Integração da descoberta de conhecimento em bases de dados como suporte a actividades de CRM. (2006, setembro). Anais do Encontro Anual da Associação Nacional de Programas de Pós-Graduação e Pesquisa Em Administração, Salvador, BA, Brasil. Renner. (2009). Mercado brasileiro de varejo. Recuperado em 28 de janeiro, 2010, de http://www.b2i.us/profiles/investor/fullpage.asp?f=1&BzID=1251&to=cp&Nav=0&La ngID=3&s=0&ID=3485. Sisodia, R. S. (1992). Marketing information and decision support systems for services. The Journal of Services Marketing, 6 (1), 51-65. Swartz, T. A., & Iacobucci, D. (Eds). (1999). Handbook of services marketing & management. Thousand Oaks: Sage.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 153-168

www.jistem.fea.usp.br

168 Queiroz, J. P., Oliveira, B.

Vergara, S. C. (2004). Projetos e relatórios de pesquisa em administração. São Paulo: Atlas. Yin, R. K. (2005). Estudo de caso: planejamento e métodos. Porto Alegre: Bookman.

The operational theoretical model used in the field research, by graphic representation, as in the illustrated one, was not found in the most recent editions of the work done by the author.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 153-168

www.jistem.fea.usp.br

JISTEM - Journal of Information Systems and Technology Management Revista de Gestão da Tecnologia e Sistemas de Informação Vol. 11, No. 1, Jan/Apr., 2014 pp. 169-192 ISSN online: 1807-1775 DOI: 10.1590/S1807-17752014000100010

EVALUATION OF MANAGEMENT CONTROL SYSTEMS IN A HIGHER EDUCATION INSTITUTION WITH THE PERFORMANCE MANAGEMENT AND CONTROL Ilse Maria Beuren Universidade Federal do Paraná, Curitiba, Paraná, Brazil Silvio Aparecido Teixeira Universidade Regional de Blumenau, Blumenau, Santa Catarina, Brazil ______________________________________________________________________

ABSTRACT This study aims to investigate the structure and operation of management control systems for performance evaluation in a private higher education institution (HEI). To capture the structure and operation of Management Control Systems (MCS) of the HEI, the survey instrument proposed by Ferreira and Otley was used (2006), called Performance Management and Control (PMC). Thus, an adapted PMC structure questionnaire was sent to 55 managers responsible for strategic actions and plans established in the business of the HEI, having 48 answered questionnaires. Also, four out of the five directors of the business school of the HEI were interviewed, besides the documentary research, for the triangulation of data and to validate the results. In data analysis, the techniques of descriptive statistics and information entropy were used. The results show that the overall level of performance evaluation for MCS reached a mean of 3.62 on the scale, less than partial agreement on the items presented. Some weaknesses were pointed out in the system used, and it was also denoted the need for better alignment among strategy, performance and control. It is concluded that the adapted PMC structure questionnaire managed to capture the structure and operation of MCS for performance evaluation in the HEI. Keywords: Performance evaluation; Management Control Systems; Performance Management and Control; Higher Education Institution.

_____________________________________________________________________________________________ Manuscript first received/Recebido em 03/04/2013 Manuscript accepted/Aprovado em:02/03/2014 Address for correspondence / Endereço para correspondência Ilse Maria Beuren. Doutora em Controladoria e Contabilidade pela FEA/USP. Professora do Programa de PósGraduação em Contabilidade da Universidade Federal do Paraná – UFPR. Address: Av. Prefeito Lothário Meissner, 632 - Campus III – Jardim Botânico. CEP: 80210-070 – Curitiba/PR – Brasil. E-mail: ilse.beuren@gmail.com Phone: (41) 3360-4386. Silvio Aparecido Teixeira. Doutorando do Programa de Pós-Graduação em Ciências Contábeis da Universidade Regional de Blumenau – FURB. Address: Rua Antônio da Veiga, 140 – Sala D 202 - Bairro Victor Konder. CEP 89012-900 – Blumenau/SC – Brasil. E-mail: silvioteixeira@sercomtel.com.br Phone: (47) 3321-0565 Published by/ Publicado por: TECSI FEA USP – 2014 All rights reserved.

170

Beuren, I. M., Teixeira, S. A.

INTRODUCTION

The performance measurement has evolved from financial and non-financial, predominant in the 1990s, to more complex structures, based on the balanced set of measures that seek to align these measures with the organization's strategy. The performance measurement is articulated in an integrated manner, from the strategic level to the operational level, addressing factors that consider the market and the internal costs to support the strategy (Lynch & Cross, 1995). It seeks stakeholdersâ&#x20AC;&#x2122; satisfaction and it contributes to all players by integrating strategy, processes and resources (Neely & Adams, 2001; Neely, Adams & Kennerley, 2002). The integration of the processes and resources into strategy, from the operational to the strategic level, requires management control systems for the evaluation and management of performance. The designs of the management control systems sometimes take the format of causal maps, which show the operational implications for different strategies. However, these are derived, largely, from case studies and consulting experiences (Kaplan & Norton, 1996, 2001; McNair, Lynch & Cross, 1990). The development of research on management control systems requires greater theoretical foundation in field investigations (Chantal, 2003; Covaleski, Evans III, Luft & Shields, 2003). Another criticism is that the research often carries compartmentalized approaches, focused on specific aspects of the design of the management control systems, as opposed to a more comprehensive and integrated vision (Covaleski et al., 2003). In this sense, Ferreira and Otley (2006) adopted the Performance Management and Control (PMC) structure in an empirical study, extending the previous designs of Simons (1995) and Otley (1999) to characterize the MCS in four different European organizations, with the justification that previous studies had shortcomings. In this new design it was developed a more complete instrument for the systems of management control and performance, proposing the PMC figure with 12 questions. It is possible that the reapplication of this model in another segment requires enlarging the structure proposed by Ferreira and Otley (2006), with further details of some specific points of the management control and performance. Based on the above, the following research question was formulated: How are the systems of management control used to systematically measure, monitor and manage performance? In this sense, the goal of this study is to investigate the structure and functioning of management control systems for performance evaluation in a private higher education institution. The survey instrument proposed by Ferreira and Otley (2006) was used to capture the structure and functioning of the Performance Management and Control (PMC). The study is justified by the importance of experiencing the application of the research instrument proposed by Ferreira and Otley (2006) in another context, here in a private higher education institution. The managerial approach, in particular the one related to strategy, is relatively recent in universities and the analysis of the PMC in a community higher education institution is a greater challenge (Meyer Jr, Pascucci & Mangolin, 2012). According to the authors, in these institutions there is strong pressure to revise the management models due to constant changes in the educational policy, demands of the productive sector, variations in the demand for courses and renewed JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Evaluation of Management Control Systems in Higher Education Institution with the Performance Management and Control

171

needs and expectations of the students. However, the main contribution of this study is to adapt the structure of the Performance Management and Control (PMC). The new structure has been redefined in the format of a questionnaire, which allows expanding and accelerating the collection of data in organizations in order to identify weaknesses and blind spots in its management control system to evaluate performance. Its application to a case study in a large higher education institution, with the ongoing strategic plan, performance evaluation system and improvement in management controls support the PMC structure. 2. MANAGEMENT CONTROL SYSTEMS: RELATING CORPORATE STRATEGY TO PERFORMANCE EVALUATION Management Control Systems (MCS), as described in the English literature, acquire and use information in order to assist the coordination of planning and organizational control decisions, with the goal of improving the collective decisions within the organization (Horngren, Foster & Datar, 2000). MCS consider that the controls are characterized by the use in business management, covering performance measurement and reward systems by achieving predetermined levels (Otley, 1999). The managerial control can have different levels of formalization, spontaneity, financial emphasis, varying in each organization according to the system adopted, which will depend on the personal characteristics of their managers, the characteristics of the organization and characteristics of the social and organizational context. The MCS were defined in various ways, having the concept of being the process by which managers assure that resources are obtained and used effectively and efficiently in achieving the goals of the organization (Anthony, 1965). Simons (1995) defines the MCS as means for successful implementation of the strategy. Strategic uncertainties assume that decisions and unusual and difficult to evaluate behavior imply limited and insufficient predictive models for desired results. These factors also result in difficulty controlling the formulation of the planning, the performance evaluation and the necessary corrective actions. MCS were also more generally defined as mechanisms of systematic use of management accounting to achieve a goal, encompassing the use of other types of control, such as personal or cultural controls (Chenhall, 2003). Abernethy and Chua (1996) propose that it is a system that includes a combination of control mechanisms designed and implemented by managers to increase the likelihood that organizational actors will behave in a manner consistent with the organization's goals. Thus, the MCS comprise formal and informal mechanisms and processes used by organizations to measure, monitor and manage their performance in order to implement strategies and achieve their goals. The MCS must have specific characteristics that make them effective, such as the alignment with the strategies and goals of the organization, the compatibility of the organizational structure with the managersâ&#x20AC;&#x2122; responsibility for decision, the motivation to achieve the targets associated with the guidelines of the strategic plan (Horngren, Foster & Datar, 2000). These mechanisms depend on the information flow, which needs to be structured

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

172

Beuren, I. M., Teixeira, S. A.

to meet the systems of people, equipment, procedures, documents and communications, so as to collect, validate, perform operations, transform, store, retrieve and present data to be used in planning, budgeting, accounting, control and other management processes for various administrative purposes (Schwartz, 1999). The structured flow must be able to measure the level of the evaluated performance, relating strategy to organizational results. The strategy is influenced by the external environment, which also affects the design and use of MCS. These, in turn, support the process to enable the achievement of strategic goals. The performance measurement is one of the MCS components (Vieira, Major & Robalo, 2009). 2.1

Performance evaluation

The performance measurement of organizational goals poses a key issue in the management of the organizations. Without these measurement systems, organizations would hardly deal efficiently and effectively with the uncertainty inherent to the environment, resulting in poor use of resources and, consequently, increasing the probability of failure (Vieira, Major & Robalo, 2009). Initially, research questions about strategy were related to the practices of planning and implementation of the plans. Even considering the fact that they are still important questions, insights on the role of strategy in the development of performance measurement systems have emerged as a stream of research in the 1980s (LangfieldSmith, 1997). Several studies have addressed the relationship between strategy and performance and systems of measurement. These studies include the typology of conservative prospection (Miles & Snow, 1978), leadership in costs (Porter, 1985), mission (Gupta & Govindarajan, 1984; Gupta 1987) and strategies of defense (Govindarajan & Gupta, 1985). The relationships between performance measurement systems and strategic uncertainty are pointed out in several studies (Briers & Hirst, 1990). In this scenario, critical success factors include strategies related to developing innovation, in a long term and difficult to quantify, and strategies of defense, associated with low environmental uncertainty and focus on internal stability and efficiency, for which it is easy to develop goals and objective measurements of performance. This expansion of organizational performance measurement systems made a less restrictive and more holistic view of performance emerge. Non-financial indicators gained strength, especially with the appearance of the Balanced Scorecard (BSC), with the binding of the measurement systems to the strategic performance (Kaplan & Norton, 1996, 2000, 2004). The BSC highlights the balance between short and long-term strategies in various dimensions of the businesses, making it a tool to communicate the strategic intent and motivate performance towards the strategic objectives (Ittner & Larcker, 1998). Even with the achievement of practical outcomes, there were restrictions or limitations in the research to the claims and results of the BSC, as the ones presented by Malina and Selto (2001), Ittner, Larcker and Meyer (2003), Banker, Chang, and Pizzini (2004) Bryant, Jones and Widener (2004). Some studies suggest ambiguous results for the BSC, as Chenhall and Langfield-Smith (2007), who reported that the instrument was JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Evaluation of Management Control Systems in Higher Education Institution with the Performance Management and Control

173

part of the best practices of high-performing companies, but which had less well developed management techniques. Despite the limitations, measurements of non-financial performance exceeded the associated costs and included a recent development in performance management. Langfield-Smith, Thorne and Hilton (2006) indicated that such measurements are of difficult choice given the variety of measurements available, besides including the difficult management of trade-off, calculations of low stringency, lack of clarity of its conversion into financial results and tend to proliferate over time. Langfield-Smith, Thorne and Hilton (2006) reported advantages of the nonfinancial measurements, seen as explanatory factors of future performance, as opposed to financial measurements that reflect past performance. Moreover, they classify them as more actionable, i.e., more understandable by the users of the information, and direct the management attention to the roots of the problems and not merely their consequences (Langfield-Smith, Thorne & Hilton, 2006). It is observed that the advancement of non-financial performance measurements, such as those proposed in the BSC, complement the initial conservative framework, being associated with the financial measurements, to compose a measurement system of more robust performance. According to Vieira, Major and Robalo (2009), performance management serves a number of important roles in the organization; however, it must be substantiated by an adequate information flow. 2.2

Information flows

One of the most important roles of the performance measurement is to provide information for decision-making and, therefore, contribute to the creation of value. When influencing the decision making, the system supports the process of planning and control (Vieira, Major & Robalo, 2009). Also, it influences behavior to ensure that it is congruent with the organization's goals. Companies can benefit from the incorporation of ethical standards in MCS (Noreen, 1988; Merchant & Van Der Stede, 2007). Another important role of performance measurement is signaling, both in the internal and external environment. By electing key performance measurements, the organization signals to employees the importance of these strategic aspects. In the external front, the signal to the stakeholders who are part of the organizational environment, with the disclosure of non-financial information regarding performance, such as innovation, operations, levels of customer satisfaction, among others (Vieira, Major & Robalo, 2009). With defined functions, the management of corporate performance measurement should include financial and non-financial measurements, avoiding excessive emphasis on the dimension of operational results over the strategic dimension. Systems should also use the critical performance measurements related to the implementation of the strategy and targeted to areas that contribute to creating and sustaining competitive advantage (Porter, 1985). According to Spendolini (1992), benchmarking, continuous process and evaluation system of product, services, processes and practices of the organization compared to other competitors, is another important feature for the performance measurement system. Represents an external dimension, which allows to identify best JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

174

Beuren, I. M., Teixeira, S. A.

practices and an increased understanding of the processes and learning from the mistakes of other organizations. Thus, results in another feature, which is continuous improvement at the level of performance goals and performance measurement, with its setting from period to period, or the refinement of the measurements themselves (Langfield-Smith, Thorne & Hilton, 2006). Thus, performance management signals the strategic areas susceptible of evaluation for purposes of the MCS, including the determination of managersâ&#x20AC;&#x2122; remuneration or reward plans. The choice of appropriate performance measurements is an exercise that requires attention of those responsible for its design and development. The measurements should have characteristics of validity, reliability, clarity, cost effectiveness, timeliness, accessibility and controllability (Malmi & Brown, 2008). The MCS should be designed to be able to measure and manage organizational performance, aligned with the new concepts of value creation, particularly for groups that play a strategic role in the organization. The need for monitoring performance based on consistent information flows underlies the MCS. In this sense, this study will now cover the structure of performance management and control. 2.3

Structure of Performance Management and Control

The structure of Performance Management and Control (PMC) is a research instrument proposed by Ferreira and Otley (2006), which aims to capture the structure and functioning of Management Control Systems (MCS). The authors took as a reference the previous designs of Simons (1995) and Otley (1999) to characterize the MCS in four different European organizations. They concluded that both proposals had utilities, but also had weaknesses and blind spots. As a result, the authors extended the previous proposal to twelve questions, in order to provide a useful tool for those studying the project and the operation of the performance management control systems. The structure of Simons (1995) proposes four levers of control to study the implementation and control of the business strategy. The key constructs of this framework are based on core values, risk aversion, critical performance variables and strategic uncertainties. Such factors would be controlled by a given system, labeled as levers of controls. The structure of Otley (1999) to study the functioning of the MCS was based on the Theory of Contingency and on his own expertise in field research. It highlighted five main questions to support a coherent structure of performance management systems: identification of the main organizational objectives and processes involved in the evaluation and implementation of these objectives; the formulation and implementation process of strategic plans, as well as their measurement and evaluation of implementation; definition of performance targets and levels of definition; reward systems; and types of information flows required to track performance. Ferreira and Otley (2006) conceived a framework, extending the previous frames of Simons (1995) and Otley (1999). They observed the complementarity of the two studies and the important exploratory nature of the research conducted, but while the two previous structures addressed the issue regarding the conception of the control systems in slightly different formats. In this sense, they developed a new structure, increasing from five to twelve questions to be addressed, as shown in Figure 1.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Evaluation of Management Control Systems in Higher Education Institution with the Performance Management and Control

Theoretical Background

175

Authors

Mission and vision

Simons (1995); Ferreira and Otley (2006).

Key factors of success

Otley (1999); Ferreira and Otley (2006).

Strategy and plans

Simons (1995); Otley (1999); Ferreira and Otley (2006).

Structure of the organization

Ferreira and Otley (2006).

Key Indicators of Performance

Simons (1995); Otley (1999); Ferreira and Otley (2006).

Setting targets

Otley (1999); Ferreira and Otley (2006).

Performance evaluation

Ferreira and Otley (2006).

Reward system

Otley (1999); Ferreira and Otley (2006).

Information flows

Otley (1999); Ferreira and Otley (2006).

10. Types of uses of flows

Simons (1995); Ferreira and Otley (2006).

11. Changes in the PMC system

Ferreira and Otley (2006).

12. Strength components

and

consistency

Ferreira and Otley (2006).

Figure 1 - Theoretical and empirical studies (case studies) that underlie the PMC Source: Adapted and expanded from Ferreira, A., & Otley, D. (2006). The design and use of performance management systems: an extended framework for analysis. Management Accounting Research, 20(4), 263-282.

In Figure 1 it is possible to observe that the first four questions deal with strategy, addressing the dissemination of the vision, mission and goals of the organization, how the activities are implemented to ensure its success and structure available. Questions 5-8 deal with measurements to evaluate performance, its level of adequation and connection with the strategy adopted, the rewards or penalties for the results obtained. Questions 9 and 10 address the information generated by the MCS, with flow of feedback and feedforward type. The last two questions address the consistency, strength and flexibility of the management control systems of performance evaluation to respond to the dynamics of the organization and its environment as well as their consistency and strength. The next topic discusses the methodology and research procedures used to implement the Performance Management and Control (PMC) framework, adapted from Ferreira and Otley (2006), in a case study conducted in a higher education institution.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

176

Beuren, I. M., Teixeira, S. A.

METHODS AND PROCEDURES OF THE RESEARCH

The research was conducted through a case study. The strategy of the case study should be conducted by qualitative research, whose objective is characterized by the analysis of social facts in depth (Martins & The贸philo, 2009). However, Yin (2010) argues that case studies may include quantitative aspects, since the contrast between quantitative and qualitative evidence does not distinguish the various methods of research. Conducting scientific research guided by qualitative and quantitative evaluation is an increasingly common practice, characterized by the description, understanding and interpretation of facts and phenomena (Martins & The贸philo, 2009). These procedures were adopted in this research, also considering the elements of the protocol proposed by Yin (2010). The case study was conducted at a private higher education institution located in southern Brazil. Permission to carry out the research was requested to the senior management of the business area of the HEI, which was obtained with restriction on the disclosure of the corporation name. Firstly, data were collected from documents and information on the HEI websites. Then, non-structured interviews were conducted with four of the five directors of the HEI, to characterize the deliberated strategy, the existing management control system, the types of reward, the performance evaluation and the identification of managers responsible for the business areas of the institution. The interviews conducted in October, 2012 were allowed to be recorded and then transcribed. Subsequently, a structured questionnaire, directed to the managers responsible for the strategic actions and plans established in the School of Business of the institution, was applied to identify the structure of the existing management control that evaluates performance in the institution. 3.1

Research instrument

The instrument of the survey consists in a questionnaire based on the Performance Management and Control (PMC) instrument of Ferreira and Otley (2006), with an increase from twelve to sixteen questions and its transformation into a 5-point Likert scale (Hair Jr, Babin, Money & Samuel, 2005), where level 1 means totally disagree and level 5 totally agree. Levels 2 and 4 represent partially disagree and agree, respectively. Level 3 represents the position of indifferent or not applicable. These adaptations are justified by the geographical location of the respondents, allocated into five distinct units of the HEI. Figure 2 presents a comparison between the structure proposed by Ferreira and Otley (2006) and the adaptation adopted in this research.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Evaluation of Management Control Systems in Higher Education Institution with the Performance Management and Control

The PMC structure (Ferreira and Otley, 2006)

177

The PMC structure (Adapted and redefined in Likert scale)

1. I know the mission and vision stated in 1. What is the vision and mission of the the Strategic Plan of the institution since organization and how is it brought to the they are widely disseminated and attention of managers and employees? presented to managers and employees. 2. I know the goals that are believed as the 2. What are the main factors that are future success of the institution since they believed as essential to the success of are widely disseminated and presented to future global organization and how they managers and employees. are brought to the attention of managers 3. I know the activities and processes and employees? required to ensure the success of the Strategic Plan adopted by the institution 3. What strategies and plans does the organization adopt and what are the processes and activities that it deemed necessary to ensure its success? How are the plans and strategies generated and communicated to managers and employees? 4. What is the structure of the organization and what impact does it have on the design and use of performance management control system? How does it influence and is influenced by the process of implementing the strategy?

4. I received in formal character the strategic plan implemented in the institution through direct communication by the senior management of the institution.

5. I believe that the existing structure allows adopting the system of performance evaluation that influences or is influenced by the Strategic Planning.

5. What are the key performance measurements of the organization 6. The institution adopts key performance resulting from its key goals, success indicators related to the goals and success factors and strategies and plans? How factors of the strategic plan. does the organization evaluate and measure its success? 6. What level of performance is required 7. The targets set to evaluate performance to achieve each of the areas defined in the are appropriate and will help to achieve question above, and how are they the strategic goals. reflected in setting the goals? 7. Which processes does the organization use to evaluate individual, group and organizational performance? How important are information and formal and informal controls in these processes? What are the consequences of the performance evaluation processes in place?

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

8. The existing evaluation process can evaluate individual performance, the performance of the areas and the overall performance of the organization.

www.jistem.fea.usp.br

178

Beuren, I. M., Teixeira, S. A.

The PMC structure (Ferreira and Otley, 2006)

The PMC structure (Adapted and redefined in Likert scale) 9. With the current process of performance evaluation, I have access to rewards (financial and / or non-financial) in a fair and balanced way if performance targets are met.

8. What rewards (financial and nonfinancial) do the managers and other employees will receive by achieving performance targets (or, on the other hand, which penalty will they suffer for 10. With the process of performance failing to achieve them)? evaluation in its existing form, I can also be penalized (financially or otherwise) if performance targets are not achieved. 9. What specific comments and information flows of feedforward has the organization developed? What types of feedback information flows have been created to monitor current performance and adapt the current behavior? What kinds of feedforward information flows (if any) have been formulated to enable the organization to learn from its experience to generate new ideas and recreate strategies and plans?

12. I receive information of diagnostic type (feedback) that allows me to monitor performance after finalizing the results. 13. I receive information of interactive type (feedforward) that allows me to monitor the performance throughout the period, which can generate corrective actions to achieve the goals before finalizing the results.

10. What kind of use is given to the 11. I receive formal and informal feedback and feedforward information information needed to monitor my review flows and various mechanisms of control? of performance (goals). Predominant use of diagnostic, interactive, or the combination of both? 11. How has the performance management control system changed, taking into account the dynamic change of the organization and its environment? What changes have occurred in these systems in anticipation or response to these stimuli?

12. What is the strength and consistency of the connections between the performance components and the management control system (as indicated in the 11 questions above)?

14. I believe that the dynamics of the organization activity and the environment in which it operates enable the performance evaluation to be changed depending on these factors. 15. The dynamics of the activity of the organization and the environment in which it operates may lead to changes in performance evaluation. At other times these changes have already occurred. 16. I believe that the components adopted by the organization for management, control and performance evaluation are extremely strong and consistent.

Figure 2 - Comparison of the structure of Ferreira and Otley (2006) and the structure proposed in this research

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

179

Evaluation of Management Control Systems in Higher Education Institution with the Performance Management and Control

To validate the adaptation of the research instrument, a pilot test was carried out with a professor from another HEI and a professional of the area, who suggested adjustments in the text of some questions. After having permission to administer the questionnaire there was a preliminary contact by email with all the managers responsible for the business area of the institution, preceded by another explanatory email about the motivation of the research, the confidentiality of information and the analysis of the results. The questionnaire with the new PMC structure was sent by e-mail, indicating the electronic link for direct access to the survey questionnaire, using Google Docs tool. After two rounds of contacts, 48 responses were received, from a total of 55 identified managers, representing 87.27% of the population. The steps of data collection in the HEI, with the directors, the distribution and receipt of the questionnaire took place from August to October, 2012. 3.2

Data analysis procedures

Descriptive statistics were used for data analysis, basically the mean of the response, besides the analysis of the dispersion of the response in the scale levels by calculating the information entropy. Information entropy is a simple but important measurement when it comes to a diverse volume of information in a single data source (Zeleny, 1982). The calculation of the entropy has some established steps: Be

the normalized values, where:

d ik

x ik  * , which xi

characterizes the D set, in terms of the i-th attribute. It seeks Di   d ik ; i  1,2,..., n . k 1

Calculating the entropy measurement of the intensity contrast for the i-th attribute, m dk  dk  1 calculated by e(d i )   i Ln i  where    0 and emax=Ln(m). It is e D max i k 1 Di   k k noticed that 0  d i  1 and d i  0 .

d ik 1  and e(di) assumes the Di n 1 maximum value, that is, emax=Ln(m). When setting   , it is determined e max 0  ed i   1for all di´s. This normalization is necessary for comparative purposes. Thus, If all d ik are the same for a given i, then

it establishes that the total entropy of D is defined by: E   ed i  . i 1

It should be noticed that: a) The larger the e(di), the less the information supplied by i-th attribute; b) If (di)=emax=Ln(m), then i-th attribute doesn’t convey information and can be removed from the decision analysis. JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

180

Beuren, I. M., Teixeira, S. A.

Being weight  i inversely related to e(di), it uses 1-e(di) instead of e(di) and it is ~

normalized to ensure that 0   i  1 and

n ~

  i  1 . Thus, the information entropy is i 1

represented by the equation:  i 

1 1  edi   1  ed i  nE nE

For validation and reliability of the research results, it was used the proposal of Creswell (2007, p.200), which determines: make a triangulation of different sources of data, examining the evidences of the sources. Triangulation was performed with unstructured interviews with the senior management of the institution. The results were reinforced by making the transcription of excerpts from the interviews, identified as [D1], [D2], [D3] and [D4], to preserve the confidentiality of the respondents. 4.

DESCRIPTION AND ANALYSIS OF THE RESULTS

4.1

Characterization of the HEI

For confidentiality purposes, the name of the institution is not disclosed, being in this study simply designated as HEI. It is in the Third Sector and has its legal nature framed as a Community institution, founded in 1961. Its location is southern Brazil and is decentralized in five different cities. Its academic structure is divided into nine schools: business, architecture and design, agricultural sciences and veterinary medicine, communication and arts, medicine, healthcare and life sciences, education and humanities, polytechnic and law. Each decentralized unit has a director and each school has an academic dean. Altogether there are 99 undergraduate and technologist courses, about 180 specialized courses, 14 master and 10 doctoral. This set of activities is composed of approximately 1,800 professors, staff of 6,000 people and 26,000 on-campus students. As discussed in the research methodology, this study focuses on the Business School, offered in the five decentralized units. In 1998, the HEI presented its first version of strategic planning, initiating the process of decentralization and the creation of the internalization project, creating units out of its headquarter. The revisions of the strategic plan culminated in the last revision from 2008, which resulted in the project of turning the HEI into a world-class university in a period of fourteen years. With the vision of future defined, several actions were implemented, among them the adoption of a variable remuneration system applicable to managers responsible for the direction and coordination of courses, as well as administrative managers who work in key roles for the strategic performance. In 2009 there was the beginning the process of performance evaluation by goals, related to strategic goals and payment of annual bonuses to reach at least 70 % of the expected performance. After three years of its implementation in the HEI, the performance evaluation system was subsequently amended, proving to be a process of continuous development and improvement. In all schools, about 200 managers are responsible for the strategic JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

181

Evaluation of Management Control Systems in Higher Education Institution with the Performance Management and Control

actions and, therefore, subject to performance evaluation and remuneration by meritocracy. 4.2

Descriptive Statistics

In the first part of the results analysis, it is highlighted the classification of the responses, divided into questions and responses levels according to Likert scale, using TA for Totally Agree, PA for Partly Agree, NA for Not Applicable or indifferent, PD for Partly Disagree and TD for Totally Disagree. The results are shown in Table 1.

Questions

Resp

41.66

43.75

4.17

6.25

4.17

33.34

58.33

0.00

6.25

2.08

20.84

58.33

8.33

4.17

29.17

12.50

16.66

12.50

14.58

52.08

20.84

12.50

0.00

10.42

54.16

14.58

20.84

0.00

10.42

45.83

20.84

16.66

6.25

14.58

37.50

18.75

29.17

0.00

22.92

35.41

18.75

20.84

2.08

Q10

29.17

22.92

16.66

18.75

12.50

Q11

12.50

58.33

8.33

16.67

4.17

Q12

25.00

41.67

18.75

12.50

2.08

Q13

12.50

39.58

27.09

12.50

8.33

Q14

16.67

43.75

25.00

12.50

2.08

Q15

18.75

41.66

20.84

18.75

0.00

Q16

12.50

47.92

18.75

2.08

Total de respondents (Resp) = 48 Table 1 - Descriptive statistics of the research 4.2.1 Analysis of the questions about strategic plans Questions Q1 to Q4 discuss the strategy adopted. In the first statement, Q1, the results show that 41 (85.42%) totally or partially agreed, indicating a reasonable dissemination of the strategic plan. However, 5 (10.42%) respondents totally or partially disagreed with the statement and 2 (4.17%) positioned themselves as indifferent. It shows that approximately 15% of the managers do not know or know very little about

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

182

Beuren, I. M., Teixeira, S. A.

the mission and strategic vision of the HEI. The strategic goals, Q2, for being more related to the goals set, are partially known by 58.33% of the respondents, and fully known by 33.34%. Even this result indicated that the managers have a good level of knowledge of the HEIâ&#x20AC;&#x2122;s strategic goals, 8.33% indicated that there was little or no disclosure of the goals. The activities and processes necessary to achieve the objectives established, Q3, are partially or fully understood by 79.16 % of the respondents. It is important to note that 20.83 % of the managers responded disagree or could not attribute a positive response to this statement. These results are corroborated by question 4, since 41.66 % of the respondents opted for levels 1, 2 or 3, which shows that 22 managers did not have access to the knowledge of the strategy adopted or knew it unofficially. The results of the questions regarding the strategic plan denote that the institution has its mission, vision and goals declared and delineated. However, it has limitations in achieving its dissemination to the entire body of managers of the institution. Such limitations are convergent with the studies of Ferreira and Otley (2006), who found restrictions in this sense in two of the four companies studied. 4.2.2 Analysis of the questions about performance evaluation system Questions Q5 to Q10 deal with performance evaluation systems. The HEIâ&#x20AC;&#x2122;s existing structure, Q5, is identified as totally or partially adequate by 66.66% of the respondents. A significant number of participants responded as indifferent (20.84%) and partially disagreed (12.50%). The key performance indicators are related to the goals and success factors of the strategic plan, Q6. This is an assertion that had partial agreement of 54.16% of the respondents, while 10 (20.84%) believe that some indicators evaluate just a little the strategy adopted. The targets set are partially adequate (Q7) to 45.83 % of the respondents, while 22.91% evaluate the targets as inappropriate or even inadequate. The effectiveness of the existing evaluation, Q8, is contested by 29.17 % of the respondents, while 18.75 % indicated indifference to this question. Only 52.08% believe that the evaluation is complete (14.58%) or partially effective (37.50%). The rewards of the existing performance evaluation process are fair and weighted, Q9, to 22.92 % of the managers, while a large part (35.41%) partially agrees and 20.84% partially disagrees. Another factor in the perception of the managers is that 29.17% totally agree or partially agree (22.92%) that there can be penalties in case of non-achievement of the targets set (Q10). The answers to the questions regarding performance evaluation system indicates the need for improvement, since a considerable part of the managers does not align these measurements with the strategy adopted, and the other part does not know the structure or indicated it as incomplete. The effectiveness and fairness of the evaluation and the possibility of penalty were also questions identified with disabilities. Ferreira and Otley (2006) also observed differences in performance measurements and rewards in the analysis of individuals, groups or teams in the companies studied. The practice of tying rewards to factors of performance and punishment practices were observed in at least two of the four companies of that study.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Evaluation of Management Control Systems in Higher Education Institution with the Performance Management and Control

183

4.2.3 Analysis of the questions about management controls for performance evaluation The mechanisms and controls of the performance evaluation and the existing information flow for monitoring are answered in Q11 to Q16 questions. It is noticed that 70.83% respondents totally or partially receive formal and informal information to track performance and targets (Q11), while 20.84% disagreed and 8.33% indicated level 3 of the scale to the question, which represents indifference. Such information is reported with feedback, Q12, to 25% of the respondents, while 41.67% indicated that sometimes they receive this feedback. For 14.58% of the managers there are deficiencies in the feedback of the results. During the process, this feedback with feedforward information (Q13) falls to 39.58% of the respondents who partially agreed and 12.50% who totally agree, but 27.09% did not know the answer and remained indifferent, while 20.83% totally or partially disagreed. The process of performance evaluation is flexible, having been altered by the activities of the organization and the environment, Q14, having 60.42 % of the respondents totally or partially agreed, while 25% proved to be indifferent. The need for such flexibility is shared, Q15, by 29 managers who perceived the changes occurred, while other 10 (20.84%) and 9 (18.75%) indicated levels 3 and 2 of the scale, respectively. The last question, Q16, indicated that 47.92%, 23 managers, believe that the components adopted by the organization for management, control and performance evaluation is strong, though not always consistent. Other 9 managers (18.75%) disagreed, pointing out that the system is weak and inconsistent, 9 preferred level 3 (indifferent), while 6 (12.50%) consider the system as robust, and only 1 totally disagreed. The answers about management controls indicate performance limitations in providing feedback or feedforward information, as well as moderate deficiencies in the database and availability of information. Ferreira and Otley (2006) also found in their research that the feedback controls were widespread in all companies, because they relate to more operational issues. However, feedforward information was found to be scarce or unavailable and, in that study, a low level of strategic information within the system of control was diagnosed. 4.3

Information entropy

The results were also subjected to the analysis of the information entropy using the questions of the questionnaire in a global manner, by calculating the information entropy and mean. The purpose is to point out the questions that had greater variability of responses, which can point out the lack of alignment of the managers in the respective topic. The column indicating the mean identifies the point of the Likert scale in which the mean of the respondentsâ&#x20AC;&#x2122; opinion is concentrated. Table 2 presents the analysis of the results by information entropy.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

184

Beuren, I. M., Teixeira, S. A.

PMC Structure

Entropy e(di)

ď Źi

Mean

I know the mission and vision stated in the Strategic Plan of the institution, because they are widely disseminated and presented to managers and employees.

0.9901

0.0481

4.13

I know the goals that are considered as the future success of the institution. because they are widely disseminated and presented to managers and employees.

0.9933

0.0328

4.15

I know the activities and processes required to ensure the success of the Strategic Plan adopted by the institution.

0.9899

0.0494

3.83

I informally received the strategic plan implemented in the institution through direct communication by the senior management of the institution.

0.9764

0.1152

3.46

I believe that the existing structure allows adopting the system of performance evaluation that influences or is influenced by the Strategic Planning.

0.9923

0.0374

3.69

The institution adopts key performance indicators that are related to the goals and success factors of the strategic plan.

0.9903

0.0471

3.54

The targets set to evaluate the performance are appropriate and will help to achieve the strategic goals.

0.9853

0.0717

3.38

The existing process of evaluation can evaluate individual performance. performance of areas and the overall performance of the organization.

0.9870

0.0634

3.38

With the current process of performance evaluation. I have access to rewards (financial and /or nonfinancial) in a fair and balanced way if performance targets are met.

0.9863

0.0667

3.56

With the process of performance evaluation in its 10 current format. I can also be penalized (financially or not) if performance targets are not met.

0.9754

0.1200

3.38

I receive the formal and informal information 11 necessary to monitor my performance evaluation (targets).

0.9875

0.0607

3.58

I receive information of diagnostic type that allows 12 me to monitor performance after the results are finalized.

0.9892

0.0527

3.75

13 I receive information of interactive type that allows

0.9840

0.0452

3.35

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

185

Evaluation of Management Control Systems in Higher Education Institution with the Performance Management and Control

Entropy e(di)

i

Mean

I consider that the dynamics of the organization and the environment in which it operates enable that the 14 performance evaluation be changed depending on these factors.

0.9897

0.0452

3.60

The dynamics of the activity of the organization and the environment in which it operates may lead to 15 changes in the performance evaluation. At other times these changes have already occurred.

0.9886

0.0448

3.60

I consider that the components adopted by the organization for management, control and 16 performance evaluation are extremely strong and consistent.

0.9915

0.0332

3.50

PMC Structure

me to monitor the performance throughout the period. which can generate corrective actions to reach the goals before finalizing the results.

Total

1.0000 3.62 Table 2 - Analysis of the results by information entropy

It is observed in Table 2, that the mean of all questions was 3.62, i.e., ranging from level 3 (indifferent) to 4 (partially agree) of the scale. In the two columns of information entropy, Entropy e(di) represents the informational value of the i-th attribute, while ƛi column shows the value of the information entropy, that is, the greater the entropy, the greater the dispersion of the response levels of the scale. 4.3.1 Analysis of the entropy of the questions about strategic plans The block of questions about the Strategic Planning provides the best results in the mean levels of the scale: Q2, Q1 and Q3 obtained responses with mean of 4.15, 4.13 and 3.83, respectively. Q2 had the lowest rate of information entropy among all questions of the survey, with ƛi of 0.0328, showing less dispersion between the levels of response. However, Q4, from the same block, had the second lowest mean,3.46, showing deficiency in formal communication of strategic plans by the senior management. This question also had the second worst rate of information entropy, with ƛi of 0.1152, indicating dispersed responses among all levels of the scale. The results are supported by an interview with a director from the senior management of the School of Business of the HEI, as can be noticed in [D3]’s report: Managers know the mission and vision of the future, it was well worked. But the strategic projects they don’t know well, they are not so clear. [...] There are flaws or lack of communication. Not because of its format, but how it is done and the communication itself. [...] On paper it would not help much, I think the discussion, the explanation is the best way to put everyone in the same boat. It is observed that the results of the questions about the PMC structure, regarding strategic aspects, are aligned with the view of the senior management. The rates of bad entropy, especially the formal communication of strategic plans, are aligned with the JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

186

Beuren, I. M., Teixeira, S. A.

opinion of the managers, which indicated them in the interview as important factors to adjust the direction of the team efforts. These findings are consistent with Ferreira and Otley (2006), who suggest other mechanisms, such as strategic thinking and strategic workshops, to strengthen them. 4.3.2 Analysis of the entropy of the questions about performance evaluation system The questions regarding performance evaluation system indicate better results for Q5, about the structure of the organization, 3.69, which got the fifth best mean of all questions about structure. There was also a slight dispersion in this matter, with ƛi of 0.0374, being considered the third least dispersion among the levels of response. The question of the organization structure was also commented on the triangulation of data through [D4]’s :interview: We don’t have the ideal structure, but the control is sufficient. The ideal would be a more specific software package. [...] There are controls in Excel, reports issued with a certain frequency, but we don’t have a tool that facilitates monitoring, with graphical complement. We don’t have the ideal implemented, but it's enough. The access to rewards by evaluating performance in Q9 got just the ninth best mean, while the penalty for non-achievement of goals, Q10, is recognized as existent by a great part of the managers, once it had mean of 3.38 from the respondents. Q10 also had the worst entropy index of all questions, with ƛi of 0.12, pointing to greater dispersion among respondents, showing that it is unclear at the HEI this component of the performance evaluation system. This issue was addressed by [D2] in the interview: Even when we don’t reach 70 %, like last year, when 50 %, 60% of the targets met, there is no penalty. It causes some frustration, but at the same time represented we did a good job and got close to the target set. I don’t believe there is a penalty. It is clear that a failure in a continued, systematic way leads to a revision of the management condition of that area or sector. The mean of the questions in this block, Q6,3.54, Q7 and Q8, both with mean of 3.38, ranked in 10th, 13th and 14th places, respectively, among the set of 16 questions of the PMC structure proposed in this study. The results of dispersion also had greater dispersion in the levels of responses in relation to the other questions, positioning itself among the last five indexes of information entropy. The results of information entropy show that the system of performance evaluation in the HEI is still unbound and causes suspicion as to its effectiveness and fairness, questioning the access to rewards and, in particular, the likelihood of penalties. The feeling of frustration at not achieving the goals, mentioned in one of the interviews with the senior management, is consistent with the survey results. However, the opinion about penalty is not shared by the board of the institution. The results converge partially with the findings in the companies studied by Ferreira and Otley (2006), when comparing the opinion of the management with the opinion of the factory-floor personnel. 4.3.3 Analysis of the entropy of the questions about management controls for performance evaluation In the block of questions about the management control to evaluate the JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Evaluation of Management Control Systems in Higher Education Institution with the Performance Management and Control

187

performance, the best result was for Q12, which deals with the feedback of the results achieved, with 3.75 on the scale and the 4th best overall response level. The monitoring during the process, Q13, had 3.35. Receiving formal and informal information,Q11, had Ć&#x203A;i of 0.067, showed great dispersion and had the 5th worst entropy index. Information concerning these questions was discussed by [D1]: We have many difficulties to get information. There have been improvements, but we are still far from getting them with ease and agility. It would be ideal to have a panel of indicators permanently available in the system. We have a database problem, and sometimes the data are not coincident. The results indicate the need for improvements in the feedback and access to the information. Feedback information is more present than those of feedforward, but it is no consensus, since there was dispersion in the perception of the respondents. This need is shared by the senior management, which also offers suggestions and criticism to the database in the interview. The findings are consistent with the study of Ferreira and Otley (2006). Questions Q14 and Q15, about the dynamics and flexibility of the performance evaluation system, achieved mean of 3.60, while receiving formal and informal information had mean of 3.58. Last question (Q16) had mean of 3.50 and highlights the need to strengthen the management control system and components of performance evaluation and control. These aspects may also be observed in [D1]â&#x20AC;&#x2122;s report: It would be ideal that all employees, without distinction of their role, even the simplest employee of the institution, had access to some kind of reward for the overall results achieved. That's my view of the system. But we are improving; in three years since the implementation of performance evaluation, it has already been verified significant improvements in the process and we are evolving. It is a process of continuous improvement. It is inferred that, despite the flexibility of the system and the changes made in the last three years at the MCS of the HEI, the indexes of information entropy have revealed room for improvement, as the HEI has not achieved the ideal model yet. The results converge partially with those found in the study of Ferreira and Otley (2006). It is noteworthy that the information entropy highlights questions in which the dispersion of the responses had a greater level, as already discussed, which not only presents deficiencies of alignment with the strategic plan, but provides opportunities for the managers of the institution to take corrective actions. 5.

FINAL CONSIDERATIONS

The goal of this study was to investigate the structure and functioning of management control systems for performance evaluation in a private higher education institution. With the Performance Management and Control (PMC) structure proposed by Ferreira and Otley (2006) to capture the structure and functioning of the MCS, the survey instrument was adapted with questions converted to the five-point Likert scale in order to expand and accelerate the data collection in the HEI object of the case study. JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

188

Beuren, I. M., Teixeira, S. A.

The questionnaire was sent to 55 managers responsible for the strategic actions and plans established in the School of Business of the HEI, having 48 responses, representing 87.27% of the population. Also, four directors of the HEIâ&#x20AC;&#x2122;s business school were interviewed, besides the documentary research, for the triangulation of the data and to validate the results. In the data analysis the techniques of descriptive statistics and information entropy were used. Its application to a Higher Education Institution, which has strategic plan implemented and under constant review, as well as a system of performance evaluation and management control in process of implementation, allowed to point out the weaknesses of the system. The use of information entropy brought robustness to the analysis of the data, as it demonstrated the dispersion of the responses in some fundamental aspects. The highest levels of information entropy are in the questions about the possibility of penalties for the non- achievement of the goals and official communication about strategic planning from the senior management to the managers. The study found that the HEI has to promote a better alignment with the business strategy, the system of performance evaluation and management control systems. The overall mean was 3.62 on a five-point Likert scale, indicating that the agreement, at least partial, of most items was not achieved by the respondents of the HEI. Only two questions, one about the knowledge of the mission and vision, and other about the objectives of the strategy adopted, surpassed4 points on the scale. Analyzing the questions in blocks, the best results were obtained from the questions about the strategic plan, especially because it is a process in place for over a decade. It is noteworthy that, among the questions in this block, deficiency in the communication of the plan by senior management was one of the negative items listed in the survey, having the second worst dispersion in the levels of response. Another factor is that 7 of the 48 respondents expressed "indifference" to the Likert scale point, fact that can represent ignorance of the matter addressed, lack of opinion or uncertainty on the issue. In this sense, it is recommended that the institution take specific action to identify the weakness found. The block of questions concerning the measurements of the performance evaluation system, its adequation and ability to really assess the overall results, per unit and individually, the rewards and penalties, was rated the worst block and the questions had the lowest means. The greatest dispersion of entropy was due to the likelihood of penalty for the non-achievement of the targets, exceeding all of the other items surveyed. The questions in the block regarding management control for performance evaluation had intermediate achievements, between strategy and performance evaluation. It was evident, especially in the interviews that triangulated with the results of the questionnaires, that the information is not easily obtained, especially feedback and feedforward information. The testimonies of the interviews also show the need for adjustments in the database and in the agility and availability of the strategic information. In conclusion, with the application of the structure adapted from the PMC, it was possible to identify in the case study proposed how the management control system is being used in the higher education institution surveyed to measure, control and manage JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Evaluation of Management Control Systems in Higher Education Institution with the Performance Management and Control

189

the performance. Thus, the structure and functioning of management control systems used to evaluate performance in the HEI were identified. Some weaknesses were found in the system used, as well as it was indicated the need to improve the alignment among strategy, performance and control. As a contribution, in addition to the new PMC format here adopted, the usage and analysis of the data by entropy, the triangulation with the board of the organization, the framework proposed by Ferreira and Otley (2006) was applied in a complex environment, as higher education institutions are. However, given the limitations of this study, since it was applied exclusively to the business school of the institution, for future research new applications of the research instrument are recommended, as the amended structure of the one proposed by Ferreira and Otley (2006). Such applications may occur in other segments, in multi-case studies, as well as with the application of statistical methods.

REFERENCES Abernethy, M. A., & Chua, W. F. (1996). A field study of control system redesigns: the impact of institutional process on strategic choice. Contemporary Accounting Research, 13(2), 569-606. Anthony, R. (1965). Planning and control systems: a framework for analysis. Boston: Harvard Business Press. Banker, R. D., Chang, H., & Pizzini, M. J. (2004). The balanced scorecard: judgmental effects of performance measures linked to strategy. The Accounting Review, 79(1), 1-23. Briers, M., & Hirst, M. (1990). The role of budgetary information in performance evaluation. Accounting, Organizations and Society, 15(4), 373-398. Bryant, L., Jones, D. A., & Widener, S. K. (2004). Managing value creation within the firm: an examination of multiple performance measures. Journal of Management Accounting Research, 16(1), 107-132. Chenhall, R. H. (2003). Management control systems design within its organizational context: findings from contingency-based research and directions for the future. Accounting, Organizations and Society, 28(2-3), 127-168. Chenhall, R. H., & Langfield-Smith, K. (2007). Multiple perspectives of performance measures. European Management Journal, 25(4), 266-282. Covaleski, M. A., Evans III, J. H., Luft, J., & Shields, M. D. (2003). Budgeting research: three theoretical perspectives and criteria for selective integration. Journal of Management Accounting Research, 15(3), 3-47. Creswell, J. W. (2007). Projeto de pesquisa: mĂŠtodos qualitativo, quantitativo e misto. 2. ed. Porto Alegre: Artmed. JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

190

Beuren, I. M., Teixeira, S. A.

Ferreira, A., & Otley, D. (2006). The design and use of performance management systems: an extended framework for analysis. Management Accounting Research, 20(4), 263-282. Govindarajan, V., & Gupta, A. K. (1985). Linking control systems to business unit strategy: impact on performance. Accounting, Organizations and Society, 20(1), 51-66. Gupta, A. K. (1987). SBU strategies, corporate-SBU, and SBU effectiveness in strategy implementation. Academy of Management Journal, 30(3), 477-500. Gupta, A .K., & Govindarajan, V. (1984). Business unit strategy, managerial characteristics, and business unit effectiveness at strategy implementation. Academy of Management Journal, 27(1), 25-41. Hair Jr, J. F., Babin, B., Money, A. H., & Samuel, P. (2005). Fundamentos de métodos de pesquisa em administração. Porto Alegre: Bookman. Horngren, C. T., Foster, G., & Datar, S. (2000). Contabilidade de custos. Rio de Janeiro: LTC. Ittner, C. D., & Larcker, D. E. (1998). Innovations in performance measurement: trends and research implications. Journal of Management Accounting Research, 10(1), 205238. Ittner, C. D., Larcker, D. F., & Meyer, M. W. (2003). Subjectivity and the weighting of performance measures: evidence from a balanced scorecard. The Accounting Review, 78(3), 725-758. Kaplan, R. S., & Norton, D. P. (1996). The Balanced Scorecard: translating strategy into action. Boston, MA: Harvard Business School Press. Kaplan, R. S., & Norton, D. P. (2000). The strategy focused organization: how Balanced Scorecard companies thrive in the new business environment. Boston, MA: Harvard Business School Press. Kaplan, R. S., & Norton, D. P. (2004). Strategy maps: converting intangible assets into tangible outcomes. Boston, MA: Harvard Business School Press. Langfield-Smith, K. (1997). Management control systems and strategy: a critical review. Accounting, Organizations and Society, 22(2), 207-232. Langfield-Smith, K., Thorne, H. T., & Hilton, R. W. (2006). Management Accounting: information for managing and creating value. 4 ed. North Ryde, Australia: McGrawHill. Lynch, R. L., & Cross, K. F. (1995). Measure up! yardsticks for continuous improvement. Cambridge, MA: Blackwell Inc. Malina, M. A., & Selto, F. H. (2001). Communicating and controlling strategy: An empirical study of the effectiveness of the balanced scorecard. Journal of Management Accounting Research, 13(1), 47-90. Malmi, T., & Brown, D. (2008). Management control systems as a package – JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Evaluation of Management Control Systems in Higher Education Institution with the Performance Management and Control

191

opportunities, challenges and research directions. Management Accounting Research, 19(4), 287-300. Martins, G. A., & Theóphilo, C. R. (2009). Metodologia da investigação científica para ciências sociais aplicadas. 2 ed. São Paulo: Editora Atlas. McNair, C. J., Lynch, R. L., & Cross, K. F. (1990). Do financial and nonfinancial performance measures have to agree? Management Accounting Research, 72(5), 28-36. Merchant, K. A., & Van Der Stede, W. A. (2007). Management control systems. 2. ed. Harlow: Pearson Education Limited. Miles, R. W.; Snow, C. C. (1978). Organizational strategy, structure and process. New York: McGraw-Hill. Neely, A., & Adams, C. (2001). Perspectives on performance: the performance prism. Journal of Cost Management, 15(1), 7-15. Neely, A., Adams, C., & Kennerley, K. (2002). The performance prism: the scorecard for measuring and managing business success. London: Financial Times Prentice Hall. Noreen, E. (1988). The economics of ethics: a new perspective on agency theory. Accounting, Organizations and Society, 13(4), 359-369. Otley, D. (1999). Performance management: a framework for management control systems research. Management Accounting Research, 10(10), 363-382. Porter, M. E. (1985). Competitive advantage: creating and sustaining superior performance. New York, NY: Free Press. Schwartz, G. (1970). Science in marketing. New York, NY: John Wiley. Simons, R. (1995). Levers of control: how managers use innovative control systems to drive strategic renewal. Boston: Harvard Business School Press. Spendolini, M. (1992). The benchmarking book. New York, NY: Amacon. Vieira, R., Major, M. J., & Robalo, R. (2009). Investigação qualitativa em Contabilidade. In: Major, M. J., & Vieira, R. (Ed.). Contabilidade e controlo de gestão: teoria, metodologia e prática. Lisboa: Escolar Editora. 2009. pp 301-331. Yin, R. K. (2010). Estudo de caso: planejamento e métodos. 4. ed. Porto Alegre: Bookman. Zeleny, M. (1982). Multiple criteria decision making. New York: McGraw-Hill.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

192

Beuren, I. M., Teixeira, S. A.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

AVALIAÇÃO DOS SISTEMAS DE CONTROLE GERENCIAL EM INSTITUIÇÃO DE ENSINO SUPERIOR COM O PERFORMANCE MANAGEMENT AND CONTROL Ilse Maria Beuren Universidade Federal do Paraná, Curitiba, Paraná, Brazil Silvio Aparecido Teixeira Universidade Regional de Blumenau, Blumenau, Santa Catarina, Brazil ______________________________________________________________________ ABSTRACT Este estudo objetiva averiguar a estrutura e o funcionamento dos sistemas de controle gerencial para avaliação de desempenho em uma instituição de ensino superior privada. Para captar a estrutura e funcionamento dos Sistemas de Controle Gerencial (SCG) da IES utilizou-se o instrumento de pesquisa proposto por Ferreira e Otley (2006), denominado Performance Management and Control (PMC). Assim, um questionário com a estrutura PMC adaptada foi enviado aos 55 gestores responsáveis pelas ações estratégicas e planos estabelecidos na área de negócios da IES, obtendo-se 48 questionários respondidos. Também foram entrevistados quatro dos cinco diretores da escola de negócios da IES, além de pesquisa documental, para triangulação dos dados e validação dos resultados. Na análise dos dados utilizaramse as técnicas de estatística descritiva e entropia da informação. Os resultados mostram que o nível geral dos SCG para avaliação de desempenho alcançou média 3,62 na escala, inferior à concordância parcial nos quesitos apresentados. Foram apontados alguns pontos fracos no sistema utilizado, bem como se denotou a necessidade de melhor alinhamento entre estratégia, desempenho e controle. Conclui-se que o questionário com a estrutura PMC adaptada conseguiu captar a estrutura e o funcionamento dos SCG para avaliação de desempenho na IES. Keywords: Avaliação de desempenho. Sistemas de controle gerencial. Performance Management and Control. Instituição de ensino superior.

170 Beuren, I. M., Teixeira, S. A.

INTRODUÇÃO

A mensuração do desempenho tem evoluído de medidas financeiras e não financeiras, predominantes na década de 1990, para estruturas mais complexas, baseadas no conjunto equilibrado de medidas que buscam alinhar essas medidas à estratégia da organização. A mensuração do desempenho se articula de maneira integrada, do nível estratégico ao nível operacional, abordando fatores que consideram o mercado e os custos internos para suportar a estratégia (Lynch & Cross, 1995). Vislumbra a satisfação dos stakeholders e a contribuição às partes interessadas, integrando a estratégia, os processos e os recursos (Neely & Adams, 2001; Neely, Adams & Kennerley, 2002). A integração dos processos e recursos à estratégia, do nível operacional ao nível estratégico, requer sistemas de controle gerencial para efeitos da avaliação e gestão do desempenho. Os desenhos dos sistemas de controle gerencial às vezes tomam a forma de mapas causais, que mostram as implicações operacionais para estratégias diferentes. No entanto, estes são provenientes, em grande parte, de estudos de casos e experiências de consultoria (Kaplan & Norton, 1996; 2001; McNair, Lynch & Cross, 1990). O desenvolvimento das pesquisas em sistemas de controle gerencial necessita de maior embasamento teórico nas investigações de campo (Chenhall, 2003; Covaleski, Evans III, Luft & Shields, 2003). Outra crítica é que a pesquisa carrega quase sempre abordagens compartimentadas, focadas em aspectos específicos de design dos sistemas de controle gerencial, em oposição a uma visão mais abrangente e integrada (Covaleski et al, 2003). Neste sentido, Ferreira e Otley (2006) adotaram a estrutura Performance Management and Control (PMC) em um estudo empírico, estendendo os desenhos anteriores de Simons (1995) e de Otley (1999) para caracterizar os SCG em quatro diferentes organizações europeias, com a justificativa que os estudos anteriores apresentavam lacunas. Nesse novo desenho desenvolveram um instrumento mais completo para os sistemas de controle gerencial e desempenho, propondo o quadro PMC com 12 questões. É possível que a reaplicação deste modelo em outro segmento implique ampliar a estrutura proposta por Ferreira e Otley (2006), com maior detalhamento de alguns pontos específicos de controle gerencial e desempenho. Com base no exposto elaborou-se a seguinte questão de pesquisa: Como os sistemas de controle gerencial são utilizados sistematicamente para medir, controlar e gerir o desempenho? Neste sentido, o estudo objetiva averiguar a estrutura e o funcionamento dos sistemas de controle gerencial para avaliação de desempenho em uma instituição de ensino superior privada. Utilizou-se o instrumento de pesquisa proposto por Ferreira e Otley (2006) para captar a estrutura e o funcionamento dos SCG, denominado Performance Management and Control (PMC). O estudo se justifica pela importância de experimentar a aplicação do instrumento de pesquisa proposto por Ferreira e Otley (2006) em outro contexto, no caso em uma instituição de ensino superior privada. A abordagem gerencial, em especial a vinculada à estratégia, é relativamente recente em organizações universitárias e a análise do SCG em uma instituição de ensino superior comunitária é desafio maior (Meyer Jr, Pascucci & Mangolin, 2012). Segundo os autores, nessas instituições há forte pressão para revisão dos modelos de gestão, por causa das constantes mudanças na política educacional, demandas do setor produtivo, flutuações na demanda por cursos e JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Avaliação dos Sistemas de Controle Gerencial em Instituição de Ensino Superior com o Performance Management and Control

171

renovadas necessidades e expectativas dos alunos. No entanto, a principal contribuição do estudo está na adaptação da estrutura do Performance Management and Control (PMC). A nova estrutura foi redefinida em formato de questionário, o que permite ampliar e acelerar a coleta de dados nas organizações, de modo a identificar os pontos fracos e cegos do seu sistema de controle gerencial para avaliar o desempenho. Sua aplicação em um estudo de caso em instituição de ensino superior de grande porte, com o plano estratégico, sistema de avaliação de desempenho e melhoria nos controles gerenciais em pleno curso, permite o suporte à estrutura PMC. 2. SISTEMAS DE CONTROLE GERENCIAL: LIGANDO A ESTRATÉGIA EMPRESARIAL A AVALIAÇÃO DE DESEMPENHO Os Sistemas de Controle Gerencial (SCG), designados na literatura inglesa de Managment Control Systems (MCS), devem obter e utilizar a informação, com o intuito de auxiliar a coordenação das decisões de planejamento e controle organizacional, com vistas à melhoria das decisões coletivas dentro da organização (Horngren, Foster & Datar, 2000). No SCG considera-se que os controles se caracterizam pelo uso na gestão empresarial, contemplando a mensuração de desempenho e o sistema de recompensas pelo alcance de níveis preeestabelecidos (Otley, 1999). O controle gerencial pode ter níveis diferentes de formalização, espontaneidade, ênfase financeira, variando em cada organização pela configuração do sistema adotado, que dependerá das características pessoais de seus administradores, das características da organização e das características do contexto social e organizacional. Os sistemas de SCG foram definidos de várias maneiras, como a conceituação de que é o processo pelo qual os gestores garantem que os recursos são obtidos e utilizados com eficácia e eficiência na realização dos objetivos da organização (Anthony, 1965). Simons (1995) define os SCG como um meio para a implementação com sucesso da estratégia. As incertezas estratégicas supõem que as decisões e o comportamento pouco rotineiro e de difícil avaliação implicam em modelos preditivos limitados e insuficientes para resultados desejados. Esses fatores resultam também na dificuldade de controlar a formulação do planejamento, a avaliação de desempenho e as ações corretivas necessárias. Os SCG também foram definidos mais genericamente como mecanismos de uso sistemático da contabilidade gerencial para alcançar algum objetivo, mas também englobando a utilização de outras formas de controle, como controles pessoais ou culturais (Chenhall, 2003). Abernethy e Chua (1996) propõem que se trata de um sistema que engloba uma combinação de mecanismos de controle concebidos e implementados pelos gestores para aumentar a probabilidade de que os atores organizacionais irão comportar-se de forma consistente com os objetivos da organização. Assim, os SCG compreendem mecanismos formais e informais e processos utilizados pelas organizações para medir, controlar e gerir o seu desempenho, a fim de implementar estratégias e atingir os seus objetivos. Os SCG devem possuir características específicas que os tornem efetivos, como o alinhamento com as estratégias e metas da organização, a compatibilidade da estrutura organizacional com a JISTEM, Brazil Vol. 11, No.1, Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

172 Beuren, I. M., Teixeira, S. A.

responsabilidade de decisão dos gestores, a motivação para o cumprimento das metas associada às orientações do plano estratégico (Horngren, Foster & Datar, 2000). Esses mecanismos dependem do fluxo de informações, que precisa ser estruturado para atender os sistemas de pessoas, equipamentos, procedimentos, documentos e comunicações, de modo que coleta, valida, executa operações, transforma, armazena, recupera e apresenta dados para uso no planejamento, no orçamento, na contabilidade, no controle e em outros processos gerenciais para vários propósitos administrativos (Schwartz, 1999). O fluxo estruturado deve ser capaz de mensurar o nível de desempenho avaliado, fazendo a ligação da estratégia aos resultados organizacionais. A estratégia é influenciada pelo ambiente externo, o qual também afeta a concepção e o uso dos SCG. Estes, por sua vez, subsidiam o processo para viabilizar o alcance dos objetivos estratégicos. A mensuração do desempenho é um dos componentes do SCG (Vieira, Major & Robalo, 2009). 2.1

Avaliação de desempenho

A mensuração de desempenho dos objetivos da organização surge como questão fundamental na gestão das organizações. Sem estes sistemas de medição, as organizações dificilmente lidariam de forma eficiente e eficaz com a incerteza inerente ao ambiente, resultando em utilização deficiente dos recursos e, consequentemente, no aumento da probabilidade de insucesso (Vieira, Major & Robalo, 2009). Inicialmente as questões de pesquisa em estratégia estavam ligadas às práticas de planejamento e implementação dos planos. Mesmo considerando essas questões ainda importantes, insights sobre o papel da estratégia no desenvolvimento de sistemas de mensuração de desempenho surgiram como uma corrente de investigação na década de 1980 (Langfield-Smith, 1997). Vários estudos abordaram a relação entre estratégia e desempenho e sistema de avaliações. Esses estudos incluem a tipologia de prospecção conservadora (MILES; SNOW, 1978), liderança em custos (Porter, 1985), missão (Gupta & Govindarajan, 1984; Gupta, 1987) e estratégias de defesa (Govindarajan & Gupta, 1985). As relações entre sistemas de medição de desempenho e incerteza estratégica são apontadas em vários estudos (BRIERS; HIRST, 1990). Nestas condições, os fatores críticos de sucesso incluem estratégias relacionadas ao desenvolvimento de inovação, de longo prazo e de difícil quantificação, e as estratégias de defesa, associadas com baixa incerteza ambiental e foco na estabilidade e eficiência interna, para as quais é fácil desenvolver metas e medidas objetivas de desempenho. Essa expansão dos sistemas de medição de desempenho organizacional fez surgir uma visão menos restritiva e mais holística do desempenho. Os indicadores não financeiros ganharam força, em especial com o aparecimento do Balanced Scorecard (BSC), com a vinculação da medição de desempenho aos sistemas estratégicos (Kaplan & Norton, 1996; 2000; 2004). O BSC destaca o equilíbrio entre medidas de curto e longo prazo nas várias dimensões estratégicas dos negócios, tornando-se um instrumento para comunicar a intenção estratégica e motivar o desempenho para os objetivos estratégicos (Ittner & Larcker, 1998). Mesmo com o alcance de resultados práticos, houve restrições ou limitações nas pesquisas para as reivindicações e resultados do BSC, como os apresentados por Malina JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Avaliação dos Sistemas de Controle Gerencial em Instituição de Ensino Superior com o Performance Management and Control

173

e Selto (2001), Ittner, Larcker e Meyer (2003), Banker, Chang e Pizzini (2004), Bryant, Jones e Widener (2004). Alguns estudos sugerem resultados ambíguos para o BSC, como Chenhall e Langfield-Smith (2007), que relataram que o instrumento fazia parte das melhores práticas de empresas de alto desempenho, mas que tinham menos técnicas de gestão bem desenvolvidas. Apesar das limitações, as medidas de desempenho não financeiras superaram os custos associados e compreendem um desenvolvimento recente no gerenciamento de desempenho. Langfield-Smith, Thorne e Hilton (2006) apontam que tais medidas são de difícil escolha, dada a multiplicidade de medidas disponíveis, além de incluírem tradeoff de difícil gestão, cálculos de baixo rigor, falta de clareza de sua conversão em resultados financeiros e tendem a proliferar com o passar do tempo. Langfield-Smith, Thorne e Hilton (2006) reportam vantagens das medidas não financeiras, sendo vistas como fatores explicativos de desempenho futuro, em contraposição às medidas financeiras que refletem o desempenho passado. Além disso, classificam-nas como mais acionáveis, ou seja, são mais compreensíveis pelos usuários da informação e dirigem a atenção dos gestores para as causas dos problemas e não meramente para as suas consequências (Langfield-Smith, Thorne & Hilton, 2006). Denota-se que o avanço das medidas de desempenho não financeiras, como as propostas no BSC, complementa o quadro conservador inicial, se associando às medidas financeiras, para compor um sistema de medição de desempenho mais robusto. Para Vieira, Major e Robalo (2009), o gerenciamento do desempenho serve para um conjunto de funções importantes na organização, todavia precisa estar consubstanciado por um fluxo adequado de informações. 2.2

Fluxos de informações

Uma das mais relevantes funções da mensuração do desempenho é proporcionar informação à tomada de decisão e, por consequência, contribuir para a criação de valor. Ao influenciar a tomada de decisão, o sistema apoia o processo de planejamento e controle. (Vieira, Major & Robalo, 2009). Também tem a função de influenciar o comportamento no sentido de assegurar que este seja congruente com os objetivos da organização. As empresas podem se beneficiar com a incorporação de normas éticas em SCG (Noreen, 1988; Merchant & Van Der Stede, 2007). Outra função relevante da mensuração do desempenho é a sinalização, tanto no ambiente interno como no externo. Ao eleger as medidas-chaves de desempenho, a organização sinaliza aos colaboradores a importância desses aspectos estratégicos. Em âmbito externo, há sinalização para os stakeholders que fazem parte do ambiente organizacional, com a evidenciação de informações não financeiras relativas ao desempenho, como inovação, operações, níveis de satisfação de clientes, entre outras (Vieira, Major, & Robalo, 2009). Com as funções definidas, o gerenciamento da medição de desempenho das empresas deve incluir medidas financeiras e não financeiras, evitando ênfase excessiva na dimensão de resultados operacionais em detrimento da dimensão estratégica. Os sistemas devem ainda utilizar medidas de desempenho criticas, associadas à implementação da estratégia e orientadas para as áreas que contribuem à criação e sustentação de vantagem competitiva (Porter, 1985). Segundo Spendolini (1992), o benchmarking, processo contínuo e sistema de JISTEM, Brazil Vol. 11, No.1, Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

174 Beuren, I. M., Teixeira, S. A.

avaliação de produtos, serviços, processos e práticas da organização em comparação com outros concorrentes, é outra característica importante para o sistema de medição de desempenho. Representa uma dimensão externa, que permite identificar melhores práticas e um aumento na compreensão dos processos e aprendizagem com erros de outras organizações. Dessa forma, resulta em outra característica, que é melhoria contínua ao nível das metas de desempenho e da medição de desempenho, com o seu estabelecimento de período a período ou o refinamento das próprias medidas (Langfield-Smith, Thorne & Hilton, 2006). Desta forma, o gerenciamento de desempenho sinaliza as áreas estratégicas suscetíveis de avaliação para fins do SCG, inclusive para a determinação de remuneração ou dos planos de recompensa dos gestores. A escolha das medidas de desempenho adequadas é um exercício que exige atenção dos responsáveis pela sua concepção e seu desenvolvimento. As medidas devem ter características de validade, confiabilidade, clareza, eficácia de custo, tempestividade, acessibilidade e controlabilidade (Malmi & Brown, 2008). Os SCG devem ser concebidos para conseguir medir e gerenciar o desempenho organizacional, alinhados aos novos conceitos de criação de valor, em especial para os grupos que desempenham papel estratégico na organização. A necessidade de monitorar o desempenho com base em fluxos de informações compatíveis está na base dos SCG. Neste sentido aborda-se a seguir a estrutura de gerenciamento de desempenho e controle. 2.3

Estrutura de Gerenciamento de Desempenho e Controle

A estrutura de Gerenciamento de Desempenho e Controle (Performance Management and Control – PMC) é um instrumento de pesquisa proposto por Ferreira e Otley (2006), que busca captar a estrutura e o funcionamento dos Sistemas de Controle Gerencial (SCG) (Managment Control Systems MCS). Os autores tomaram como referência os desenhos anteriores de Simons (1995) e Otley (1999) para caracterizar os SCG em quatro diferentes organizações europeias. Concluíram que as duas propostas apresentavam utilidades, mas denotavam pontos fracos e cegos. Como resultado, os autores ampliaram a proposta anterior para 12 questões, com a finalidade de fornecer um instrumento útil para aqueles que estudam o projeto e a operação dos sistemas de controle gerencial e desempenho. A estrutura de Simons (1995) propõe quatro alavancas de controle para estudar a implementação e o controle da estratégia de negócios. As construções-chaves desta estrutura se baseiam em núcleo de valores, aversão aos riscos, variáveis de desempenho críticas e incertezas estratégicas. Tais fatores seriam controlados por um determinado sistema, rotulados como alavancas de controles. A estrutura de Otley (1999) para estudar o funcionamento dos SCG foi fundamentada na Teoria da Contingência e na sua própria expertise em pesquisa de campo. Destacou cinco questões centrais para fundamentar uma estrutura coerente de sistemas de gestão de desempenho: identificação dos principais objetivos organizacionais e processos envolvidos na avaliação da execução desses objetivos; processo de formulação e execução dos planos estratégicos, bem como sua medição e avaliação de implementação; definição de metas de desempenho e níveis de definição; sistemas de recompensas; e tipos de fluxos de informações necessárias para acompanhar o desempenho. JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Avaliação dos Sistemas de Controle Gerencial em Instituição de Ensino Superior com o Performance Management and Control

175

Ferreira e Otley (2006) conceberam um quadro normativo, ampliando os quadros anteriores de Simons (1995) e Otley (1999). Observaram a complementaridade dos dois estudos e a natureza exploratória importante das pesquisas realizadas, mas ao mesmo tempo as duas estruturas anteriores abordavam a questão da concepção de sistemas de controle de forma ligeiramente diferentes. Neste sentido desenvolveram uma nova estrutura, aumentando de cinco para 12 questões a serem abordadas, conforme demonstrado na Figura 1. Fundamentos Teóricos

Autores

1. Missão e visão

Simons (1995); Ferreira e Otley (2006).

Otley (1999); Ferreira e Otley (2006).

Fatores-chaves de sucesso

3. Estratégia e planos

Simons (1995); Otley (1999); Ferreira e Otley (2006).

4. Estrutura da organização

Ferreira e Otley (2006).

5. Indicadores-chaves de desempenho

Simons (1995); Otley (1999); Ferreira e Otley (2006).

6. Definição de metas

Otley (1999); Ferreira e Otley (2006).

7. Avaliação de desempenho

Ferreira e Otley (2006).

Otley (1999); Ferreira e Otley (2006).

Sistema de recompensa

9. Fluxos de informação

Otley (1999); Ferreira e Otley (2006).

10. Tipos de uso dos fluxos

Simons (1995); Ferreira e Otley (2006).

11. Mudanças no sistema PMC

Ferreira e Otley (2006).

12. Força e coerência dos componentes

Ferreira e Otley (2006).

Figura 1 - Estudos teóricos e empíricos (estudos de casos) que fundamentaram o PMC Fonte: Adaptado e ampliado de Ferreira, A., & Otley, D. (2006). The design and use of performance management systems: an extended framework for analysis. Management Accounting Research, 20(4), 263-282. Observa-se na Figura 1 que as quatro primeiras questões tratam da estratégia, abordando a disseminação da visão, missão e objetivos da organização, como são implementadas as atividades para garantir seu sucesso e estrutura disponível. As questões 5 a 8 tratam das medidas para avaliar o desempenho, seu nível de adequação e ligação com a estratégia adotada, as recompensas ou penalidades pelos resultados obtidos. As questões 9 e 10 abordam as informações geradas pelo SCG, com fluxo do tipo feedback e feedforward. As duas últimas questões abordam a coerência, força e a flexibilidade dos sistemas de controle gerencial de avaliação de desempenho para responder à dinâmica da organização e do seu ambiente, bem como a sua coerência e força. No tópico seguinte, abordam-se o método e os procedimentos de pesquisa utilizados para aplicar a estrutura de Gerenciamento de Desempenho e Controle (Performance Management and Control – PMC), adaptada de Ferreira e Otley (2006), JISTEM, Brazil Vol. 11, No.1, Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

176 Beuren, I. M., Teixeira, S. A.

em um estudo de caso realizado em uma instituição de ensino superior. 3.

MÉTODO E PROCEDIMENTOS DA PESQUISA

A pesquisa foi conduzida por meio de um estudo de caso. A estratégia do estudo de caso deve ser conduzida pela pesquisa qualitativa, cujo objetivo caracteriza-se pela análise de fatos sociais em profundidade (Martins & Theóphilo, 2009). No entanto, Yin (2010) argumenta que estudos de casos podem incluir aspectos quantitativos, visto que o contraste entre evidência quantitativa e qualitativa não distingue os vários métodos de pesquisa. Conduzir pesquisas científicas orientadas por avaliações qualitativas e quantitativas é uma prática cada vez mais comum, caracterizada pela descrição, compreensão e interpretação de fatos e fenômenos (Martins & Theóphilo, 2009). Estes procedimentos foram adotados nesta pesquisa, considerando-se também os elementos do protocolo propostos por Yin (2010). O estudo de caso foi realizado em uma instituição de ensino superior privada, localizada na região Sul do Brasil. Solicitou-se a permissão para realizar a pesquisa à alta administração da área de negócios da IES, que foi obtida com restrição à divulgação do nome empresarial. Na coleta dos dados, inicialmente levantaram-se documentos e informações em sites da IES. Em seguida realizaram-se entrevistas não estruturadas com quatro dos cinco diretores da IES, para caracterizar a estratégia deliberada, o sistema de controle de gestão existente, as formas de recompensa, a avaliação de desempenho e a identificação dos gestores responsáveis pelas áreas de negócios da instituição. As entrevistas, realizadas em outubro de 2012, foram gravadas mediante autorização e depois transcritas por digitação. Posteriormente, foi aplicado um questionário para identificar a estrutura de controle gerencial existente que avalia o desempenho na instituição, estruturado e dirigido aos gestores responsáveis pelas ações estratégicas e planos estabelecidos na Escola de Negócios da instituição. 3.1

Instrumento de pesquisa

O instrumento de pesquisa consiste de um questionário baseado no instrumento Performance Management and Control (PMC) de Ferreira e Otley (2006), com ampliação de 12 para 16 questões e a sua transformação em escala Likert de 5 pontos (Hair Jr, Babin, Money & Samuel, 2005), em que o nível 1 significa discordo totalmente e o nível 5 concordo totalmente. Os níveis 2 e 4 representam discordo e concordo parcialmente, respectivamente. O nível 3 representa o posicionamento de indiferente ou não aplicável. Essas adaptações se justificam pela localização geográfica dos respondentes, alocados em cinco unidades distintas da IES. A Figura 2 apresenta um comparativo entre a estrutura proposta por Ferreira e Otley (2006) e a adaptação realizada para esta pesquisa.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Avaliação dos Sistemas de Controle Gerencial em Instituição de Ensino Superior com o Performance Management and Control

177

Estrutura PMC (Adaptada e redefinida em Escala (Ferreira e Otley, 2006) Likert) 1. Conheço a missão e a visão declarada 1. Qual é a visão e a missão da no Planejamento Estratégico da organização e como isso é trazido à instituição, pois são amplamente atenção de gerentes e funcionários? divulgadas e trazidas aos gestores e colaboradores. Estrutura PMC

2. Quais os principais fatores que são acreditados como centrais para o sucesso da organização global futura e como eles são trazidos à atenção de gerentes e funcionários?

2. Conheço os objetivos que são acreditados como sucesso futuro da instituição, pois são amplamente divulgados e trazidos aos gestores e colaboradores. 3. Conheço as atividades e processos necessários para garantir o sucesso do Planejamento Estratégico adotado pela instituição.

3. Que estratégias e planos a organização adota e quais são os processos e atividades que ela decidiu que são necessários para garantir seu sucesso? Como os planos e estratégias são gerados e comunicados aos gerentes e funcionários?

4. Recebi em caráter formal o plano estratégico implementado na instituição, por meio de comunicação direta pela alta administração da instituição.

4. Qual é a estrutura da organização e qual o impacto que tem sobre a concepção e utilização do sistema de gerenciamento e controle de desempenho? Como ela influencia e é influenciada pelo processo de implementação da estratégia?

5. Considero que a estrutura existente permite adotar o sistema de avaliação de desempenho que influencia ou é influenciado pelo Planejamento Estratégico.

5. Quais são as medidas-chaves de desempenho da organização decorrentes 6. A instituição adota indicadores-chaves de seus principais objetivos, fatores de de desempenho ligados aos objetivos e sucesso e estratégias e planos? Como a fatores de sucesso do plano estratégico. organização avalia e mede o seu sucesso? 6. Qual o nível de desempenho é 7. As metas fixadas para avaliar o necessário atingir em cada um dos desempenho são adequadas e permitem domínios definidos na pergunta acima e alcançar os objetivos estratégicos. como se refletem na fixação de metas? 7. Quais os processos que a organização usa para avaliar o desempenho individual, de grupos e organizacional? Quão importantes são as informações e os controles formais e informais nesses processos? Quais são as consequências dos processos de avaliação de desempenho usados? JISTEM, Brazil Vol. 11, No.1, Jan/Apr 2014 pp. 169-192

8. O processo de avaliação existente consegue avaliar o desempenho individual, o desempenho de áreas e o desempenho geral da organização.

www.jistem.fea.usp.br

178 Beuren, I. M., Teixeira, S. A.

Estrutura PMC (Ferreira e Otley, 2006)

Estrutura PMC (Adaptada e redefinida em Escala Likert) 9. Com o atual processo de avaliação de desempenho, tenho acesso às recompensas (financeiras e/ou não financeiras) de forma justa e ponderada caso as metas de desempenho sejam alcançadas.

8. Que recompensas (financeiras e não financeiras) gerentes e outros empregados ganharão por atingir metas de desempenho (ou, por outro lado, qual penalidade sofrerão por não conseguir 10. Com o processo de avaliação de alcançá-los)? desempenho na forma existente, também posso ser penalizado (financeiramente ou não) caso as metas de desempenho não sejam alcançadas. 9. Que comentários específicos e fluxos de informação de feedforward a organização desenvolveu? Quais tipos de fluxos de informação de feedback foram criados para monitorar o desempenho atual e adaptar o comportamento atual? Quais tipos de fluxos de informação de feedforward (se houver) foram formulados para permitir que a organização possa aprender com sua experiência, para gerar novas ideias e recriar estratégias e planos?

12. Recebo informações do tipo diagnóstico (feedback), que me permitem monitorar o desempenho depois de finalizados os resultados. 13. Recebo informações do tipo interativas (feedforward), que me permitem acompanhar o desempenho durante o período, podendo gerar ações de correção para alcançar as metas antes de finalizar os resultados.

10. Que tipo de uso é dado para os fluxos de informação de feedback e feedforward 11. Recebo informações formais e e os vários mecanismos de controle? Uso informais necessárias para acompanhar a predominante de diagnóstico, interativo minha avaliação de desempenho (metas). ou combinação de ambos? 14. Considero que a dinâmica da atividade da organização e do ambiente em que opera, faz com que a avaliação de desempenho possa ser alterada em função desses fatores. 15. A dinâmica da atividade da organização e o ambiente em que opera podem levar a mudanças na avaliação de desempenho. Em outras oportunidades essas alterações já ocorreram. 12. Qual a força e coerência das ligações 16. Considero que os componentes entre os componentes de desempenho e adotados pela organização para gestão, do sistema de gestão e controle (como controle e avaliação de desempenho são indicado nas 11 perguntas acima)? extremamente fortes e coerentes. 11. Como é que o gerenciamento de desempenho e o sistema de controle são alterados, tendo em conta a dinâmica de alteração da organização e do seu ambiente? Quais alterações ocorreram nesses sistemas como antecipação ou resposta a esses estímulos?

Figura 2 - Comparativo da estrutura de Ferreira e Otley (2006) e a estrutura proposta nesta pesquisa JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Avaliação dos Sistemas de Controle Gerencial em Instituição de Ensino Superior com o Performance Management and Control

179

Para validar a adaptação do instrumento de pesquisa realizou-se um teste piloto com um docente de outra IES e um profissional da área, em que apenas ajustes de redação em algumas questões foram sugeridos. Após a permissão para aplicação do questionário houve um contato preliminar por e-mail com todos os gestores responsáveis pela área de negócios da instituição, precedido de outro e-mail explicativo sobre a motivação da pesquisa, a confidencialidade das informações e o tratamento dos resultados. O questionário com a nova estrutura PMC foi enviado por meio de correio eletrônico, com indicação do link eletrônico para acesso direto ao questionário da pesquisa, valendo-se da ferramenta Google Docs. Após duas rodadas de contatos, obtiveram-se 48 questionários respondidos, de um total de 55 gestores identificados, representando 87,27% da população. As etapas de coleta de dados na IES e com os diretores e a remessa e recepção do questionário foram realizadas no período de agosto a outubro de 2012. 3.2

Procedimentos de análise dos dados

Para a análise dos dados, utilizou-se da estatística descritiva, basicamente médias das respostas, além da análise da dispersão nos níveis de escala da resposta por meio de cálculo da entropia da informação. A entropia da informação é uma medida simples, porém relevante quando se trata de um volume diverso de informação em uma mesma fonte de dados (ZELENY, 1982). O cálculo da entropia tem algumas etapas estabelecidas: Sejam caracteriza

x ik os valores normalizados, onde:  * , que xi em termos do i-ésimo atributo. Busca-se d ik

conjunto

D i   d ik ; i  1,2,..., n . Calculando a medida de entropia do contraste de intensidade k 1

d ik  d ik  1 0 e Ln  , onde   e max k 1 Di  Di  m

para o i-ésimo atributo é calculado por e(d i )   emax=Ln(m). Percebe-se que 0  d ik  1 e d ik  0 .

d ik 1 Caso todos os forem iguais para um dado i, então  e e(di) assume Di n 1 valor máximo, isto é, emax=Ln(m). Ao se fixar   , determina-se 0  ed i   1 para e max todos os di´s. Essa normalização é necessária para efeito comparativo. Assim define-se d ik

que a entropia total de D é estabelecida por: E   ed i  . i 1

Deve-se observar que: a) Quanto maior for e(di), menor é a informação transmitida pelo i-ésimo atributo; b) Caso e(di)=emax=Ln(m), então o i-ésimo atributo não transmite informação e JISTEM, Brazil Vol. 11, No.1, Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

180 Beuren, I. M., Teixeira, S. A.

pode ser removida da análise decisória. ~

Por ser o peso  i inversamente relacionado a e(di), usa-se 1-e(di), em vez de ~

e(di) e normaliza-se para assegurar que 0   i  1 e

n ~

  i  1 . Desse modo, a entropia i 1

da informação é representada pela equação: ~

i 

1 1  edi   1  ed i  nE nE

Para validação e confiabilidade dos resultados da pesquisa foi utilizada a proposta de Creswell (2007, p.200), que determina: faça uma triangulação de diferentes fontes de informação de dados, examinando as evidências das fontes. Foi realizada a triangulação com entrevistas não estruturadas com a alta administração da instituição. Na análise transcreveram-se trechos das entrevistas para reforçar os resultados, identificados como [D1], [D2], [D3] e [D4], para preservar a confidencialidade dos respondentes. 4.

DESCRIÇÃO E ANÁLISE DOS RESULTADOS

4.1

Caracterização da IES

Em razão de confidencialidade não é divulgado o nome da instituição, que para fins deste estudo é designada simplesmente de IES. Situa-se dentro do Terceiro Setor e tem sua natureza jurídica enquadrada como instituição comunitária, fundada em 1961. Localiza-se na Região Sul do Brasil e está descentralizada em cinco diferentes cidades. Sua estrutura acadêmica divide-se em nove escolas: negócios, arquitetura e design, ciências agrárias e medicina veterinária, comunicação e artes, medicina, saúde e biociências, educação e humanidades, politécnica e direito. Cada unidade descentralizada possui um diretor e cada escola possui um decano acadêmico. Ao todo são 99 cursos de graduação e tecnólogos, cerca de 180 cursos de especialização, 14 de mestrado e dez de doutorado. Esse conjunto de atividades é composto por aproximadamente 1.800 professores, 6.000 funcionários e 26.000 alunos presenciais. Como abordado na metodologia da pesquisa, este estudo focaliza a Escola de Negócios, presente nas cinco unidades descentralizadas. Em 1998, a IES apresentou sua primeira versão do planejamento estratégico, dando início ao processo de descentralização e criação do projeto de interiorização, criando as unidades fora de sede. As revisões do plano estratégico culminaram na última revisão de 2008, que resultou no projeto de tornar a IES uma universidade de classe mundial em um período de 14 anos. Com a visão de futuro definida, foram implementadas diversas ações, dentre elas a adoção do sistema de remuneração variável, aplicável aos gestores responsáveis por direção e coordenação de cursos, além de gestores administrativos que atuam em funções-chaves para o desempenho estratégico. Em 2009 iniciou o processo de avaliação de desempenho por metas, ligadas aos objetivos estratégicos e pagamento de bônus anual para alcance de pelo menos 70% do desempenho esperado.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

181

Avaliação dos Sistemas de Controle Gerencial em Instituição de Ensino Superior com o Performance Management and Control

Após três anos implantado na IES, o sistema de avaliação de desempenho foi alterado seguidamente, evidenciando-se como um processo em evolução e com melhoria contínua. Em todas as escolas, cerca de 200 gestores são responsáveis pelas ações estratégicas e, portanto, sujeitos a avaliação de desempenho e remuneração por meritocracia. 4.2

Estatística descritiva

Na primeira parte da análise dos resultados evidencia-se a discriminação das respostas, divididas por questões e níveis de respostas de acordo com a escala Likert, sendo CT para Concordo Totalmente, CP para Concordo Parcialmente, NA para Não se Aplica ou Indiferente, DP para Discordo Parcialmente e DT para Discordo Totalmente. Os resultados estão apresentados na Tabela 1.

Questões

Resp

41,66

43,75

4,17

6,25

4,17

33,34

58,33

0,00

6,25

2,08

20,84

58,33

8,33

4,17

29,17

12,50

16,66

12,50

14,58

52,08

20,84

12,50

0,00

10,42

54,16

14,58

20,84

0,00

10,42

45,83

20,84

16,66

6,25

14,58

37,50

18,75

29,17

0,00

22,92

35,41

18,75

20,84

2,08

Q10

29,17

22,92

16,66

18,75

12,50

Q11

12,50

58,33

8,33

16,67

4,17

Q12

25,00

41,67

18,75

12,50

2,08

Q13

12,50

39,58

27,09

12,50

8,33

Q14

16,67

43,75

25,00

12,50

2,08

Q15

18,75

41,66

20,84

18,75

0,00

Q16

12,50

47,92

18,75

2,08

Total de respondentes (Resp) = 48 Tabela 1 - Estatística descritiva da pesquisa 4.2.1 Análise das questões dos planos estratégicos As questões Q1 a Q4 abordam a estratégia adotada. Na primeira afirmação (Q1) os resultados apontam que 41 (85,42%) concordam total ou parcialmente, indicando razoável disseminação do plano estratégico. Porém, 5 (10,42%) respondentes JISTEM, Brazil Vol. 11, No.1, Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

182 Beuren, I. M., Teixeira, S. A.

discordaram total ou parcialmente da afirmativa e 2 (4,17%) se posicionaram indiferentes. Isso mostra que uma parcela de aproximadamente 15% dos gestores não conhece ou conhece pouco a missão e a visão estratégica da IES. Os objetivos estratégicos (Q2), até mesmo por serem mais ligados às metas estipuladas, são conhecidos parcialmente por 58,33% dos respondentes e totalmente por 33,34%. Mesmo que esse resultado indique um bom nível de conhecimento dos objetivos estratégicos da IES pelos gestores, 8,33% ainda indicaram haver pouca ou nenhuma divulgação dos objetivos. As atividades e processos necessários para alcançar os objetivos estabelecidos (Q3) são conhecidos parcialmente ou totalmente por 79,16% dos respondentes. É importante notar que 20,83% dos gestores responderam discordar ou não conseguiram aplicar resposta positiva nesta afirmação. Esses resultados são corroborados pela questão 4, já que 41,66% dos respondentes optaram pelos níveis 1, 2 ou 3, o que evidencia que 22 gestores não tiveram acesso ao conhecimento da estratégia adotada ou a conheceram de forma não oficial. Os resultados das questões pertinentes ao plano estratégico denotam que a instituição tem a missão, a visão e os objetivos declarados e delineados. Porém, apresenta limitações no alcance de sua disseminação à totalidade do corpo de gestores da instituição. Tais limitações são convergentes com os estudos de Ferreira e Otley (2006), que encontraram restrições nessa direção em duas das quatro empresas estudadas. 4.2.2 Análise das questões do sistema de avaliação de desempenho As questões Q5 a Q10 tratam dos sistemas de avaliação de desempenho. A estrutura existente na IES (Q5) é apontada como adequada total ou parcialmente por 66,66% dos respondentes. Uma parcela significativa respondeu como indiferente (20,84%) e discordando parcialmente (12,50%). Os indicadores-chave de desempenho são ligados aos objetivos e fatores de sucesso do plano estratégico (Q6). Esta é uma assertiva que obteve concordância parcial por 54,16% dos respondentes, enquanto 10 (20,84%) acreditam que os indicadores avaliam pouco a estratégia adotada. As metas fixadas são parcialmente adequadas (Q7) para 45,83% dos respondentes, enquanto 22,91% avaliam as metas como pouco adequadas ou mesmo inadequadas. A eficácia da avaliação existente (Q8) é contestada por 29,17% dos pesquisados, enquanto 18,75% assinalaram indiferença à questão. Apenas 52,08% acreditam que a avaliação é completa (14,58%) ou parcialmente eficiente (37,50%). As recompensas do processo de avaliação de desempenho existente são justas e ponderadas (Q9) para 22,92% dos gestores, enquanto que grande parte (35,41%) concorda parcialmente e 20,84% discordam parcialmente. Outro fator na percepção dos gestores é que 29,17% concordam totalmente ou parcialmente (22,92%) que pode haver penalização no caso de não atingimento das metas estabelecidas (Q10). As respostas às questões relativas ao sistema de avaliação de desempenho inferem necessidades de melhorias, já que razoável parcela dos gestores não alinha tais medidas com a estratégia adotada e outra parte desconhece a estrutura ou indicou que ela é incompleta. A eficácia e a justiça da avaliação e a possibilidade de penalização também foram questões com deficiências apontadas. Ferreira e Otley (2006) também observaram divergências nas medidas de desempenho e recompensas na análise de indivíduos, grupos ou equipes nas empresas estudadas. A prática de vincular JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Avaliação dos Sistemas de Controle Gerencial em Instituição de Ensino Superior com o Performance Management and Control

183

recompensas a fatores de desempenho e penalização foram práticas observadas em pelo menos duas das quatro empresas daquele estudo. 4.2.3 Análise das questões dos controles de gestão para avaliação de desempenho Os mecanismos e controles da avaliação de desempenho e o fluxo de informações existentes para acompanhamento são respondidas nas questões Q11 a Q16. Nota-se que 70,83% responderam que recebem total ou parcialmente informações formais e informais para acompanhar o desempenho e metas (Q11), enquanto 20,84% discordaram e 8,33% apontaram o nível 3 da escala para a questão, que representa indiferença. Tais informações são reportadas com feedback (Q12) para 25% dos respondentes, enquanto 41,67% apontaram receber às vezes esse retorno. Para 14,58% dos gestores há deficiências na devolutiva dos resultados. Durante o processo essas devolutivas com informações feedforwards (Q13) caem para 39,58% de respondentes que concordam parcialmente e 12,50% que concordam totalmente, mas 27,09% não souberam responder e se mantiveram indiferentes, enquanto 20,83% discordaram total ou parcialmente. Para a questão 14 (Q14), que cita que o processo de avaliação de desempenho é flexível, tendo sido alterado em função das atividades da organização e do ambiente , para 60,42% dos respondentes houve concordância total ou parcial, enquanto 25,00% se mostraram indiferentes. A necessidade dessa flexibilidade é compartilhada (Q15) por 29 gestores, que perceberam mudanças ocorridas, enquanto outros 10 (20,84%) e 9 (18,75%) assinalaram os níveis 3 e 2 na escala, respectivamente. A última questão (Q16) indicou que 47,92% (23 gestores) acreditam que os componentes adotados pela organização para gestão, controle e avaliação de desempenho é forte, porém nem sempre coerente. Outros 9 gestores (18,75%) discordaram, apontando que o sistema é fraco e pouco coerente, 9 preferiram o nível 3 (indiferente), enquanto 6 (12,50%) consideram o sistema robusto e apenas 1 discordou totalmente. As respostas sobre os controles de gestão de desempenho indicam limitações na devolutiva de informações, sejam informações de feedback ou feedforward, bem como deficiências moderadas na base de dados e disponibilidade de informações. Ferreira e Otley (2006) também constataram em sua pesquisa que os controles de feedback estavam difundidos em todas as empresas, por se relacionarem a questões mais operacionais. Contudo, informações feedforward se apresentaram escassas ou inexistentes e o baixo nível de informações estratégicas no âmbito do sistema de controle foi diagnosticado naquele estudo. 4.3

Entropia da informação

Os resultados também foram submetidos à análise da entropia da informação utilizando as questões do questionário de forma global, por meio do cálculo da entropia da informação e da média. A finalidade é apontar as questões que apresentaram maior variabilidade de respostas, o que pode apontar falta de alinhamento dos gestores no respectivo tópico. A coluna que indica a média identifica o ponto da escala Likert em que se concentrou a média de opinião dos respondentes. A Tabela 2 apresenta a análise dos resultados pela entropia da informação.

JISTEM, Brazil Vol. 11, No.1, Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

184 Beuren, I. M., Teixeira, S. A.

Estrutura PMC

Entropia e(di)

i

Média

Conheço a missão e a visão declarada no Planejamento Estratégico da instituição, pois são amplamente divulgadas e trazidas aos gestores e colaboradores.

0,9901

0,0481

4,13

Conheço os objetivos que são acreditados como sucesso futuro da instituição, pois são amplamente divulgados e trazidos aos gestores e colaboradores.

0,9933

0,0328

4,15

Conheço as atividades e processos necessários para garantir o sucesso do Planejamento Estratégico adotado pela instituição.

0,9899

0,0494

3,83

Recebi em caráter formal o plano estratégico implementado na instituição, por meio de comunicação direta pela alta administração da instituição.

0,9764

0,1152

3,46

Considero que a estrutura existente permite adotar o sistema de avaliação de desempenho que influencia ou é influenciado pelo Planejamento Estratégico.

0,9923

0,0374

3,69

A instituição adota indicadores-chaves de desempenho que são ligados aos objetivos e fatores de sucesso do plano estratégico.

0,9903

0,0471

3,54

As metas fixadas para avaliar o desempenho são adequadas e permitem alcançar os objetivos estratégicos.

0,9853

0,0717

3,38

O processo de avaliação existente consegue avaliar o desempenho individual, o desempenho de áreas e o desempenho geral da organização.

0,9870

0,0634

3,38

Com o atual processo de avaliação de desempenho, tenho acesso às recompensas (financeiras e/ou não financeiras) de forma justa e ponderada caso as metas de desempenho sejam alcançadas.

0,9863

0,0667

3,56

Com o processo de avaliação de desempenho na forma existente, também posso ser penalizado 10 (financeiramente ou não) caso as metas de desempenho não sejam alcançadas.

0,9754

0,1200

3,38

Recebo informações formais e informais necessárias 11 para acompanhar a minha avaliação de desempenho (metas).

0,9875

0,0607

3,58

Recebo informações do tipo diagnóstico, que me 12 permitem monitorar o desempenho depois de finalizados os resultados.

0,9892

0,0527

3,75

13 Recebo informações do tipo interativas, que me permitem acompanhar o desempenho durante o

0,9840

0,0452

3,35

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Avaliação dos Sistemas de Controle Gerencial em Instituição de Ensino Superior com o Performance Management and Control

185

Entropia e(di)

i

Média

Considero que a dinâmica da atividade da organização e do ambiente em que opera, faz com 14 que a avaliação de desempenho possa ser alterada em função desses fatores.

0,9897

0,0452

3,60

A dinâmica da atividade da organização e o ambiente em que opera podem levar a mudanças na 15 avaliação de desempenho. Em outras oportunidades essas alterações já ocorreram.

0,9886

0,0448

3,60

Considero que os componentes adotados pela 16 organização para gestão, controle e avaliação de desempenho são extremamente fortes e coerentes.

0,9915

0,0332

3,50

1,0000

3,62

Estrutura PMC

período, podendo gerar ações de correção para alcance das metas antes de finalizar os resultados.

Total

Tabela 2 - Análise dos resultados pela entropia da informação Observa-se na Tabela 2 que a média de todas as questões foi de 3,62, ou seja, variando entre o nível 3 (indiferente) e 4 (concordo parcialmente) da escala. Nas duas colunas da entropia da informação, a Entropia e(di) representa o valor informacional do i-ésimo atributo, enquanto a coluna ƛ i evidencia o valor da entropia da informação, ou seja, quanto maior a entropia, maior a dispersão dos níveis de resposta na escala. 4.3.1 Análise da entropia das questões dos planos estratégicos O bloco de questões sobre o Planejamento Estratégico apresenta os melhores resultados das médias nos níveis da escala: Q2, Q1 e Q3 obtiveram respostas com média de 4,15, 4,13 e 3,83, respectivamente. A Q2 obteve o menor índice de entropia da informação entre todas as questões da pesquisa, com ƛi de 0,0328, mostrando menor dispersão entre os níveis de resposta. Contudo, a Q4, do mesmo bloco, obteve a segunda pior média (3,46), que mostra deficiência na comunicação formal dos planos estratégicos pela alta administração. Essa questão também teve o segundo pior índice de entropia da informação, com ƛi de 0,1152, que aponta respostas dispersas entre todos os níveis da escala. Os resultados são suportados pela entrevista com um dos diretores da alta administração da Escola de Negócios da IES, como pode ser observado no relato de [D3]: Os gestores conhecem a missão e a visão de futuro, isso foi bem trabalhado. Já os projetos estratégicos não conhecem bem, não estão tão claros. [...] Existe falha ou falta de comunicação. Não pela forma, mas como se faz isso e a própria comunicação. [...] Em papel não ajudaria muito, acho que a conversa, a explanação é a melhor forma de colocar todo mundo no mesmo barco. Denota-se que os resultados das questões sobre a estrutura PMC, relativas a aspectos estratégicos, estão em consonância com a opinião da alta direção. Os índices de entropia ruins, em especial a comunicação formal dos planos estratégicos, alinham-se JISTEM, Brazil Vol. 11, No.1, Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

186 Beuren, I. M., Teixeira, S. A.

com a opinião da direção, que os apontou na entrevista como fatores importantes para ajustar a direção dos esforços de equipe. Esses achados são coerentes com Ferreira e Otley (2006), que sugerem outros mecanismos, como oficinas estratégicas e reflexões estratégicas, para reforçá-los. 4.3.2 Análise da entropia das questões do sistema de avaliação de desempenho As questões relativas ao sistema de avaliação de desempenho indicam melhor resultado para Q5, referente à estrutura da organização (3,69), que obteve a quinta melhor média entre todas as questões da estrutura. Também houve dispersão discreta nesta questão, com ƛi de 0,0374, sendo considerada a terceira menor dispersão entre os níveis de resposta. A questão da estrutura da organização também foi comentada na triangulação dos dados por meio da entrevista de [D4]: Não temos a estrutura ideal, mas o controle é suficiente. O ideal seria um software mais específico. [...] Existem controles em Excel, relatórios divulgados com certa frequência, mas não temos uma ferramenta que nos facilite consultas, com complemento gráfico. Não temos o ideal implantado, mas é suficiente. O acesso às recompensas por meio da avaliação de desempenho na Q9 obteve apenas a nona melhor média, enquanto a penalização pelo não alcance de metas (Q10) é reconhecida como presente por boa parte dos gestores, pois obteve média 3,38 dos respondentes. A Q10 também obteve o pior índice de entropia entre todas as questões, com ƛi de 0,12, que aponta a maior dispersão entre os respondentes, mostrando que não está claro na IES esse componente do sistema de avaliação de desempenho. Esta questão foi tratada na entrevista por [D2], conforme segue: Mesmo quando não atingimos o percentual de 70%, como no ano passado, quando tivemos 50%, 60% de metas atingidas, não há penalização. Causa uma certa frustração, mas ao mesmo tempo representou que fizemos um bom trabalho e chegamos perto da meta estipulada. Não acredito haver penalização. É evidente que o insucesso de forma sistemática, continuada, leva a uma revisão da condição de gestão daquela área ou setor. As médias das demais questões deste bloco, Q6 (3,54), Q7 e Q8 (ambas com média 3,38), resultaram na classificação em 10º, 13º e 14º lugares, respectivamente, entre o conjunto de 16 questões da estrutura PMC proposta nesse estudo. Os resultados de dispersão também ficaram com maior dispersão nos níveis de respostas em relação às demais questões, se posicionando entre os cinco últimos índices de entropia da informação. Os resultados da entropia da informação evidenciam que o sistema de avaliação de desempenho na IES ainda não está consolidado e causa desconfiança quanto à sua eficácia e justiça, sendo questionado ainda o acesso a recompensa e, em especial, a probabilidade de penalidades. O sentimento de frustração pelo não alcance das metas, apontado em uma das entrevistas com a alta administração, coaduna com os resultados da pesquisa. Porém, a opinião de penalização não é compartilhada pelo board da instituição. Os resultados convergem parcialmente com os achados nas empresas estudadas por Ferreira e Otley (2006), ao comparar a opinião da administração com o pessoal do chão de fábrica.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Avaliação dos Sistemas de Controle Gerencial em Instituição de Ensino Superior com o Performance Management and Control

187

4.3.3 Análise da entropia das questões dos controles de gestão para avaliação de desempenho No bloco de questões sobre o controle gerencial para avaliar o desempenho, o melhor resultado ficou para a Q12, que trata do feedback dos resultados alcançados, com 3,75 na escala e 4º melhor nível de resposta no geral. Já o acompanhamento durante o processo, na Q13, ficou com 3,35. O recebimento de informações formais e informais (Q11) obteve ƛi de 0,067 e mostrou alta dispersão e o 5ª pior índice de entropia. As informações relativas a estas questões foram discutidas por [D1], a saber:Para conseguir informações temos muitas dificuldades. Já houve melhoras, mas ainda estamos distantes de consegui-las com facilidade e agilidade. O ideal é que tivéssemos um painel de indicadores à disposição, permanentemente disponível no sistema. Temos um problema de base de dados, e por vezes os dados não são coincidentes. Os resultados apontam necessidade de melhorias na devolutiva e no acesso as informações. As informações de feedback são mais presentes que as de feedforwards, porém não é consenso, já que houve dispersão na percepção dos respondentes. Essa necessidade é compartilhada pela alta administração, que inclusive traz sugestões na entrevista e críticas à base de dados. Os achados são condizentes com os do estudo de Ferreira e Otley (2006). As questões Q14 e Q15, sobre a dinâmica e flexibilidade do sistema de avaliação de desempenho, obtiveram média 3,60, enquanto o recebimento formal e informal de informações ficou com média 3,58. A última questão (Q16) obteve média de 3,50 e evidencia a necessidade de fortalecer o sistema de controle gerencial e os componentes de avaliação e controle do desempenho. Estes aspectos também podem ser observados no que foi relatado por [D1], como segue: O ideal seria que todos os colaboradores, indistintamente de sua função, até o mais simples colaborador da instituição, tivesse acesso a algum tipo de recompensa pelo resultado geral alcançado. Essa é minha visão do sistema. Mas estamos melhorando; em três anos de implantação da avaliação de desempenho já se verificaram expressivas melhorias no processo e estamos em evolução. É um processo com melhoria contínua. Depreende-se que, apesar da flexibilidade do sistema e as mudanças realizadas nos últimos três anos no SCG da IES, os índices de entropia da informação revelam haver espaço para melhorias, pois o modelo ideal ainda não foi obtido pela IES. Os resultados convergem parcialmente com os encontrados no estudo de Ferreira e Otley (2006). Ressalta-se que a entropia da informação evidencia questões em que a dispersão de respostas se apresentou em maior nível, já discutidas, o que não só apresenta deficiências de alinhamento ao plano estratégico, mas oportuniza ações de correção pelos gestores da instituição. 5.

CONSIDERAÇÕES FINAIS

Este estudo objetivou averiguar a estrutura e o funcionamento dos sistemas de controle gerencial para avaliação de desempenho em uma instituição de ensino superior privada. Com a estrutura Performance Management and Control (PMC), proposta por JISTEM, Brazil Vol. 11, No.1, Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

188 Beuren, I. M., Teixeira, S. A.

Ferreira e Otley (2006) para captar a estrutura e o funcionamento dos SCG, adaptou-se o instrumento de pesquisa com questões reformatadas para a escala Likert de cinco pontos, a fim de ampliar e acelerar a coleta de dados na IES, objeto do estudo de caso. O questionário foi enviado aos 55 gestores responsáveis pelas ações estratégicas e planos estabelecidos na Escola de Negócios da IES e foram obtidas 48 respostas, representando 87,27% da população. Também foram entrevistados quatro diretores da Escola de Negócios da IES, além de pesquisa documental, para triangulação dos dados e validação dos resultados. Na análise dos dados utilizaram-se as técnicas de estatística descritiva e entropia da informação. A sua aplicação em uma instituição de ensino superior, que possui um plano estratégico implantado e em constante revisão, além de um sistema de avaliação de desempenho e controle gerencial em pleno curso de implantação, propiciou apontar os pontos fracos do sistema. A utilização da entropia da informação trouxe robustez ao tratamento dos dados, pois permitiu verificar a dispersão das respostas em alguns quesitos fundamentais. Os maiores índices encontrados de entropia da informação residiram nas questões sobre a possibilidade de penalidades pelo não alcance das metas e na comunicação oficial da alta administração para os gestores sobre o planejamento estratégico. O estudo apontou que a IES ainda precisa promover um melhor alinhamento entre a estratégia empresarial, o sistema de avaliação de desempenho e os sistemas de controle gerencial. A média geral ficou em 3,62 na escala Likert de cinco pontos, o que revela que a concordância pelo menos parcial da maioria dos itens não foi alcançada por parte dos respondentes da IES. Apenas duas questões, uma sobre o conhecimento da missão e visão e outra sobre os objetivos da estratégia adotada, superaram os 4 pontos na escala. Avaliando as questões em bloco, os melhores resultados ficaram nas questões do plano estratégico, até mesmo porque é um processo já implantado há mais de uma década. Ressalta-se que, dentre as questões desse bloco, a deficiência da comunicação do plano pela alta administração foi um dos itens negativos apontados na pesquisa, inclusive com a segunda pior dispersão nos níveis de resposta. Outro fator é que 7 dos 48 respondentes se manifestaram com “indiferente” no ponto da escala Likert, o que pode representar desconhecimento do quesito tratado, falta de opinião formada ou incerteza sobre o tema. Nessas questões é recomendável a instituição ter uma ação específica para identificar a fragilidade apontada. O bloco de questões que trataram das medidas do sistema de avaliação de desempenho, sua adequação e capacidade para realmente avaliar o resultado global, por unidade e individual, as recompensas e penalidades, foi o pior bloco avaliado e estas questões obtiveram as piores médias. A maior dispersão por entropia ficou por conta da probabilidade de penalidade pelo não alcance das metas, superior a todos os demais itens pesquisados. As questões referentes ao bloco de controle gerencial para avaliação de desempenho tiveram desempenho intermediário, entre a estratégia e a avaliação de desempenho. Ficou evidenciado, em especial nas entrevistas que triangularam com os resultados dos questionários, que as informações não são facilmente obtidas, em especial as informações feedback e feedforward. Os depoimentos das entrevistas mostram também a necessidade de adequações no banco de dados e na agilidade e disponibilidade das informações estratégicas. Como conclusão, com a aplicação da estrutura adaptada da PMC, foi possível JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Avaliação dos Sistemas de Controle Gerencial em Instituição de Ensino Superior com o Performance Management and Control

189

identificar no estudo de caso proposto como o sistema de controle gerencial está sendo utilizado na instituição de ensino superior pesquisada para medir, controlar e gerir o desempenho. Desta forma, foram identificados a estrutura e o funcionamento dos sistemas de controle gerencial utilizados para avaliar o desempenho na IES. Foram apontados alguns pontos fracos no sistema utilizado, bem como se denotou a necessidade de melhorar o alinhamento entre estratégia, desempenho e controle. Como contribuição, além do novo formato da PMC aqui adotado, da utilização e tratamento dos dados pela entropia, da triangulação com o board da organização, a estrutura proposta por Ferreira e Otley (2006) foi aplicada em um ambiente complexo, como é o caso das instituições de ensino superior. No entanto, dadas as limitações deste estudo, já que foi aplicado exclusivamente à Escola de Negócios da instituição, para futuras pesquisas recomendam-se novas aplicações do instrumento de pesquisa, com a estrutura alterada da proposta de Ferreira e Otley (2006). Tais aplicações poderão ocorrer em outros segmentos, em estudos multicasos, bem como com a aplicação de outros tratamentos estatísticos.

REFERENCES Abernethy, M. A., & Chua, W. F. (1996). A field study of control system redesigns: the impact of institutional process on strategic choice. Contemporary Accounting Research, 13(2), 569-606. Anthony, R. (1965). Planning and control systems: a framework for analysis. Boston: Harvard Business Press. Banker, R. D., Chang, H., & Pizzini, M. J. (2004). The balanced scorecard: judgmental effects of performance measures linked to strategy. The Accounting Review, 79(1), 1-23. Briers, M., & Hirst, M. (1990). The role of budgetary information in performance evaluation. Accounting, Organizations and Society, 15(4), 373-398. Bryant, L., Jones, D. A., & Widener, S. K. (2004). Managing value creation within the firm: an examination of multiple performance measures. Journal of Management Accounting Research, 16(1), 107-132. Chenhall, R. H. (2003). Management control systems design within its organizational context: findings from contingency-based research and directions for the future. Accounting, Organizations and Society, 28(2-3), 127-168. Chenhall, R. H., & Langfield-Smith, K. (2007). Multiple perspectives of performance measures. European Management Journal, 25(4), 266-282. Covaleski, M. A., Evans III, J. H., Luft, J., & Shields, M. D. (2003). Budgeting research: three theoretical perspectives and criteria for selective integration. Journal of Management Accounting Research, 15(3), 3-47. Creswell, J. W. (2007). Projeto de pesquisa: métodos qualitativo, quantitativo e misto. 2. ed. Porto Alegre: Artmed.

JISTEM, Brazil Vol. 11, No.1, Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

190 Beuren, I. M., Teixeira, S. A.

Ferreira, A., & Otley, D. (2006). The design and use of performance management systems: an extended framework for analysis. Management Accounting Research, 20(4), 263-282. Govindarajan, V., & Gupta, A. K. (1985). Linking control systems to business unit strategy: impact on performance. Accounting, Organizations and Society, 20(1), 51-66. Gupta, A. K. (1987). SBU strategies, corporate-SBU, and SBU effectiveness in strategy implementation. Academy of Management Journal, 30(3), 477-500. Gupta, A .K., & Govindarajan, V. (1984). Business unit strategy, managerial characteristics, and business unit effectiveness at strategy implementation. Academy of Management Journal, 27(1), 25-41. Hair Jr, J. F., Babin, B., Money, A. H., & Samuel, P. (2005). Fundamentos de métodos de pesquisa em administração. Porto Alegre: Bookman. Horngren, C. T., Foster, G., & Datar, S. (2000). Contabilidade de custos. Rio de Janeiro: LTC. Ittner, C. D., & Larcker, D. E. (1998). Innovations in performance measurement: trends and research implications. Journal of Management Accounting Research, 10(1), 205238. Ittner, C. D., Larcker, D. F., & Meyer, M. W. (2003). Subjectivity and the weighting of performance measures: evidence from a balanced scorecard. The Accounting Review, 78(3), 725-758. Kaplan, R. S., & Norton, D. P. (1996). The Balanced Scorecard: translating strategy into action. Boston, MA: Harvard Business School Press. Kaplan, R. S., & Norton, D. P. (2000). The strategy focused organization: how Balanced Scorecard companies thrive in the new business environment. Boston, MA: Harvard Business School Press. Kaplan, R. S., & Norton, D. P. (2004). Strategy maps: converting intangible assets into tangible outcomes. Boston, MA: Harvard Business School Press. Langfield-Smith, K. (1997). Management control systems and strategy: a critical review. Accounting, Organizations and Society, 22(2), 207-232. Langfield-Smith, K., Thorne, H. T., & Hilton, R. W. (2006). Management Accounting: information for managing and creating value. 4 ed. North Ryde, Australia: McGrawHill. Lynch, R. L., & Cross, K. F. (1995). Measure up! yardsticks for continuous improvement. Cambridge, MA: Blackwell Inc. Malina, M. A., & Selto, F. H. (2001). Communicating and controlling strategy: An empirical study of the effectiveness of the balanced scorecard. Journal of Management Accounting Research, 13(1), 47-90. Malmi, T., & Brown, D. (2008). Management control systems as a package – opportunities, challenges and research directions. Management Accounting Research, JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

Avaliação dos Sistemas de Controle Gerencial em Instituição de Ensino Superior com o Performance Management and Control

191

19(4), 287-300. Martins, G. A., & Theóphilo, C. R. (2009). Metodologia da investigação científica para ciências sociais aplicadas. 2 ed. São Paulo: Editora Atlas. McNair, C. J., Lynch, R. L., & Cross, K. F. (1990). Do financial and nonfinancial performance measures have to agree? Management Accounting Research, 72(5), 28-36. Merchant, K. A., & Van Der Stede, W. A. (2007). Management control systems. 2. ed. Harlow: Pearson Education Limited. Miles, R. W.; Snow, C. C. (1978). Organizational strategy, structure and process. New York: McGraw-Hill. Neely, A., & Adams, C. (2001). Perspectives on performance: the performance prism. Journal of Cost Management, 15(1), 7-15. Neely, A., Adams, C., & Kennerley, K. (2002). The performance prism: the scorecard for measuring and managing business success. London: Financial Times Prentice Hall. Noreen, E. (1988). The economics of ethics: a new perspective on agency theory. Accounting, Organizations and Society, 13(4), 359-369. Otley, D. (1999). Performance management: a framework for management control systems research. Management Accounting Research, 10(10), 363-382. Porter, M. E. (1985). Competitive advantage: creating and sustaining superior performance. New York, NY: Free Press. Schwartz, G. (1970). Science in marketing. New York, NY: John Wiley. Simons, R. (1995). Levers of control: how managers use innovative control systems to drive strategic renewal. Boston: Harvard Business School Press. Spendolini, M. (1992). The benchmarking book. New York, NY: Amacon. Vieira, R., Major, M. J., & Robalo, R. (2009). Investigação qualitativa em Contabilidade. In: Major, M. J., & Vieira, R. (Ed.). Contabilidade e controlo de gestão: teoria, metodologia e prática. Lisboa: Escolar Editora. 2009. pp 301-331. Yin, R. K. (2010). Estudo de caso: planejamento e métodos. 4. ed. Porto Alegre: Bookman. Zeleny, M. (1982). Multiple criteria decision making. New York: McGraw-Hill.

JISTEM, Brazil Vol. 11, No.1, Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

192 Beuren, I. M., Teixeira, S. A.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014 pp. 169-192

www.jistem.fea.usp.br

JISTEM - Journal of Information Systems and Technology Management Revista de Gestão da Tecnologia e Sistemas de Informação Vol. 11, No. 1, Jan/Apr., 2014 pp. 193-214 ISSN online: 1807-1775 DOI: 10.1590/S1807-17752014000100011

ELICITING AND DEFINING REQUIREMENTS BASED ON METAEVALUATION: THE CASE OF THE CRAS 2008 CENSUS Edilson Ferneda Catholic University of Brasília, Brasilia, Distrito Federal, Brazil Hércules Antonio do Prado Catholic University of Brasília, Brasilia, Distrito Federal, Brazil Empresa Brasileira de Pesquisa Agropecuária Marcelo Gomes Gadelha Ministério do Desenvolvimento Social e Combate à Fome, Brasilia, DF, Brazil Helga Cristina Hedler Catholic University of Brasília, Brasilia, Distrito Federal, Brazil ______________________________________________________________________

ABSTRACT The Brazilian Ministry of Social Development and Fight against Hunger (MDS) regularly promotes the evaluation of its social programs, such as those developed in the Reference Centers for Social Assistance (CRAS). Such evaluations make use of a web system that supports the collection and processing of information as well as the dissemination of its results to local, regional and central government officials through the so-called CRAS Census. A meta-evaluation of the CRAS 2008 Census was carried out based on criteria ____________________________________________________________________________________________ Manuscript first received/Recebido em 26/09/2011 Manuscript accepted/Aprovado em:11/02/2014 Address for correspondence / Endereço para correspondência Edilson Ferneda, Professor at the Graduate Program on Knowledge Management and Information Technology at Universidade Católica de Brasília (UCB). Doctor’s Degree in Informatics from the Laboratoire d’Informatique, Robotique et Microélectronique de Montpellier (LIRMM/CNRS/UM2, Montpellier, France), Master’s Degree in Informatics from the Federal University of Campina Grande (UFCG, Campina Grande, PB. Address: SGAN, Módulo B, room A-115 - 70.970-160 Brasília, DF. Phone: 55-61-3448-7159. Email: eferneda@pos.ucb.br Hércules Antonio do Prado, Catholic University of Brasília. Doctor´s Degree in Computer Science by the Informatics Institute of Federal University of Rio Grande do Sul (UFRGS, Porto Alegre, RS), Master´s Degree in Systems and Computer Engineering by Federal University of Rio de Janeiro (COPPE/UFRJ, Rio de Janeiro, RJ). Lecturer at the Graduate Program on Knowledge Management and Information Technology and Analyst at the Secretariat for Strategic Management of Embrapa. Address: Catholic University of Brasília - SGAN, Módulo B, room A-115 - 70.970-160 Brasília, DF. Phone: 55-61-34487159. Email: hercules@ucb.br / Brazilian Agricultural Research Corporation (Embrapa) - Parque Estação Biológica - PqEB s/n - 70.770-901 Brasília, DF. Phone: 55-6134484287. Email: hercules.prado@embrapa.br Marcelo Gomes Gadelha, Ministry of Social Development and Fight against Hunger (MDS). Master’s Degree in Knowledge Management and Information Technology from the Catholic University of Brasília (UCB, Brasília, DF), Specialist in Software Process Improvement from the Federal University of Lavras (UFLA, Lavras, MG. Systems Coordinator at Ministry of Social Development and Fight against Hunger (MDS). Address: SEPN 515, Edifício Ômega, Bloco B, room T28A - Brasília, DF. Phone: 55-61-2030-3922. Email: marcelo.gadelha@mds.gov.br Helga Cristina Hedler, Catholic University of Brasília (UCB). Doctor and Master Degree in Social Psychology of Work and Organizations form the Institute of Psychology, University of Brasilia (UnB, Brasília, DF). Lecturer at the the Graduate Program in Knowledge Management and Information Technology. Address: SGAN, Módulo B, room A-114 – 70.970-160 Brasília (DF). Phone: 55-61-3448-7208. Email: eferneda@pos.ucb.br Published by/ Publicado por: TECSI FEA USP – 2014 All rights reserved.

194 Ferneda, E., Prado, H. A. do, Gadelha, M. G., Hedler, H. C.

specified by the Joint Committee (1994), from which we elicited requirements that enabled improvements of the web system. The article reports new requirements elicited from the meta-evaluation of the CRAS 2008 Census, held in the period 2009-2010. The approach of meta-evaluation as an alternative source of requirements elicitation took into consideration results from evaluations of social programs in order to identify system problems without the usual need of intense interaction with users. This approach revealed opportunities for improvements in the evaluation process that led to the elicitation of requirements for the computerized system. Some of the elicited features were incorporated into the Census 2010 and others may be incorporated in future censuses. Keywords: Requirements Elicitation; Evaluation; Metaevaluation.

INTRODUCTION

Requirements Engineering (RE), in the context of Software Engineering, can be seen as an important activity that permeates the communication and modeling activities in order to build a bridge between the need for software and its design and implementation (Pressman, 2010, p. 120). For IEEE (1990), the set of requirements of a system includes: (i) conditions or potentialities required by a user to solve a problem or achieve an objective; (ii) conditions or potentialities a system, component, or product must exhibit to be accepted, and (iii) the documentation related to these two items. Pressman (2010, p. 121) categorize the RE tasks as: conception, elicitation, elaboration, negotiation, specification (modeling), management, and validation of the requirements of a software package. Additionally, the author warns about the possibility of overlapping these activities along a project schedule. This work describes the elicitation of nonfunctional, functional, and business rules requirements in the context of the information systems involved in the evaluation of the CRAS 20081 Census. In the case studied the starting point for eliciting requirements were not the traditional method of applying interviews to clients to figure out their expectations, but rather the results of a critical analysis based on widely accepted evaluation standards from the metaevaluation realm (Coosky; Caracelli, 2009). 2.

THE CASE STUDIED

The Centers of Reference in Social Assistance (CRAS) are units managed by the Brazilian Ministry of Social Development and Fight against Hunger (MDS) spreaded along the 5,560 municipalities in which social services are provided. These centers are partially supported by the Federal Government, via the MDS, having the management under the municipalitiesâ&#x20AC;&#x2122; responsibility. Their decentralized structure led to an expressive participation of municipalities. On the other hand, it hindered the follow-up of the 1

The CRAS Census is carried out annually by MDS and refers to the data collection and analysis in the Centers of Reference in Social Assistance. This census has been performed regularly since 2007.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

Eliciting and defining requirements based on metaevaluation: the case of the CRAS 2008 Census

195

policies implementation in the municipalities and the quality management of this process by the Federal Government. For this reason, in 2007, by means of the Secretariat for Evaluation and Information Management (SAGI) and the National Secretariat for Social Assistance (SNAS), MDS carried out a census to quantify, identify, and collect information from CRAS. Along with this effort, the SAGI designed, created, developed, and has maintained the information systems for this purpose. By its side, the SNAS did the mobilization, system definitions, and the contact with the municipalities. For this census, the SAGI enabled an Internet site in which the municipal manager inserts information about the local CRAS. Also in 2007, a data collection about location, human resources, physical infrastructure, and capacity to articulate with other public and private agencies was done. The first results led the MDS to propose indicators and limits to be satisfied by the CRAS in terms of services provided, physical infrastructure, and quality, to be measured in future census. This way, a traditional census took an evaluation characteristic, by subsidizing the delimitation of criteria useful to assess the services offered by the CRAS. In the subsequent years, beyond the annual census for the CRAS, it was carried out a census in the Specialized Centers for Social Assistance Reference (CREAS), in the municipal and state boards of social assistance, and in the social assistance management. However, the evaluation process with indicators and evaluation criteria was first deployed in the CRAS Census conducted in 2007 (Brazil, 2008a). The results of this evaluation were published in 2010. The evaluation process of the CRAS Census was the first in the two involved secretariats that used information systems, indicators, training, and mobilization for enhancing the decision-making process in the State and Federal management level considering data collected from the Brazilian municipalities. Considering the possibility of improvements in any evaluation process, for example, in terms of quality or precision (Posavac; Carey, 2003), a metaevaluation for this purpose was carried out, based on the Joint Committee´s (1994) evaluation standards. According to Hedler (2007, p. 59), metaevaluation is “a research method for re-evaluating one or more steps of an evaluation study already done; the previous evaluation is compared with quality and validity standards accepted by the scientific community, and a new evaluation of the evaluation study in analysis is issued in the end”. The standards were used as references for the metaevaluation, as done in Hedler (2007), and applied to the CRAS Census process performed in 2007. The standards of the Joint Committee (1994) were applied to subsidize the improvements in the information systems used in the census. In short, a requirements elicitation was performed based on the results of the metaevaluation.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

196 Ferneda, E., Prado, H. A. do, Gadelha, M. G., Hedler, H. C.

THEORETICAL REFERENCES

3.1

Requirements engineering

Software Engineering (SE), as a research field, provides methodological support for the development/construction of software packages, making available techniques, methods, and standards that can be applied to the complete lifecycle of a software package. Usually, the first phase of the software development process is supported by the RE, a subdivision of the SE responsible for defining objectives and the limits of software (Pressman, 2006, p. 116; Paula Filho, 2009, p.165; Wiegers, 2003, p. 380). SE also offers standards like the Capability Maturity Model Integration (CMMI), adopted both by practitioners and academy, which has processes targeted to requirements development: requirements definition (SEI, 2010, p. 325) and requirements management (SEI, 2010, p. 341). Wiegers (2003, p. 47) divides the activities of the RE related to the development of requirements as: (i) elicitation, (ii) analysis, (iii) specification, and (iv) validation. The requirements elicitation phase focuses on discovering the requirements and the communication between the developers and the clients. If the communication step fails, the resulting software tends to be unfitted to satisfy the necessities and expectance of the client. This is the most critical phase (WIEGERS, 2003, p. 115) and, usually, applies interviews, information collection, and group discussion as the main methodological approaches. Pressman (2006, p. 118) points out the main problems in the requirements development process: (i) problems with the project scope, (ii) of problem understanding, and (iii) in the volatility or changes in the requirements during the project. Saiedian and Dale (1999) add other problems: (i) poor communication, (ii) resistance to changes from the involved people, (iii) problems of articulation among the involved people, and (iv) different perspectives among the target clients. Some software organizations offer alternatives to mitigate this problem by creating standards to be used in the RE: the CMMI from the Software Engineering Institute (SEI) and the General Guide for Improving the Brazilian Software Process (MPSBr) from Softex (2009) are some examples. Among the possible inputs for the requirements discovery (SEI 2010, p. 329-330) it can be found: (i) questionnaires, interviews, and scenarios; (ii) prototypes and models; (iii) market questionnaires; (iv) brainstorm; (v) use cases; (vi) business cases analysis; (vii) software tests; (viii) technology demonstration; (ix) business policies; (x) legacy products; (xi) regulatory statutes, and (xii) standards. In this work, the requirements elicitation was performed on the basis of programs evaluation standards. Based on the SE concepts and in the RE processes recommended by the IEEE (1990), and considering the software quality standard from ISO 91262 and the concept of 2

The ISO/IEC 9126 standard (INTERNATIONAL STANDARD ORGANIZATION, 2001) focuses on the quality of the software product. It establishes a quality model based on the following components: (i) the development process, (ii) quality of the final product, and (iii) software product quality in use.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

Eliciting and defining requirements based on metaevaluation: the case of the CRAS 2008 Census

197

Business Process Management – BPM3, Castro and Guimarães (2010) proposed eXtreme Requirements (XR) to the production of requirements according the phases: business analysis, solution proposal, requirements definition, prototyping, tests, and requirements managements. XR classifies the requirements in non-functional, functional, complementary, and business rules. Functional requirements are the functionalities or activities mandatory for the system to perform. Complementary requirements are characteristics or properties derived from the detailing of functional requirement. Business rules come from the organizational context, like regulations, conditions, or standards required to perform each functionality. Non-functional requirements are characteristics related to the software quality. 3.2

Evaluation

For Worthen, Sanders and Fitzpatrick (2007, p. 35), an evaluation refers to the identification, clarification, and application of defensible criteria to determine the value, merit, usefulness, effectiveness, or importance of the object evaluated. Stufflebeam and Shinkfield (2007, p. 16) define evaluation as a systematic process to delimitate, obtain, report, describe, and judge the information on the merit, value, integrity, feasibility, security, significance or equity of an object. Weiss (1997, p. 4) argues that evaluation is an approach to attest, systematically, the operation and results of a policy or program compared to a set of standards as a way to contribute to the improvement of the policy or program. It can be noticed, by these definitions, that evaluations should judge or clarify an issue on the basis of standards or criteria in order to qualify a social program, a person, an organization, or a process. The first definition is broader, mainly due to the fact it is not limited to social programs, and was the one adopted in this research. According to Worthen, Sanders and Fitzpatrick (2007, p. 44), the results of an evaluation can bring improvements to the object, program, or policy evaluated. Conversely, an evaluation barely constructed and implemented is a poor guide for management decisions. The reason for an evaluation failure can be a bad methodological planning or even the lack of ethics from the people or organizations involved in the evaluation process. The need to attest the quality and improve the construction of new evaluations led several organizations and authors to propose standards. These standards can be used as metaevaluation mechanisms. According to Stufflebeam (2001), “metaevaluation is the process of delineating, obtaining, and applying descriptive information and judgmental information about an evaluation’s utility, feasibility, propriety, and accuracy and its systematic nature, competence, integrity/honesty, respectfulness, and social responsibility to guide the 3

Business Process Management is a management approach focused on identifying, designing, implementing, documenting, measuring, monitoring, controlling and improving business processes, automated or not, to achieve the desired results, consistent and aligned with the strategic goals of an organization (ASSOCIATION OF BUSINESS PROCESS MANAGEMENT PROFESSIONALS, 2009).

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

198 Ferneda, E., Prado, H. A. do, Gadelha, M. G., Hedler, H. C.

evaluation and publicly report its strengths and weaknessesâ&#x20AC;?. Among the evaluation standards available, there are the Educational Evaluation Standards from the Joint Committee (1994), the Guiding Principles for Evaluators from the American Evaluation Association (AEA, 2004) and the Government Auditing Standards from the U.S. Government Auditing Office (USGA, 2007). The Joint Committee was formed initially by a group of authors in the evaluation field that worked together from 1970 to 1990, discussing sets of standards to guide and evaluating the construction of evaluations. It proposes 30 standards, organized in four groups, created to subsidize the evaluation of programs and educational projects, aiming at stimulating and improving the interchange of ideas among professional involved in evaluations. The Joint Committee (1994, p. 4), however, encourages the use of standards from other evaluation methods that, according Stufflebeam and Shinkfield (2007, p. 92), are also applicable to metaevaluation. The Guiding Principles for Evaluators (AEA, 2004) has five principles: (i) systematic inquiry, (ii) competence, (iii) integrity and honesty, (iv) respect for people, and (v) responsibilities for general and public welfare. Stufflebeam and Shinkfield (2007, p. 110), however, argue that these principles are already considered in the Joint Committee standards. The guiding principles of the Government Auditing Standards (USGA, 2007) were proposed to assure the achievement of high quality auditing that is essential to accountability and transparence of public resources investments. For this, auditing must be objective, based on facts, impartial, capable of measuring the program performance, and make available information related to decision making. According to Stufflebeam e Shinkfield (2007), these standards present similarities with the Joint Committee proposal and consider standards for independent auditing, judgment of professionals, competence, control and quality, fieldworks, reports, and performance of auditing. The standards from the Joint Committee - USGA and AEA - as suggested by Stufflebeam and Shinkfield (2007, p. 109), are diverse in details and guidance, noncontradictory and complementary. So, for this work, the Joint Committee criteria (1994) were adopted. 4.

METHODOLOGICAL ISSUES

This work departs from the assumption that an evaluation involving Information Systems (IS) can be metaevaluated in order to generate input to the requirements elicitation of software aiming at improving future evaluations. Figure 1 illustrates this assumption, according to the continuous improvement approach shown as a spiral that starts in the computerized evaluation.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

Eliciting and defining requirements based on metaevaluation: the case of the CRAS 2008 Census

199

Figure 1 - Improvement cycle of evaluation systems by means of metaevaluation based on information systems 4.1

Specifying a method to study the case CRAS 2008 Census

The evaluation of the CRAS 2008 Census was taken as starting point to indentify new requirements in the evaluation process of the CRAS. The requirementsâ&#x20AC;&#x2122; elicitation was carried out on the basis of results from the metaevaluation performed using the Joint Committee standards (JOINT COMMITTEE, 1994). This metaevaluation involved an investigation based on interviews, documents, and process regulation. The standards from the Joint Committee were chosen due to its generality, comprehensiveness, and wide acceptance by the scientific community (Coosky and Caracelli, 2009). Moreover, these patterns suggest recommendations and common errors in assessments, enabling its use as a guide for identifying weaknesses in the evaluation process whose solution may involve the use of Information and Communication Technologies (ICT). Table 1 lists the documents, systems, and interviews examined in the metaevaluation of the CRAS.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

200 Ferneda, E., Prado, H. A. do, Gadelha, M. G., Hedler, H. C.

Recommendations and errors

Source* 1 2 3 4

Standard “Complete and Fair Evaluation” Recommendations 1. The reports must indicate the strengths and weaknesses in the evaluation and justify them. 2. Request critical and meticulous comments from the people involved in the process before publication of results. 3. Report restrictions on the cost and time for completion of the evaluation.



 

Main errors occurred in the evaluations - Joint Committee (1994) 4. Manipulation of strengths and weaknesses in order to favor individuals or groups. 5. Promotion or protection of personal interests of evaluators or bias in the evaluation. 6. Report judgments or interpret findings of strengths or weaknesses without taking into account other points of view that can change the conclusions. 7. Report speculatively or try to control how the strengths or weaknesses are presented. 8. Do not show what methods were used to define the strengths and weaknesses. 9. Report only negative points.

 

Standard “Systematic Information” Recommendations  

1. Assure that the evaluation team is trained to follow the data collection rules. 2. Verify systematically errors in the collection, processing and dissemination of information in order to assure quality in the process. 3. Using external audits to verify data entry. 4. Keep track of the original data so that the integrity of the information can be protected. 5. Adopt and implement standards and procedures for information retrieval. 6. Create routines with stakeholders to confirm that the data is accurate.

   

Main errors occurred in the evaluations - Joint Committee (1994) 7. 8. 9. 10. 11. 12. 13. 14.

   

Assume whoever collects information follows the rules. Assume that the people in charge read, understand, and follow the instructions given them. Do not check the accuracy of instruments for data collection. Assume that the results of tests performed by machines are correct only because they were made by machines. Failures in the management of large volumes of information. Hiring inexperienced people. Ignore control of large volume of information. Failure in controlling information access.

   

Standard “Conclusions Justified” Recommendations 1. Develop conclusions that address simultaneously the issues of assessment and reliably reflect the processes and findings. 2. Relate the findings with the information collected. 3. Generating, evaluating, and reporting alternative conclusions and indicate why these findings were not used. 4. Limiting what the situations, time, people, contexts and purposes were for each finding of the evaluation. 5. Caution from the staff when interpreting the data found. 6. Request "feedback" from the participants of the evaluation on the credibility of the interpretation, explanations, conclusions and recommendations before completing the report.



 



 



Main errors occurred in the evaluations - Joint Committee (1994) 7. Focus on answering the questions involved without worrying about the limitations of the assessment procedures and data. 8. Support the conclusions on insufficient or inadequate information. 9. Be very cautious in interpreting the evaluation results. 10. Failure in reporting the evaluation limits.

   

  

* (1) Evaluation report; (2) Management system of the CRAS 2008; (3) Business expert; (4) IT expert

Table 1 - Recommendations and main errors according to the instructions from the Joint

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

Eliciting and defining requirements based on metaevaluation: the case of the CRAS 2008 Census

201

Committee (1994) The Joint Committee (1994) provides standards to verify the adequability, usefulness, and precision of an evaluation. Three standards were chosen for the analysis: (i) Evaluation Complete and Fair, (ii) Systematic Information, and (iii) Conclusions Justified. These standards were chosen by a committee of three senior professionals in the ICT that agreed that these standards are related to the identification of new software requirements. As detailed in Table 1, the main input to the metaevaluation was: (i) the evaluation report (Brasil, 2010), (ii) semi-structured interview with a business expert, (iii) the software Manager CRAS 2008 (BRASIL, 2008b), and (iv) semi-structured interview with the IT expert from the MDS. Both the interviewees were active in the evaluation process. The business expert worked the CRAS 2008 evaluation process from the formulation of questions to the generation of the final evaluation report. The IT expert participated in the software development and followed the information processes of the evaluation. According Coosky and Caracelli (2009), this input is frequently used in metaevaluation processes. The requirements definition was done by means of the Requirements Definition Document (RDD), proposed by Castro and Guimarães, as part of eXtreme Requirements method. This artifact is used to identify the software requirements, the business rules, the traceability matrix, and the prioritization of requirements from the business processes evaluated. This document includes: (i) functional requirements, (ii) complementary requirements, (iii) non-functional requirements, (iv) business rules, (v) processes flow, (vi) users list, and (vii) risk analysis. The traceability of requirements was not considered since the focus did not include the requirements management. 5. RESULTS OF CRAS 2008 METAEVALUATION AND REQUIREMENTS ELICITATION According the standard “Evaluation Complete and Fair”, an evaluation must point out the strengths and weaknesses of the evaluated program, allowing the emphasis of successful issues beyond the correction of existing errors. A summary of the data collected with respect to this standard is shown in Table 2.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

202 Ferneda, E., Prado, H. A. do, Gadelha, M. G., Hedler, H. C.

Recommendations 1. The reports must indicate the strengths and weaknesses in the evaluation and justify them. 2. Request critical and meticulous comments from the people involved in the process before publication of results. 3. Report restrictions on the cost and time for completion of the evaluation. Main errors 4. Manipulation of strengths and weaknesses in order to favor individuals or groups. 5. Promotion or protection of personal interests of evaluators or bias in the evaluation. 6. Report judgments or interpret findings of strengths or weaknesses without taking into account other points of view that can change the conclusions. 7. Report speculatively or try to control how the strengths or weaknesses are presented. 8. Do not show what methods were used to define the strengths and weaknesses. 9. Report only negative points.

Results

Improvements in IT

Yes

Partial

Made available a transactional tool for recording comments

No Results Undetermined

Suggestion of a financial/accounting information system to record the expenses or the use of any software project management Improvements in IT Suggested the use of software agents to verify all published data Made available a transactional tool for data verification by the States

Partial

Made available a transactional tool for enabling that States and municipalities issue their opinions

Yes

Table 2 - Analysis of the standard “Complete and Fair Evaluation” Next standard, “Systematic Information”, advocates that the information collected, processed, and included in the reports must be revised and corrected if an error is found. The results by applying this standard are shown in Table 3. The main IT improvements found in the application of this standard refer to communication among the people involved in the evaluations and user authentication.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

Eliciting and defining requirements based on metaevaluation: the case of the CRAS 2008 Census

Recommendations 1. Assure that the evaluation team is trained to follow the data collection rules. 2. Verify systematically errors in the collection, processing and dissemination of information in order to assure quality in the process. 3. Using external audits to verify data entry. 4. Keep track of the original data so that the integrity of the information can be protected. 5. Adopt and implement standards and procedures for information retrieval. 6. Create routines with stakeholders to confirm that the data is accurate. Main errors 7. Assume whoever collects information follows the rules. 8. Assume that the responsible people read, understand, and follow the instructions given them. 9. Do not check the accuracy of instruments for data collection. 10. Assume that the results of tests performed by machines are correct only because they were made by machines. 11. Failures in the management of large volumes of information. 12.

Hiring inexperienced people.

13. Ignore control of large volume of information. 14. Failure in controlling information access.

203

Results

Improvements in IT

Yes

Partial

Improve the data collection module to strengthen the interaction with the user

Partial

The system should enable notification of problems found in data by external audits

Yes

Results

The system Manager should enable the visualization of data collected by an organized society with a communication channel to report bugs or request clarification Improvements in IT

Partial

yes

Partial

Improvements in the process of authentication in order to assure that the responsible for feeding the system was trained Improvements in the process of tools testing

Partial

Improvements in the authentication mechanism

Table 3 - Analysis of the standard “Systematic Information” In the standard “Conclusions Justified”, the Joint Committee (1994) argues that the conclusions of an evaluation must be explicitly justified to enable their analysis by the people involved in the evaluation or program. In Table 4 a summary of the analysis based JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

204 Ferneda, E., Prado, H. A. do, Gadelha, M. G., Hedler, H. C.

on this standard is shown. The main improvement arising with respect to IT is to enable the people involved to provide feedback related to their experience in the evaluation. Recommendations 1. Develop conclusions that address simultaneously the issues of assessment and reliably reflect the processes and findings. 2. Relate the findings with the information collected. 3. Generate, evaluate, and reporte alternative conclusions and indicate why these findings were not used. 4. Limit what the situations, time, people, contexts and purposes were for each finding of the evaluation. 5. Caution from the staff when interpreting the data found. 6. Request "feedback" from the participants of the evaluation on the credibility of the interpretation, explanations, conclusions and recommendations before completing the report. Main errors 7. Focus on answering the questions involved without worrying about the limitations of the assessment procedures and data. 8. Support the conclusions on insufficient or inadequate information. 9. Be very cautious in interpreting the evaluation results. 10.Failure in reporting the evaluation limits.

Results

Improvements in TI

Yes

To collect data by means of an information system to cope with the feedback from the involved people

Results

Improvements in IT

yes

Table 4 - Analysis of the standard “Conclusions Justified” Next, a detailed discussion on the results from the standard “Complete and Fair Evaluation” is presented in which items 1, 2, and 3 refer to the recommendations and the others (4 to 9) are related to the main errors: 1. All indicators that point out the strengths and weaknesses for each CRAS were found in the system Manager. There is no information on the methodology for generating these indicators; however, this information is included in the evaluation report (Brasil, 2010, p. 137-173). 2. There are no records of comments from the people involved neither in the system Manager nor in the final report. Asked about this fact, the business expert reported meetings occurred with the representatives from States, municipalities, and the

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

Eliciting and defining requirements based on metaevaluation: the case of the CRAS 2008 Census

205

Federal Government to criticize the evaluation process, but no record was done. An improvement could be the adoption of a transactional information system integrated with the Manager to enable the managers from States and municipalities to criticize the data and indicators to be published. 3. Nothing was found with respect to this recommendation neither in the report nor in the system Manager. There are financial and accounting information systems that can be used in the evaluation. It is also possible to adopt systems for project management. 4. No inconsistency was detected. However, the analysis of data requires further investigation that could be accomplished, for example, by a software agent 4 that compares the results with the original data. States can also check the results via system Manager and with the support of a system for data recording and information about the manipulation of any report. 5. The business expert stated that neither promotion nor protection of the MDS interests happened. 6. There is no information about any other possible approaches for analyzing data and indicators neither in the system Manager nor in the final report. 7. According to business expert, the generation of indicators was made preventing any great loss to the CRAS and municipalities. 8. The methods for defining strengths and weaknesses (Brasil, 2010 p. 137173) are available in the evaluation report. 9. Both the system Manager and the evaluation report present the strengths and weaknesses of each CRAS. A summary of the strengths and weaknesses found and respective recommendations for the IT team is shown in Table 5. This table details the requirements elicitation based on the metaevaluation.

According to Russell and Norvig (2010, p. 34), an agent is an entity that perceive its environment by mean of sensors and actuate over this environment by means of actuators, processing information and knowledge.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

206 Ferneda, E., Prado, H. A. do, Gadelha, M. G., Hedler, H. C.

Standards

Complete and Fair Evaluation

Weaknesses Absence of critical and meticulous comments from the involved people before the publication of the results. Issues related to costs and timetable were not reported.

Publication of reports without considering other points of view. Verification of errors in the data collection.

Absence of external audits to validate the data.

Information Systematic

Lack of routines to validate the data by the stakeholders.

Assume that the involved people read and understand the instructions for data collection.

Justified Conclusions

Verification of the tools for data collection. Failure to control and access to information. Request feedback from participants about the credibility of the evaluation and interpretation, explanations, conclusions and recommendations before completing the report.

Recommendations 1. A transactional tool for recording comments can be made available. 2. A financial/accounting information system can be used to record the expenses or any software for project management can be adopted. 3. Development of a transactional tool to record opinions of States and municipalities. 4.1 It is possible to use software agents to verify all published data. 4.2 Offer of transactional tool for the States validates the data. 5. The checking and data conferencing could be more formal by means of a collaboration tool with the participation of States and municipalities 6. The checking and data conferencing could be more formal by means of a collaboration tool with the participation of States and municipalities 7. Improve the authentication process of system to ensure that the people who completed the system were trained. 8. Improve the process of tool testing. 9. Improve the authentication process.

10. Collect data by means of an information system, considering the feedback from participants.

Table 5 - Summary of the results from the requirements elicitation by means of metaevaluation of the CRAS 2008 Census 6.

DEFINITION OF SOFTWARE REQUIREMENTS

Based on the recommendations raised during the metaevaluation of the CRAS 2008 Census, a definition of desirable requirements was formulated for the improvement

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

Eliciting and defining requirements based on metaevaluation: the case of the CRAS 2008 Census

207

of the ITC solutions available in the context of the CRAS evaluation. For this, the construction of requirements for recommendations 1, 3, 4.2, 5, 6, 7, 9, and 10 was chosen, as shown in Table 5. These recommendations refer to the registration information and comments from States and municipalities regarding the collected data or conclusions of the evaluation. Thus, the construction of a system that provides a formal channel of communication between federal, State and municipal levels and that integrates in the “Manager CRAS System” is one way of implement the recommendations suggested by the metaevaluation. Recommendations 2, 4.1, and 8 were not considered because they are not directly related to an activity of designing and developing information systems. 6.1

The activity flow of the CRAS 2008 Census

The activity flow of the evaluation of the CRAS 2008 is shown in Figure 2. In this figure, each column represents one of the three actors involved in the process (MDS, STATES and MUNICIPALITIES) and activities under their responsibility. It begins making available, by the technical and business team from the MDS, of the online questionnaire for completion by managers of the CRAS. Completed questionnaires are analyzed by the technique and business team of the MDS and disseminated to the States and other officials through the “Manager CRAS System”. However, feedback, critiques and supplementary information from States can only be done informally and not controlled. Therefore, the MDS is solely responsible for the analysis and interpretation of data and the generation of evaluation findings. In other words, there is no formal participation of States and municipalities in the review process.

Figure 2: Activity flow from the CRAS 2008 Census

In the suggested process (Figure 3), a communication module among the MDS,

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

208 Ferneda, E., Prado, H. A. do, Gadelha, M. G., Hedler, H. C.

States, and municipalities was added in the CRAS (MODCCRAS) evaluation. This module allows the formal record of comments and questions and data verification by all involved in the evaluation. This way, they can opine about the evaluation process and in the generate information. These opinions can be considered for the analysis and support the decision on new treatments of information. Among the evaluation results there are the problems found in the implementation of policies related to the CRAS that are potentially useful to recast these policies.

Figure 3: Activity flow proposed 6.2

Requirements

For the implementation of process depicted in Figure 3, it is proposed the functional requirements, complementary requirements, business rules, and non-functional requirements, respectively shown in Tables 6, 7, 8, and 9.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

Eliciting and defining requirements based on metaevaluation: the case of the CRAS 2008 Census

Id:

209

Description

- Comments and Critics Module RF1: The system must record all comments/criticisms of the managers of the MDS / States / Municipalities (RC1, RC2, RNG1, RNG2, RNG25) RF2: The system should record comments/criticisms of each table, graph and map available in the “Manager CRAS System” (RC1, RC2, RNG3) RF3: The system should allow MDS users to delete criticisms and comments (RNG5, RNG6) RF4: Generate Query: quantitative Comment (RC3) RF5: Search for comments / criticisms that contain specific words (RC4, RNG7) RF6: Find all comments by a particular user (RC5) RF7: Search amount of user access (RC6) RF8: Generate query: list comments / criticisms unanswered (RC7, RNG4, RNG7) RF9: The system should allow MDS users to moderate posts (RC8, RNG6) - Errors Control Module RF10: The system should record all errors found by the managers of the MDS / States / Municipalities (RC8, RNG8, RNG9) RF11: The system should log errors in each table, graph and map available in the “Manager CRAS System” (RC8, RNG9, RNG10) RF12: The system should allow MDS users to moderate error messages (RC9, RNG11) RF13: The system should allow users to delete the MDS error messages (RNG11, RNG12) RF14: The system should allow registering possible solutions to the problems encountered (RNG13) RF15: Generate query: List of reported errors that remain to be answered (RC10) RF16: Generate query: List users who reported errors (RC11) - Poll Module RF17: The system should allow the construction of a poll by MDS (RC12, RNG14) RF18: The system should allow to create a list of users who will fill out the survey (RC13, RNG15) RF19: The system should allow MDS users to enable compulsory voting in a poll (RNG15, RNG16) RF20: The system should allow the MDS association with polls, comments and criticisms (RNG17) RF21: Generate Report: for each poll, the system generates a report with the result (RC14, RNG18) - Authentication Module RF24: The system can only give access to users who perform authentication (RC15, RNG19, RNG20) RF25: The system should allow users with an “MDS” profile to create users with “States” and “Municipalities” Profiles (RC16, RNG21) RF26: The system should identify users from the MDS through the LDAP (RNG22) RF27: The system should allow users of States to create the municipality users (RC16, RNG23) RF28: The system must allow the MDS users to lock states and municipality users (RNG23) RF29: The system must allow users of states to lock municipal users (RNG24) RF30: Generate report with all system authentications (RC17)

Table 6 - Functional requirements

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

210 Ferneda, E., Prado, H. A. do, Gadelha, M. G., Hedler, H. C.

Id: RC1: RC2: RC3: RC4: RC5: RC6:

RC7: RC8: RC9: RC10: RC11: RC12: RC13: RC14: RC15: RC16: RC17:

Description The comments / criticisms must be enabled to attachment files Review / critique must register the user, date and time Number of access per user, total quantity of access and users who have not accessed the tools Search in title and body of the comment Search by username, all comments / criticisms made by this manager A report containing the login user ID, name, type of manager (MDS / State / Municipality) last access, number of accesses performed and amount of comments / criticisms made should be generated The system must record each error by a unique identifier with date and time of each release Every record must contain a unique ID, title, date, time and user Allow or disallow the publication Detailing the time, date, user ID number and allow the order based on these criteria List of users who reported errors in alphabetical order The poll must have an identification number, a theme and select Options. User profiles are the MDS, states or municipalities The report must contain: users who answered the poll and results The user must be authenticated by a valid email address and an alphanumeric password with six or more digits Name, CPF, state, county, type of manager (state or municipality) must be registered The report must include date, time, ip, user, type of manager

Table 7: Complementary requirements Id: RNG1: RNG2: RNG3: RNG4: RNG5: RNG6: RNG7: RNG8: RNG9: RNG10: RNG11: RNG12: RNG13: RNG14: RNG15: RNG16: RNG17: RNG18: RNG19: RNG20: RNG21: RNG22: RNG23: RNG24: RNG25:

Description States should be able to record comments / criticisms of tables, graphs and maps available in the â&#x20AC;&#x153;Manager CRAS Systemâ&#x20AC;? Municipalities should be able to record comments / criticisms of tables, graphs and maps available in the Manager CRAS System The MDS / States / Municipalities can answer all comments and criticisms The system must allow the ordering of comments / criticism on the criteria of quantity of visits, voting participants and chronologically The MDS should have right to delete existing comments and criticisms The MDS must have the right to moderate comments and criticisms The System must display the unanswered comments and criticisms The MDS should be able to log errors found in data States should be able to log errors found in the data Municipalities should be able to log errors found in data The MDS should be able to moderate the error logs The MDS must have the right to delete registry errors The MDS should be able to record possible solutions to the problems encountered The MDS must have the option of opening polls for municipalities / states to comment Each poll should be planned to receive a list of municipalities / states The vote of the people on this list may or may not be mandatory Each poll must be associated with a discussion / review / critique. Each poll should generate a report available for the MDS Each system user must have a login and password Users of the MDS / States / Municipalities must authenticate to gain access to the system The MDS should be able to register the teams of States and Municipalities Users of the MDS must be identified with the LDAP authentication States are responsible for the registration of local users MDS is able to block any user Comment or criticism must have a unique identification number

Table 8: Business rules proposed

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

Eliciting and defining requirements based on metaevaluation: the case of the CRAS 2008 Census

Id:

211

Description

Functionality RNF1: All reports must be issued in available formats: HTML, PDF and CSV - Reliability RNF2: The data generated by the software must be stored in a RAID 1 environment and the backup must be in another environment or city RNF3: Every data exchange should be carried out using a secure connection (HTTPS) - Performance RNF4: The system must support up to 100 concurrent transactions - Portability RNF5: The system must be compatible with browsers (Internet Explorer , Firefox and Chrome) RNF6: The system should run on web platform (Linux + Apache + PostGres + PHP) - Usability RNF7: The system must be accessible to the visually impaired - Maintainability RNF8: The system needs to have availability of 99% of time in operation RNF9: The databases should be checked daily for errors and failures through intelligent agents

Table 9: Non-functional requirements The functional requirements were divided into components that comprise the MODCCRAS module: (i) comments and criticisms, (ii) error control, (iii) survey, and (iv) authentication. The purpose of these four components is to troubleshoot authentication and registration of feedback, critics and errors. The non-functional requirements of the system were not obtained directly from the metaevaluation process; they were discussed with the IT expert who participated in this process. Thus, they were inserted by being part of the artifact RDD of the eXtreme Requirements method. 6.3

Profiles, permissions and risk analysis

There are three user profiles accredited in the system: employees of the MDS, users of States, and users of municipalities. Everyone can access any component, but with pre-defined constraints on functional requirements and business rules. Risk analysis consists of mapping the possible problems or interferences that a project may face during its execution. For example, the risk mapped for this project comes from municipalities and States that participated in the evaluation process. As shown in Figure 4, without their active participation, it would not be possible to construct an evaluation fitting the recommendations of metaevaluation.

JISTEM, Brazil Vol. 11, No.1,Jan/Apr 2014, pp. 193-214

www.jistem.fea.usp.br

212 Ferneda, E., Prado, H. A. do, Gadelha, M. G., Hedler, H. C.

Identification

Projection

Risk assessment

Identification

Description