Development of a measure of Generic Work Competencies. by Ioannis Nikolaou

The Development and Validation of a Measure of Generic Work Competencies Ioannis Nikolaou Department of Management Science and Technology Athens University of Economics and Business

Competency management has attracted much attention, especially between business consultants and human resources professionals. Nevertheless, the lack of a unified framework of generic work competencies has been a significant obstacle for the further development of the field, both in research and in practice. This article discusses the development, validation, and psychometric properties of a measure of generic work competencies assessing three types of generic work competencies.

The assessment of work competencies and especially the development of valid and reliable measures for their assessment have attracted limited attention among researchers in the field of personnel psychology. On the contrary, human resources practitioners have used the competency approach at length, especially during the 1980s and 1990s. Boyatzis (1982) first defined competencies as the certain characteristics or abilities of the person that enable him or her to demonstrate the appropriate specific actions, thereby leading to effective work performance. Thus, on an individual level, competencies are the capabilities the employee brings to the job. Sparrow (1997), in a review of the use of organizational competencies in personnel selection and assessment, defined competencies as those behavioral repertoires (sets of behavioral patterns) that some people can carry out more effectively than others, including all those behaviors that employees bring into the organization to perform well. Boam and Sparrow (1992), describing the major changes occurring in human resources management, argued that organizations are looking for an approach that will enable them to bring changes “by describing global issues in a way that is sensitive to the local context … drawing upon the language of line managers Requests for reprints should be sent to Ioannis Nikolaou, 39 Euripides St., Piraeus, Greece 185 32. E-mail: inikol@aueb.gr

310

NIKOLAOU

and facilitating long-term changes in individuals’ behaviour” (p. 11), suggesting that competency-based approaches can offer this alternative. Sparrow and Bognanno (1993) claimed that the strategic change in organizations during the 1990s reinforced the increased use of competencies. The realization that success in the new competitive environment depends largely on the ability to learn faster than competitors, and subsequently reconstruct and adapt the organization, has focused attention on internal resources and the capabilities of the organization, such as employees’ competencies, and the integration of human resources policies and practices with business strategies. Another factor in the development of the competency-based approach may have been the recession in the early 1980s, which was followed by the globalization of business in the 1990s. Traditional personnel practices proved no longer effective following the wave of change in western European and North American economies. The established job descriptions tailored along the lines of employees and managers, who heretofore retained their positions until retirement, ceased to exist, probably forever. Expressions (e.g., leadership potential, innovation, creativity, strategic vision) started appearing as qualifications in job advertisements clearly seeking a new type of employee. The competencies approach may be a very influential part of the human resources strategy of any organization. Because the appropriate competencies have been identified, they can be applied in many organizational activities, including personnel selection and assessment, training, career development, and performance management. Sparrow and Bognanno (1993) claimed that the creation of a core set of effective behaviors such as competencies may be used in a large set of assessment settings encouraging mutual behavioral reinforcement across human resources policy areas. Feltham (1992) claimed that the competency-based approach contributed to the effectiveness of selection and assessment methods in three ways: the process, the implementation, and the evaluation of the approach. The main benefits in adopting a competency approach in selection and assessment, according to Feltham (1992), are creating shared understanding among various levels of the hierarchy of the kind of personnel needed for new systems; creating more informed human resources options; agreeing on standards and enlisting more systematic and scientific recruitment processes; writing realistic job previews; and identifying the most appropriate assessment methods. Another application of the competency approach, according to Craig (1992), is in career planning and development. He also argued that competencies in career development are important in two stages: prior to promotion and toward the middle of the job grade when both organization and the individual should be preparing for movement toward the next grade. He described the following flow:

• Competencies set standards for progression.

GENERIC WORK COMPETENCIES

311

• Competencies provide the referent for assessment. • Competencies, expressed in terms of strengths and development needs, set the referent for development. • Competencies set the standards for progression to the next level. The use of competencies in training resulted from the need to establish cost–benefit links between training and improved job performance (Antonacopoulou & FitzGerald, 1996). Applying competencies in training may be very appropriate in deciding what to train (Weightman, 1994) and in identifying specific competencies that require further improvement at an individual, departmental, or organizational level. Another area where competencies have been widely used is in competence-related pay and performance-related pay schemes (Torrington & Blandamer, 1992). The suitability of competencies in those schemes is justified because they can clarify and identify objectives and goals that have to be reached, both for the appraiser and the appraisee, thereby facilitating the appraisal process. In addition, it offers a common language between them, as well as between personnel management and trade unions. Armstrong and Brown (1998) claimed that the use of competencies can help to address traditional failings of performance-related pay, being particularly appropriate in sectors where employees’ skills and behaviors are considered key to competitive success. Kurz and Bartram (2002), describing a theoretical model of competency and individual performance, incorporated competency assessment as an integral part of their model for the World of Work, and Tett, Guternamn, Bleier, and Murphy (2000) developed a taxonomy of 53 competencies clustered under nine general areas: traditional functions, task orientation, dependability, open-mindedness, emotional control, communication, developing self and others, occupational acumen, and concerns. A thorough examination of the competency management literature could not identify a well-researched and documented model or an instrument measuring generic work competencies across occupations and job levels. The Hay Group, a consulting company, has developed a model of competencies that, although its practical application is extensive worldwide, it is not well supported with research evidences regarding its psychometric properties. All of the researchers or practitioners who had worked in the field used organization-specific measures, sometimes developed after a job analysis. Although significantly, organizations did develop tailor-made measures assessing a company’s core competencies, some felt that the existence of a simple, reliable, valid, and easy-to-use measure of generic work competencies would facilitate research and practice in the field of competency management. Personnel psychology researchers will be able to use the new instrument in criterion-related studies, and human resources practitioners will have a basis to use when developing their own competencies measures. Therefore, the cur-

312

NIKOLAOU

rent study describes an attempt to develop and validate a measure of generic work competencies. METHOD Based on a detailed literature review, experts’ratings analyzed with verbal protocol analysis (Green, 1995), and a small study carried out in the United Kingdom (Nikolaou, 1999), four major clusters (task orientation, the action–leadership orientation, the people, and the communication skills orientation) were identified with 40 items. In the task-orientation cluster the employee is actively engaged and orientated toward accomplishing his or her duties or tasks, working systematically toward it. The employee’s major concern is successfully executing the job duties. In the action–leadership orientation the employee is a leading figure within the work environment working actively and dynamically to inspire and motivate colleagues. In the people- orientation cluster the employee is sensitively and positively orientated toward his or her colleagues, taking into account their opinions and suggestions. The employee respects her or his colleagues, showing consideration for their problems. Finally, in the communication skills cluster the employee has good oral and written communication abilities. During oral presentations he or she communicates in a simple and comprehensible way, clearly and fluently. The employee writes clearly and concisely, using correct grammar, style, and language. The participants were managers in various Greek firms, and they were asked to rate their immediate subordinates on each of the 40 items on a scale ranging from 0 to 7. The higher the value the employee received, the better the employee compared on this particular competency to his or her colleagues. Participants were also asked to complete a short (6-item) questionnaire assessing overall job performance, which would be used as a separate index of the criterion-related validity of the competencies measure. This measure was successfully used in numerous studies (e.g., Robertson, Baron, Gibbons, MacIver, & Nyfield, 2000; Robertson, Gibbons, Baron, MacIver, & Nyfield, 1999) as an overall job performance measure eliciting internal consistency reliability of .86. It consists of 6 items where the supervisor has to indicate whether he or she agrees or disagrees with the behavior described on a 5-point scale. Sample items include: “S/he achieves the objectives of the job, demonstrates expertise in all aspects of the job, fulfils all the requirements of the job,” and so forth. An overall job performance score was calculated by averaging the raw scores of the 6 items. RESULTS AND CONCLUSIONS Study 1 Twenty-four managers participated providing useful data for 107 subordinates. The majority of the managerial sample consisted of men (58%) between ages 35

GENERIC WORK COMPETENCIES

313

and 45 (67%) who were university graduates (46%). Table 1 shows the descriptive results of the four scales of the competencies clusters and the job performance index along with their intercorrelations. For ease of comparison, the scale scores are the average of the individual items of each scale. Table 1 clearly shows that the competency scales are highly intercorrelated. The first two clusters, the task and the action–leadership orientation, particularly have a very high correlation coefficient of .92. Also, the job performance index has statistically significant correlations with all four clusters. According to the managers of the sample, a strong positive relation exists among all four types of work competencies and job performance, although the strength of this relation varies among clusters of competencies. The reliability of the competencies measure was very high. The Cronbach’s alpha was .98 and the Guttman’ s split-half was .97 (corrected for length). All the individual scales had also very satisfactory Cronbach’s alpha ranging from .92 up to .96. Similar results came up for the job performance index (α = .95). The high internal consistency reliability is attributed to the high homogeneity and similarity of the competency items. To explore the adequacy of the sample for carrying out the factor analysis, the Kaiser-Meyer-Olkin measure was estimated, reaching a value of .93, which is considered very acceptable (Norussis, 1994). Moreover, another index of the sample suitability, the Bartlett’s test of sphericity, reached a statistically significant level. Following Gorsuch’s (1983) recommendations, these are two basic requirements to proceed with the exploratory factor analysis because they explore whether the intercorrelation matrix is suitable for performing the factor analysis. An initial exploratory factor analysis using principal components and oblimin rotation was carried out. The extraction method of principal components was chosen to extract linear combinations of the variables explaining the majority of the variance (Kline, 1994), whereas oblimin rotation was applied because it was expected that the resulted factors would be highly intercorrelated. The latter was indeed the case as Table 1 shows. The results of the factor analysis yielded four factors explaining 76% of the variance; the people-orientation cluster and the TABLE 1 Descriptives and Intercorrelations of the Scales (Study 1) Scales Task orientation (1) Action–leadership orientation (2) People orientation (3) Communication skills (4) Job performance (5) Note.

107 107 107 107 107

4.64 4.25 4.86 4.81 3.62

1.61 1.60 1.45 1.38 0.99

.92 .96 .96 .93 .95

.92 .64 .82 .76

.72 .78 .75

.62 .44

.60

All correlations are significant at the .01 level (two-tailed).

314

NIKOLAOU

communication skills orientation emerged very clearly but the remaining items were distributed between the remaining two factors. The first point that should be made here is that the participants could apparently distinguish the concepts of people orientation and communication skills easier than the other two competencies clusters (i.e., the task orientation and the action–leadership orientation). Subsequently, these two clear-cut factors (people and communication skills) were included in the subsequent analyses and that the focus would turn to the interpretation of the remaining two factors. Numerous exploratory factor analyses were conducted, repetitively excluding items with low factor loadings, trying to identify a simple, coherent, and interpretable factor structure, but all the results were similar. Four factors were always coming up; two of them were the people and communication factors, whereas the other two were combinations of the task and action–leadership items. Subsequently, because the problem mainly concerned the task and action–leadership clusters, one of the clusters was dropped because the participants could not distinguish between these two groups of competencies. Thus, the action–leadership cluster was kept and the task- orientation items were omitted. This was considered more appropriate because it would be the only way of measuring leadership potential, which was of more interest to the participant-managers, who also received feedback for their responses, compared to the task-orientation competencies. A hierarchical regression analysis (Cohen & Cohen, 1983) was also conducted to examine whether the remaining items could predict the dropped items of the task-orientation cluster. The three scales (people, communication, and action–leadership orientation) were entered as independent variables in the equation with the task scale as dependent. Table 2 summarizes the results of the regression analysis, which shows that two of the remaining three scales (the action–leadership orientation and the communication skills) predict the task orientation very well with a multiple R = .932 (R2 = .869, F(2,104) = 345.877, p < .000). Dropping the task-orientation items from the subsequent analyses should not be considered inappropriate because the remaining three competencies clusters explained almost 87% of its variance. The first exploratory factor analysis, using principal component analysis and oblimin rotation, including all but the task-orientation items resulted again in a 4-factor solution, with the action–leadership items divided in two groups. But when a 3-factor solution was requested, a very clear and easily identified solution came out, explaining 72.237% of the total variance. Each competency scale emerged very clearly corresponding to each of the three factors. The first factor, where the action–leadership items were loaded, explained the majority of the variance (56.71%). To make the measure more efficient, a few more items, particularly those with smaller loadings on the perspective factors, were omitted. Six items were chosen for the action–leadership and the people-orientation clusters respectively, and five for the communication skills cluster. Table 3 shows the resulting factor pattern matrix, explaining a total variance of 79.5%.

TABLE 2 Hierarchical Regression Analysis of the Competencies Scales (Study 1) Steps

Variables

Multiple R

Adjusted R2

Action–leadership

.92

.84

573.37

.00

Action–leadership communication

.93

.87

345.87

.00

SE of B

Sig of T

Action–leadership Constant

.93 .71

.04 .18

.92

23.94 4.05

.00 .00

Action–leadership Communication Constant

.73 .29 .14

.06 .06 .20

.72 .25

12.65 4.37 0.70

.00 .00 .48

1 2

Note.

Dependent variable: the “dropped” task orientation scale.

TABLE 3 Exploratory Factor Analysis of the Competencies Scale (Study 1) Factor Variables Action–leadership

People

Communication

Items

Looks for new activities within his or her work environment Takes the role of the leader in group activities Looks for stimulation at work Motivates his or her colleagues at work Is willing to commit himself or herself to new tasks Behaves dynamically at work Displays kindness toward his or her colleagues Shows consideration for his or her colleagues Has good relationships with most of his or her colleagues Shows friendly behavior within the organization Respects his or her colleagues Shows positive feelings toward his or her colleagues Uses correct grammar in writing Uses correct spelling in writing Uses an appropriate style in writing Uses suitable language both in writing and speaking Speaks clearly

.92

–.02

.00

.89 .84 .81 .77 .59 .02 .08 .03 .04 .16 .04 –.00 .08 .30 .41 –.15

–.04 .00 –.05 –.17 –.16 –.94 –.92 –.90 –.86 –.85 –.77 .02 .02 .16 .11 –.37

–.12 .12 .08 .02 .21 –.05 –.05 .03 .10 –.07 .07 .95 .88 .78 .67 .67

Note. Numbers in bold represent XXXXXXXXXXX. Extraction principalcomponentanalysis; rotation method: oblimin with Kaiser normalization.

method:

315

316

NIKOLAOU

Following that, the relation of these three scales with job performance was explored. Thus, a hierarchical regression analysis was conducted to examine how well the job performance index is predicted by the three competency clusters. Table 4 summarizes the results of this regression analysis, which shows that the best predictor of the job-performance index was the action–leadership competencies cluster and not the other two clusters. The action–leadership was included in the equation with a multiple R = .706 (R2 =. 499, F(1,105) = 104.629, p < .000). Subsequently, one can argue that the participants of the study do not consider good communication skills or friendly behavior toward others within the organization as being associated with higher job proficiency, at least not as much as the competencies included in the action–leadership orientation cluster. Study 2 Following the development of the measure, we decided to conduct a second study to examine the criterion-related validity of the new instrument. The sample consisted of 57 managers, the majority of whom were men (72%) between the ages of 35 and 45 (53%) and were university graduates (49%). The participants were employed in 22 Greek organizations that completed the competencies and the overall job performance measures for their subordinates (N = 218). The majority of the firms operated in the services sector (72%) whereas 18% were industries and 10% were commercial companies. Table 5 presents the descriptive characteristics and the intercorrelations of the study’s variables, which all show satisfactory internal consistency with action–leadership competencies showing the highest correlation with job performance ratings. An exploratory factor analysis of the competencies measure, using principal component analysis and oblimin rotation, yielded results similar to the first study establishing the construct validity of the measure (see Table 6). The three expected factors emerged very clearly explaining a total variance of 84.9%. A hierarchical regression analysis was also conducted with overall job performance as the deTABLE 4 Hierarchical Regression Analysis of the Three Competencies Scales (Study 1) Steps 1

Variables

Multiple R

Adjusted R2

Action–leadership

0.71

.50

.49

104.62

.00

SE of B

Sig of T

0.39 2.04

.04 .17

.70

10.23 12.06

.00 .00

Action–leadership Constant Note.

Dependent variable: overall job performance.

GENERIC WORK COMPETENCIES

317

TABLE 5 Descriptives and Intercorrelations of the Variables (Study 2) Scales Action–leadership orientation (1) People orientation (2) Communication skills (3) Job performance (4) Note.

218 218 218 218

4.74 5.33 5.13 4.00

1.58 1.72 1.62 0.77

.93 .97 .95 .93

.76 .66 .57

.68 .39

.39

All correlations are significant at the .01 level (two-tailed).

TABLE 6 Exploratory Factor Analysis of the Competencies Scale (Study 2) Factor Variables People

Communication

Action–leadership

Items

Displays kindness toward his or her colleagues Respects his or her colleagues Shows positive feelings toward his or her colleagues Shows friendly behavior within the organization Has god relationships with most of his or her colleagues Shows consideration for his or her colleagues Uses correct spelling in writing Uses correct grammar in writing Uses suitable language both in writing and speaking Uses an appropriate style in writing Speaks clearly Takes the role of the leader in group activities Motivates his or her colleagues at work Behaves dynamically at work Looks for new activities within his or her work environment Looks for stimulation at work Is willing to commit him or herself to new tasks

1.02 .97 .93 .91 .88

–.01 .08 .00 .01 .01

–.06 –.09 .02 .03 .05

.81 .05 –.03 .06 –.04 .10 –.03 .05 .02 .12

–.01 .96 .95 .92 .92 .68 –.02 –.08 .08 .03

.16 –.09 .03 –.02 .08 .14 .93 .90 .81 .79

–.04 .41

.25 .03

.74 .45

Note. Numbers in bold represent XXXXXXXXX. Extraction method: principal component analysis; Rotation method: oblimin with Kaiser normalization.

pendent variable and the three competency scales as the predictors, eliciting similar results with the results of the first study (see Table 7). The results of these studies provide evidence for the construct and criterion-related validity of a new measure of generic work competencies. The measure captures three types of competencies: action–leadership, people orientation, and communication skills. The existence of a valid and reliable measure of generic work

318

NIKOLAOU

TABLE 7 Hierarchical Regression Analysis of the Three Competencies Scales (Study 2) Steps 1

Variables

Multiple R

Adjusted R2

Action–leadership

0.57

.32

104.59

.00

SE of B

Sig of T

0.28 2.67

.02 .13

.57

10.22 19.44

.00 .00

Action–leadership Constant Note.

Dependent variable: overall job performance.

competencies in Greece is a very important first step for the development of competency management as part of Greek human resources management. Our current methodology approached competencies in a “psychological–psychometric” way in terms of conception and assessment to differentiate from “broad” approaches in competencies in the field of human resources management, which include in the definition of competencies concepts such as motives, traits, attitudes or values, content knowledge, or cognitive or behavioral skills (Hooghiemstra, 1992). Adapting this approach allowed us to build a simple-to-use but simultaneously effective instrument both for academics researching competence at work as well as human resources professionals in performance and competency management.

REFERENCES Antonacopoulou, E. P., & FitzGerald, L. (1996). Reframing competency in management development. Human Resource Management Journal, 6, 27–47. Armstrong, M., & Brown, D. (1998). Relating competencies to pay: The UK experience. Compensations and Benefits Review, 30, 28–39. Boam, R., & Sparrow, P. (1992). The rise and rationale of competency-based approaches. In R. Boam & P. Sparrow (Eds.), Designing and achieving competency (pp. 3–15). London: McGraw-Hill. Boyatzis, R. (1982). The competent manager. New York: Wiley. Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences. London: Lawrence Erlbaum Associates, Inc. Craig, S. (1992). Using competencies in career development. In R. Boam & P. Sparrow (Eds.), Designing and achieving competency. A competency-based approach to developing people and organizations (pp. 111–127). London: McGraw-Hill. Feltham, R. (1992). Using competencies in selection and recruitment. In R. Boam & P. Sparrow (Eds.), Designing and achieving competency. A competency-based approach to developing people and organizations (pp. 89–103). London: McGraw-Hill. Gorsuch, R. L. (1983). Factor analysis (2nd ed.) Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Green, A. (1995). Verbal protocol analysis. The Psychologist, 8, 126–129.

GENERIC WORK COMPETENCIES

319

Hooghiemstra, T. (1992) Integrated management of human resources. In A. Mitrani, M. Dalziel, & D. Fitt (Eds.), Competency based human resource management. London: Kogan Page. Kline, P. (1994). An easy guide to factor analysis. London: Routledge. Kurz, R., & Bartram, D. (2002). Competency and individual performance: Modelling the world of work. In I. T. Robertson, M. Callinan, & D. Bartram (Eds.), Organizational effectiveness. The role of psychology, pp. 227–255. Chichester, England: Wiley. Nikolaou, I (1999). The five-factor model of personality and work behaviour in Greece. Unpublished doctoral dissertation, University of Manchester Institute of Science and Technology, England. Norussis, M. J. (1994). SPSS professional statistics 6.1. Chicago: SPSS Inc. Robertson, I. T., Baron, H., Gibbons, P., MacIver, R., & Nyfield, G. (2000). Conscientiousness and managerial performance. Journal of Occupational and Organisational Psychology, 73, 171–180. Robertson, I. T., Gibbons, P., Baron, H., MacIver, R., & Nyfield, G. (1999). Understanding management performance. British Journal of Management, 10, 5–12. Sparrow, P. R. (1997). Organizational competencies: Creating a strategic behavioural framework for selection and assessment. In N. Anderson & P. Herriot (Eds.), International handbook of selection and assessment (pp. 343–368). Chichester, England: Wiley. Sparrow, P. R., & Bognanno, M. (1993). Competency requirement forecasting: Issues for international selection and assessment. International Journal of Selection and Assessment, 1, 50–58. Tett, R. P., Guternamn, H. A., Bleier, A., & Murphy, P. J. (2000). Development and content validation of a “hyperdimensional” taxonomy of managerial competence. Human Performance, 13, 205–251. Torrington, D., & Blandamer, W. (1992). Competence, pay and performance management. In R. Boam & P. Sparrow (Eds.), Designing and achieving competency. A competency-based approach to developing people and organizations (pp. 137–145). London: McGraw-Hill. Weightman, J. (1994). Competencies in action. London: IPD.