Project Management Training

A formal course can reap benefits.

By Bryan Leach

Success in consulting engineering depends in par t on the effective management of projects for (or on behalf of) clients. Most young engineers entering the field, however, receive little to no project management training during their academic schooling.

Such training is available through the Project Management Institute (PMI) and its book, A Guide to the Project Management Body of Knowledge. This process can lead to Project Management Professional (PMP) certification.

In consulting engineering workplaces, on the other hand, formal p roject management training is often absent. Instead, the relevant skills are acquired ‘on the job,’ under the guidance of an experienced project manager.

Such guidance, however, is not always sufficient. In response to an undesirable level of project writeoffs, claims and client pressure, an international consulting engineering firm introduced a more formal, thr ee-day project management training course.

The intent was to provide the knowledge and tools to help the firm’s staff become better project managers. In parallel with the course, a program of project management audits was also introduced.

These audits were undertaken in the firm’s offices every three months. They involved checking a random selection of projects against a 12-point list of essential project management documentation— e.g. signed agreement with conditions, authorization to proceed, approved change orders and review of final work product. This list was introduced as part of the course as a tool to guide the firm’s project managers. While the firm’s stated goal for the course and the list was to improve project delivery, it never explicitly defined the desired outcomes. The implicitly desired and discussed outcomes were to reduce the project write-offs and claims and to increase profitability and client satisfaction. The success of the course was internally evaluated, using the participants’ course satisfaction ‘scores.’

After the course had been in place for about four years, five courses had been presented in one of the company’s major offices. Independent of the company, this author u ndertook an evaluation of the benefits of the course to that office. This evaluation focused on the

54% extent to which the course had resulted in:

• Participant satisfaction.

• Implementation of the requirements of the list.

• A decr ease in write-offs and claims.

• A positive change in profitability and client satisfaction.

The process was undertaken using the Kirkpatrick Model’s four levels of evaluation for the course:

• Reaction (Level 1) – Did they like it?

• Learning (Level 2) – D id they learn it?

• Behaviour (Level 3) – Did they use it?

• Results (Level 4) – Did it make a difference?

Did they like it?

The firm’s target was an average participants’ satisfaction score above 7.0 out of 9.0 (i.e. 78%). Following the evaluation, the firm considered the course to have been a great success, based on the con- sistently high scores achieved and due to very positive feedback from the course participants.

Reactions were evaluated using the participants’ course satisfaction feedback sheets, which were made available in the subject office for all of the presented courses. Administered at the end of each course, they represented the limit of evaluation undertaken by the firm itself.

The sheets solicited feedback from the participants on the course content, presentation, format and venue and asked whether the course fell below, met or exceeded their expectations.

As mentioned, there was a high degree of participant satisfaction. The average scores for course content, presentation and format were 87%, 91% and 85%, respectively and the participants’ expectations were consistently met (54%) or exceeded (46%).

Did they learn it?

To evaluate the increase of the participants’ knowledge, a self-assessment questionnaire was designed.

It asked them to measure the extent to which their knowledge had increased in 26 specific areas, related to nine aspects of project management, using the following system:

• 1 indicates no (0%) increase.

• 3 indicates a moderate (50%) increase.

• 5 indicates a significant (100%) increase.

The knowledge questionnaire was sent to 71 individuals who had participated in the course in the subject office. Some 31 of these were completed and returned, for a response rate of 44%.

Five responses were from experienced and 26 from inexperienced project managers. In an attempt to ensure honest and unbiased reporting, the questionnaires were completed anonymously and returned to an impartial co-ordinator.

The inexperienced project managers reported an overall increase in knowledge of 60%. The two greatest areas of knowledge increase (78%) were understanding the importance of clearly defining the project scope and the value of regular communications with the client.

The experienced project managers reported an overall increase in knowledge of 28%. The greatest area of knowledge increase (56%) was in understanding the skills and attributes of a successful project manager. Other indicated areas of enhanced knowledge included satisfying the client, understanding the interface with the client, issues related to schedule achievability and the importance of implementing the list.

Did they use it?

A similar self-assessment questionnaire was designed to measure the degree of change in 15 specific project management behaviours of the participants, related to nine aspects of project management, again as per the following system:

• 1 indicates ‘ never,’ no (0%) change.

• 3 indicates ‘sometimes,’ moderate (50%) change.

• 5 indicates ‘always,’ dramatic (100%) change.

The behaviour questionnaire was sent with the knowledge questionnaire to the same 71 individuals and saw the same response rate. The inexperienced project managers reported a 53% positive change in behaviours. The greatest change, at 72%, was in practising their delegation skills; while a 70% increase was reported for satisfying the client and paying more attention to exercising leadership skills.

The experienced project managers, meanwhile, reported a 38% positive change in behaviours. The two greatest changes in behaviour, both at 60%, were in better understanding the client’s organizational structure and improving communications with the client. A 52% increase was noted for referring to and implementing the list.

Did it make a difference?

The most difficult part of the evaluation process was quantification of the results. There was a desire from management to prove the value— i.e. return on investment (ROI)—of the training in monetary terms.

The results were evaluated by considering the findings of the quarterly project management audits conducted in the subject office over a period coincident with the presentation of the courses. The office’s project write-offs and

12-month running average percentage of profitability over the same period were also reviewed.

Unfortunately, work in progress and receivables were not included in the evaluation. In retrospect, these two elements would perhaps have been the best direct measures of project management effectiveness.

Over the seven quarters coincident with the presentation of the courses, project management audit scores increased from 74% and 69% in the first two quarters, respectively, to 94%, 79% and 89% in the last three quarters, respectively. Project writeoffs were mixed at 0.88% and 1.19% (as a percentage of revenues) in the first two quarters, respectively, and at 0.76%, 2.18% and 1.76% for the last three quarters, respectively.

There was a 5% increase in the 12-month running average profitability of the office over the same period. There was also limited, anecdotal evidence of improved client satisfaction, based on client interviews.

Making a direct causal link between the course and changes in office profitability is difficult. Office profitability is primarily controlled by the cyclic nature of office revenues and the chargeability (i.e. billability) of staff. Another important consideration was the number of project managers who had participated in the course in the subject office.

Unfortunately, the benefits of behavioural changes due to the course are not immediate. Sufficient time must elapse between the course and the evaluation of its effectiveness, value and benefits, so as to allow the participants’ increase in knowledge and positive changes in behaviours, while working on new projects, to take effect and make a difference.

Indeed, it was difficult to provide proof of a positive ROI for the course. Rather, it was only possible to provide a weight of evidence that it had been a success.

Participants enjoyed the course and expressed satisfaction with it. Their knowledge increased and their behaviours underwent positive change.

It was particularly interesting to note the greatest increases in knowledge and behavioural changes, for both inexperienced and experienced project managers, were in ‘soft skills’ like leadership, delegation, communication, understanding clients and satisfying their needs. Experienced project managers significantly increased (52%) their knowledge of the importance of the list and changed their behaviour (52%) by increasing their refer- ence to and implementation of it when managing projects. These findings confirmed the benefit and value of introducing the list as an effective tool for project managers.

QuickVic™ System:

Unmatched ease of installation and enhanced visual verification. Three times more rigid joint with built in safeguards against over torquing and over tightening.

Potable Water:

Utilizes Grade P gasket material for improved resistance to typical potable water disinfectants. Unique housing colour to denote use in potable water.

Motion Control:

Market leader in system design and products to inherently deal with system movement. Carbon and stainless steel anchors, riser movement fittings, and anchor outlet fittings are engineered products.

Modular Construction:

High quality construction in a controlled environment for greater efficiency and productivity onsite.