ABLA 2016’s 7th Lesson Learned

Taken from http://www.relacionesculturales.edu.mx/abla-2016/

ABLA 2016’s 7 th Lesson Learned: “Measuring the Impact of Training”

By Prof. Jonathan Acuña-Solano, M. Ed. School of English Faculty of Social Sciences Universidad Latina de Costa Rica Sunday, September 11, 2016 Post 291

“One of the most important aspects of teacher training has to do with its impact in teaching skills and student learning” [ CITATION Góm16 \l 1033 ]. But how long does it take for a training team in a language school such as a Binational Center (BNC) to see the results of teacher coaching materialized in the staff’s teaching skills displayed in the classroom and to find evidence of how changes in instruction trigger an effect in student learning? The fact is that in terms of teacher training, instructor-coaching faculty members cannot expect to see or find any immediate result or evidence that the training has been moved into the class boundaries; outcomes can only be perceived later on, and they should be, as Gómez (2016) proposes, analyzed and measured in different time intervals to really see the effects of that teacher training.

Though I do not watch over instructor training at the BNC I work for, being in charge of the Curriculum Development Team makes my partners and me participate

in teacher coaching and also wonder how this can have a long-lasting effect in our teaching staff. During the ABLA 2016 convention in Houston, TX, German Gómez, from World Learning, talked about a training program they are currently carrying out in the Sultanate of Oman based on the Kirkpatrick Model for evaluating training impact, which was a complete novelty for me and a sounding measuring program after reading about it. Now, based on Gómez’s

talk at ABLA, “training aims at

developing 4 key aspects ” [ CITATION Góm161 \l 1033 ]. These crucial aspects are “knowledge” or “content required for the job,” “awareness” or “self-introspection of skill,” “skill” or “practical application of knowledge,” and “attitude,” or “the attitude towards a specific belief or practice.” These are the aspects that a training team needs to concentrate to consider how coaching sessions can have a long-lasting effect on teaching professionals working for our BNCs. These are the pillars supervisors and training decision makers should give some a thorough follow-up to measure the understanding, application, and/or impact of “knowledge, awareness, skill, and attitude” presented to educators to positively affect “teaching skills and student learning.”

During German Gómez’s ABLA presentation (2016), and bearing in mind the positive effect training can have in “teaching skills and student learning,” he asked the audience two questions regarding teacher coaching. “Why is measuring the impact of training important in your context / position?” [ CITATION Góm161 \l 1033 ]. First of all, as I commented this point with Gómez himself, in my curriculum development position, it is crucial that language trainees get the proper coaching to be fully functional in a new language program or with improvements made to an existing program, whether that is for adults, teenagers, or children. The idea behind training, in the way we perceive it at the BNC I work for, is to enhance the teaching skills and style any of our teachers have. The enhancement in skills and style can have a positive effect in the way language trainees are being exposed to the

contents they must learn from the various thematic units in the courses they take with us. Second of all, the constant visiting or re-visiting to topics related with our curricula allows the institution and the training team to somehow measure what the impact of training is in the short run. Measuring helps Academic Department training staff to actually see what is happening in our classrooms and how our classes are being delivered by our instructors to help us guarantee the quality of teaching our students are getting. Measuring is part of the quality control our BNC wants to exercise in its language programs to satisfy the demands of our current and future learners.

“What are some ways in which you as a trainer / coordinator / administrator measure the impact of training in your teaching staff?” [ CITATION Góm161 \l 1033 ]. When German Gómez asked us to share with our peers in the room, lots of things started to come to my memory: the times in which I was a supervisor and the way how supervision is now handled at our BNC in Costa Rica. At the beginning we coaches had to create our evaluation rubrics for class visits and observations.

Based on those rubrics supervisors had to write a report of their trainees that had to be submitted to our site coordinator for them to see who was doing what was expected of them and what kind of corrective actions needed to be taken to help the teacher overcome any teaching issue s/he might be experiencing. But in hindsight, what was exactly expected? There were different types of trainings, but I cannot recall any real follow-up for those specific coaching sessions we had. Best practices were proposed, but in the end everyone decided to either use them or forget all about them. In the end, were we really measuring “the impact of training” in “teaching skills and student learning?” Most of our rubrics were connected to expected and ideal performances in the classroom, but our training team never asked ELT educators how they thought was the best way to measure the implementation of any of those so-called best practices in their courses.

Taken from G贸mez (2016), Measuring the Impact of Training (PPTx)

German G贸mez gave many of us ABLA 2016 participants a good slap of reality in terms of what is actually and theoretically done to work with teaching professionals and their training nowadays. Participants were introduced to the theoretical model World Learning is currently using in the Sultanate of Oman

to

professionals Kirkpatrick

coach over Model.

EL

teaching

there: To

The

have

a

sample what this model is about, G贸mez had us watch the a video to see -in 60 seconds- what the model can do for us and for our instructors in terms of the measurement of training and the time span needed to see the results of any training event. After watching the video about the model and comprehending that the model has four levels: reaction, learning, behavior, and results, the scope of action towards measurement changes drastically [ CITATION The15 \l 1033 ] [ CITATION G贸m161 \l 1033 ].

After listening to what German Gómez was explaining to us, my conception of language trainer coaching changed and widened my understanding of the four levels presented by the Kirkpatrick Model. Seeing each of the four levels, the following can be noted. Reaction refers to “the degree to which participants find the training favorable, engaging and relevant to their jobs” [ CITATION Thenc \l 1033 ]. Based on Gómez (2016), teachers should be measured to find out their immediate reaction right after the event by means of surveys, direct questionnaires, and so on. Learning encases “the degree to which participants acquire the intended knowledge, skills, attitude, confidence and commitment based on their participation in the training” [ CITATION Thenc \l 1033 ]. Gómez (2016) proposes that language instructors can also be “measured” in their learning after the training event by the same means. Behavior entails “the degree to which participants apply what they learned during training when they are back on the job” [ CITATION Thenc \l 1033 ].

Language educators, as stated by Gómez (2016) in his presentation, need to be measured three to five months after being trained by means of surveys, classroom observations, and interviews. Finally, results comprise “the degree to which targeted outcomes occur as a result of the training and the support and accountability package” [ CITATION Thenc \l 1033 ]. Gómez (2016) posits that these results can only be measured from six to twelve months after the training event by means of customer satisfaction results and manager results. All these facts are intended to make us BNC’s Academic Department coordinators understand that a training session held today will have different immediate effects or short or long-term sequels. The real impact of training can only be started to be measured after six months after the training event; before that moment, it is quite premature.

Some of the most salient points connected to my 7 th lesson learned at ABLA is that the Kirkpatrick Model is not a training program per se; training programs must be designed and developed by BNCs to guarantee quality in their programs. This

model can be used as a solid ground to base any coaching program for teaching professionals in our institutions. The program, as it was also shared by Gómez (2016) with his experience in the Sultanate of Oman, needs to be open and participative for the teachers; that is, language trainers also need to have access to the training agenda in our schools to provide the institution with feedback related to what pedagogical areas they need to be trained on and how they should be measured by the team of teacher-coaching experts. Perhaps, as explained by Gómez (2016), teachers can participate in the creation of the rubrics the BNC can use to evaluate their own performance and accountability for the quality the training is looking for in their “teaching skills and student learning.”

References Gómez, G. (2016, August 18). Measuring the Impact of Training (PPTx). PPTx for the ABLA 2016's Talk . Houston, Texas, United States: World Learning.

Gómez, G. (2016, August 16-19). Measuring the Impact of Training. 21st Century Challenges ABLA 2016 Convention Program . Monterrey, Nuevo León, Mexico: Centro Mexicano Norteamericano de Relaciones Culturales.

The Daily Project Manager. (2015, August 3). Kirkpatrick's 4 Levels of Evaluation.

Retrieved from YouTube.Com: https://youtu.be/aw9sqEvfuf8 The Kirkpatrick Partners. (n.d.). The Kirkpatrick Model. Retrieved from The Kirkpatrick Partners: http://www.kirkpatrickpartners.com/OurPhilosophy/TheKirkpatrickModel