Author: Gabriela Acosta Camacho
Published: 4 September 2024
Lectorate Social Justice and Diversity in the Arts
Internal Transformations
A Qualitative Evaluation Study of the Experiences of Participants in the 'Inclusive Working and Learning Environment' Module at the Amsterdam University of the Arts
Introduction
The past two years, the research group received many requests from colleagues and students at the Amsterdam University of the Arts (AHK) to ‘help out’ with creating environments where difficult topics about social safety and inclusion can be discussed and exist in a non-polarizing way Much of this was driven by the Inclusivity Pathway Training, which gained momentum within the AHK following a research project by the Dance Teacher department (Academy of Theatre and Dance) in collaboration with the Social Justice and Diversity in the Arts research group Word spread that this initiative was ‘really working and making a difference’, and so, the journey began
We wanted to translate our work from the research group into a more sustainable educational program, where we could combine the Inclusivity Pathway Training with the Five Pillars and other tools we had developed. Then, an opportunity presented itself: the research group was asked to develop a study program for staff members about social safety and inclusion by the coordinator ‘teacher development’ 1 program within the AHK.
I accepted this challenge and combined researching, creating and teaching the pilot that took place from February to April 2024. It was open to all who were interested and mandatory for those obtaining the Basic Didactic Competence qualification (BDB) within the AHK. The aim was to create a program that invites teachers and staff members to work and (un)learn about creating and maintaining inclusive work and learning environments.
The module’s approach and assignments are grounded in real-life issues from practice. Many teachers and staff, both within and beyond the AHK, struggle with addressing sensitive topics related to their work practice. How do you work towards an inclusive working and learning environment with students as well as with co-workers?
To evaluate the pilot, a qualitative evaluation research was conducted The purpose of the study was to understand the learning experiences of the participants in the Inclusive Work and Learning Environment module (do they have the courage to start acting differently?), the value of the module, and possible recommendations for improvement
1 https://www.ahk.nl/en/opleidingen/nascholing/teacher-development/
Content of the Module Inclusive Work and Learning Environments
How do we create an inclusive learning and working environment within the AHK? The module ‘Inclusive Work and Learning Environment’ is for all teachers and staff members at the AHK who want to work on, and learn about, creating a more inclusive working and learning environment. Goals for the participants were:
● Gaining new insights and/or sharpening existing ones regarding the creation of an inclusive working and learning environment, which can more strongly support your own vision;
● Analyzing one’s own working practice and connecting it to the methodologies and theoretical frameworks provided;
● Gaining practical tools that can be immediately applied in one’s own professional practice;
● (Re)considering one’s own work practice and that of colleagues from various perspectives and learn from them;
● Engaging in a valuable network of colleagues within the AHK who also deal with questions and issues of social safety and inclusiveness.
The module is built from different theories and tools from the research group, including the Inclusivity Pathway Training (IPT) 2, Story Circle Method 3, and the Five Pillars 4. The implementation of the pilot took place from February to April 2024. The module required 30 hours of study time and consisted of four physical meetings of three hours each, self-study, and assignments.
The first gathering took place on the 7th of February 2024. One of the participants couldn’t join, so we started with four out of five participants. This first gathering was an introduction to the program, but also about introducing ourselves. We had some coffee and tea and we started with the question “What brings you here today?”. Before starting this module, participants were asked to write a small motivation, by writing something in their own time, which can be done with great precision, many words and time to think. It was important to also ask in real-life about their motivation in an open-ended question to get to the heart of motivation in a different way: voice and body language were very important in this.
After exchanging with each other, we continued with several IPT exercises around "getting to know each other”. Exercises included: 'head and body' (walking in space and responding to the facilitators request to perform something physical, such as clapping and jumping. Then reverse the physical action, so the word clapping requires jumping, and vice versa). We did a round of names with two movements that tell something about you that you love and the pet peeves-exercise.
After these exercises we used the I am from - poem. Every participant recited their poem. We sat in a circle. Listeners closed their eyes or looked at the floor. They were assigned to listen and feel where in the body something resonated during the recitation of the poem. We got to the core of this module: being able to connect.
2 www.inclusivitypathwaytraining.com
3 https://www.ahk.nl/en/sjda/projects/story-circle/
4 https://www.ahk.nl/en/sjda/method/
Around this exercise, we also had a conversation about the (learning) environment: working with a laptop, or often "behind" a laptop, actually creates instant distance. So what could you do in your work practice to connect? How do you sit or stand with each other? And where in space?
To process the first part of the meeting and get acquainted with the theory of the Five Pillars, participants went out in pairs with some "Five Pillar Question" cards. We closed with a question-answer session and a look ahead to the next session.
The second gathering took place a few weeks later on the 28th of February. In the meantime participants worked on their first assignments: reading handed material and taking notes (form can be written, audio or audiovisual) by using the Pillars as a starting point to reflect on the meetings and study assignments of the module. Questions that could help participants could be: What did you learn? What was new? What was confirmed? What cost you effort? Were you able to stay hooked? What would you like to know or learn more about? What can you take into your practice?
We were with three participants. Two hours were used for an IPT session. The remaining time was reserved for questions, reflections and to start thinking about creating an own IPT inspired exercise.
For the IPT 5, we started with the guidelines and looked back at exercises we did at the first gathering. We did the following IPT-exercises: take a place in space, ET-movement, back-to-back, snapshots, circle of judgment/non-judgment, dominant & other. We also looked at the ABC theory together and used a clapping game emphasizing breathing in between exercises.
Between gathering 2 and 3 participants had a few weeks to create an IPT-inspired exercise in their own study-time. Participants felt a bit unsure about creating their own IPTinspired exercise and testing it in their work environment. They were handed different tools (the IPT book, cards, website with material and an online video about Holding Space). As the trainer, I told them they were free to contact me anytime to help them get started. I also reflected on their motivation and gave them specific advice to, for example, create an exercise that can be fully executed in a non-verbal way, or in a certain timeframe. This helped to ‘just do it’.
One participant contacted me in between to exchange thoughts and ideas.
On April 3rd the 3rd gathering took place. In the meantime it became clear that two participants would not be able to join anymore (see subchapter ‘participants’). The three remaining participants were asked to bring their created IPT exercise with them, so we could experience (part of it) together. Even though we were with a small group, there was a big variety in exercises. All of the exercises had a non-verbal and physical part interwoven and were tested in the environments of the participants.
We created time for questions and exchanging ideas on (for example) how to add the body more into their exercise or how to differentiate for different groups and environments. There was an exercise that went difficult when put in practice. We discussed this situation with each other, but also felt the resistance and discomfort by doing this exercise ourselves. In this moment it was important as a trainer to make clear the importance of being allowed to try out. Not everything has to work, being able to work inclusively is about how you can deal
5 Aminata Cairo, Winnie Roseval, Bilal Ona, Somya Bouzaggou, Inclusivity Pathway Training; Bright Spots on the Inclusivity Pathway, Aminata Cairo Consultancy, 2023.
with discomfort. We talked about the importance of aftercare: make sure you have time and space when doing these exercises.
The participants gave back that, even though the program was really packed, it was really inspiring to do this session together. The participants also shared the exercises among themselves.
The 4th and last gathering took place on the 17th of April. All participants were asked to give something back to the group (a story, a word, a movement). We used the Story Circle Method for sharing our learning outcomes and surprises. The IPT Evaluation Tool was also used for reflections for the learning journeys. Three questions were asked: From which Pillar have you learned the most? Which Pillar would you like to learn more from? Which Pillar do you feel most connected to?
We ended the session with one of the gifts of a participant and expressed the wish to meet again in a while and catch up on how everyone is doing.
Methodology
Research questions and research set up
The purpose of this study is to understand the learning experiences of participants in the Inclusive Work and Learning Environment module, the value of the module and what recommendations and improvements there are for the module. The following questions were addressed:
● What are the learning experiences gained while following the ‘Inclusive Work and Learning Environment’ module according to the participants?
● To what extent do participants feel (more) competent and dare to (start to) act differently?
● What recommendations do participants have for a continuation of this module?
To answer the research question, several interconnected methodological approaches were applied. Lub and Rensen 6 (2022) describe qualitative evaluation research as research that focuses specifically on estimates of the efficacy of a particular methodology, method or (behavioral) intervention. This method is consistent with the purpose of this study, which focuses on the experiences of participants in a new module on inclusiveness in the work and learning environment. These experiences are not measured at the interval or rational level and therefore, in part, fall under the category of qualitative research (Swanborn, 2004) 7. This approach formed the base of the study and was combined with the Five Pillars evaluative framework according to the research group Social Justice and Diversity in the Arts (2023) 8 and Ways of Knowing in Indigenous Knowledge Research according to Keane, Khupe and Muza 9 (2016).
Figure 1 combines the adapted version of Keane, Khupe and Muza (2016) where they build on DeMarrais' (1998) 10 framework and the approaches that Lub and Rensen (2022) write about when it comes to evaluative qualitative research. The bold printed words are methodological approaches that can potentially be used for this type of research. In addition, the Five Pillars of the research group form a methodological approach and at the same time a theoretical concept to make connections between the data found.
6 Vasco Lub, Peter Rensen [2020], Checklists kwalitatief evaluatieonderzoek en mixed methods onderzoek, Met toelichting, Utrecht: Movisie, 2022.
7 Peter Swanborn, Evalueren. Het ontwerpen, begeleiden en evalueren van interventies: een methodische basis voor evaluatie-onderzoek, Amsterdam: Boom Onderwijs, 2004.
8 https://www.ahk.nl/en/sjda/method/
9 Moyra Keane, Constance Khupe, Blessing Muza, ‘It matters who you are; Indigenous knowledge research and researchers’, in Education as Change, UNISA, University of Johannesburg and UNISA Press, 2016, 20(2), pp. 163-183.
10 Kathleen B. deMarrais, Inside stories; Qualitative research reflections, New Jersey: Lawrence Erlbaum Associates, Inc, 1998.
Figure 1: Mix of methodological approaches
Ways of knowing in Indigenous Knowledge research’ (Keane, Khupe en Muza, 2016).
Archival knowing Narrative knowing Observational knowing Relational knowing
People’s artefacts People’s stories Own stories
People’s behaviours
People’s shared learning & understanding
Qualitative research Lub & Rensen (2022)
Methodological approach
Historical research
Data Source Letters, newspapers, photographs, tools, objects
Narratology Autobiography Oral history
In-depth interviews, stories, songs
Ethnography Participation Action research Listening
Transformation research Intuition/ listening
Co-creating Participation of researcher and community Sharing - ‘studying up’
Participant observation, field notes, participant assignments
Dialogue, community discussions, ceremonies, meeting notes
Document analysis
Co-creating
Survey
written notes, reflections, assignments, evaluations
Research Group Social Justice and Diversity in the Arts (2023)
Five Pillars
Relationality Temporality
Transformation
Cost
Joy
IPT (Cairo, 2022) Story Circle (O’Neal, z.d)
IPT Evaluation Tool (Acosta Camacho, 2023) Co-creating
Dialogue, community discussions, ceremonies, meeting notes, tools, stories, participant assignments written notes, reflections, assignments, evaluations
Participants
Initially, a total of five participants started the pilot program. All participants were teachers working in different programs at the AHK with different (art) backgrounds. Some teach more practical subjects and others more theoretical subjects. Some participants already knew each other. The participants were informed about the research and signed an active consent form. Participants were instructed that they could withdraw their active consent at any time.
One participant could not make the first gathering and had to withdraw from the program because of personal circumstances. Another participant joined the first gathering, but had to withdraw due to a turbulent workplace situation with associated workload. Only three participants joined the full program and also completed it. Because of the number of participants, a few parts were altered within the module: the gatherings were reduced to four meetings instead of the original five. Also, the peer exchange (visiting each other’s workplace) did not take place in this pilot phase.
In addition to the participants, the trainer of the module also actively participated as a
participant of the module and conducted the research at the same time. Following the checklist by Lub & Rense, this would raise questions around bias and the position of the researcher, but following Keane, Khupe and Muza (2016) it’s illustrated that “the researcher’s life experience shapes the research purpose, design and credibility.” (p.163) Benefits of working with a small group of participants is to be able to engage and exchange with each other and provide space for different voices to be heard. This might even stimulate a closer understanding of the learning process that takes place.
Findings and results are very specific and cannot be generalized, but can be implemented for the follow-up of this module and as a starting point for follow-up research.
Research methods
The data for this study were collected using notes and (written) assignments following the module, exercises (related to the IPT and ceremonies), stories and listening (Story Circle Method), tools, group discussions, individual coaching interviews and a survey after the module. The participants were asked to complete seven assignments. They submitted this to the online platform ‘Leerpodium’. The data is analyzed thematically and roughly follows the steps as described by Verhoeven 11 (2023): (1) explore, (2) code, (3) thematize, (4) revise and refine, (5) establish and structure, and (6) present. The four gatherings took place (live) at the AHK LearningLab in Amsterdam at the following times:
11 Nel Verhoeven, Thematische analyse: patronen vinden bij kwalitatief onderzoek, Meppel: Boom uitgevers, 2023.
Date Time Subject
14th of February 2024 09.30 - 12.30 Gathering 1: Introduction, acquaintance and expectations stories participant assignments written notes dialogue community discussions tools
28th of February 2024 09.30 - 12.30 Gathering 2: IPT participant assignments written notes dialogue community discussions tools
March 2024 ntb Peer exchange participant assignments written notes observations
First week of April 2024 ntb online Individual interviews with trainer written notes dialogue community discussions
3rd of April 2024 09.30 - 12.30 Gathering 3: Exchange IPT created exercises participant assignments written notes dialogue community discussions tools
17th of April 2024 09.30 - 12.30 Gathering 4: Reflection and goodbyes ceremonies notes for meetings resources participant assignments dialogue community discussions, written notes reflections
April 2024 After the program survey
Data analysis
For this study, information from (written) assignments, a survey, oral interviews and practical experience was analyzed. Participants submitted interim assignments that were reflective in nature. Some of the assignments were submitted in writing, but other forms such as (audio)visual were also allowed. Conversations that took place, the program with exercises carried out and reflections on them were documented in writing after each meeting. Because of the limited number of participants and discussion of personal and sensitive topics, a choice was made not to record the meetings so that participants felt truly free to express themselves.
For the document analysis of assignments made, an exploration of retrieved (written) data took place. The written assignments submitted by respondents are analyzed and coded (open and axially) in order to then make themes in the data found. The assignments are first explored and open coded per assignment, in order to then make connections between all assignments collectively. Assignments were coded and categorized into aggregate themes focused on learning outcomes and actions.
Participants conducted individual coaching interviews and reported on them as part of the assignments. These conversations were conducted both live and online during April 2024. The conversations lasted 60 minutes. Due to privacy reasons, these conversations were not recorded. The participants provided their own reflections of the conversations
(assignment 5) and the teacher-researcher took notes of the conversations. After submitting the participant's reflection, the teacher-researcher (using own recorded notes) sent an individual response to the participants.
The participants created their own IPT-inspired exercises. The exercises were tested in practice and the exercises themselves were recorded in writing. The exercises were again (partly) tested and implemented with the participants in a live meeting. There was a verbal exchange about the experiences of the participants themselves and the experiences gained in work practice. There was room for questions, tips and advice to sharpen and further develop the exercises (a small form of co-creation).
After the module, a survey was distributed to the participants for evaluation. This survey including questions was distributed from the coordinator "teacher professionalization" at the AHK. The survey was completed by 2 participants. Questions asked could be answered with an open answer, except for question 4. The questions were as follows: What were your expectations of this module and did the module meet these expectations? How did you experience the structure of this module? What did this module bring you? What would you have liked to see different about this module? What grade would you give this module? Do you have any compliments or tips for the trainer of this module? Do you have any compliments or tips for the organization of this module? Do you have anything else you would like to say?
From the data analysis, the findings were presented using the Five Pillars as a framework. Quotes from participants were acquired in this, to hear the learning experiences from different voices (Tufford & Newman, 2010) 12. These findings were shared with the participants to make sure they could recognize themselves and whether the researcher correctly interpreted the meaning of their words in the right context. None of the participants requested a change in quotations. Additionally, an advice was given to take “space for processing time” into consideration for the recommendations.
Lessons Learned | Findings through the Five Pillar Reflection
In the next paragraph I will share the analysis by using the Five Pillars as a framework. I will use quotes from the participants, coming from their written reflections and assignments. Participants will be numbered for anonymity as participant 1, participants 2 and participant 3.
Relationality
One of the questions formulated by the research group to work with the Pillars is What relationships do you tend (not) to engage in? Throughout the module emerged the question of the relationship between the human and personal versus the professional attitude in work practice. The final reflections clearly showed that this balance can be found and that both can be done. Working in a group and peer-to-peer learning contributed to this, as participant 1 mentions: "I learned [from other participants] that you can both get personal in a lesson without compromising professional distance, integrity." With the relationality Pillar, it’s important to be able to see context (time, place, history, power relations, emotional and physical relationships, etc.). But this context is not easily compartmentalized. Much of our conditioned thinking does rely on those compartments and our habit of binary thinking.
12 Lea Tufford, Peter Newman, ‘Bracketing in qualitative research’, in Qualitative Social Work, 2010, 11(1), 8096.
Working with the Pillars and exercises in this module stimulated another way of thinking. For example, about setting boundaries as participant 2 wrote:
"A boundary is not a fixed thing that the other person can or cannot cross and if that happens then there are those and those consequences. Or yes, on the one hand it is. And on the other hand, a boundary is always a conversation with another person. You see, there are also costs to setting boundaries and also to the way you set boundaries. And that too is a topic of conversation when we talk about costs and boundaries. In that, I have learned to look at that more fluidly, without losing my belief in the need for clear guidelines."
In particular, the IPT exercise 'dominant and the other' generated discussion and also resistance. One insight gained was that we do a lot of binary thinking and categorization, but that this gets in the way of inclusive working. Participant 1: "You can't work inclusively if you leave the human touch out of work, something that does happen a lot in the work culture." The exercise showed again that we cannot compartmentalize the multi-layered context of human beings and history, even though we often try to. It brings into perspective when sometimes you belong to a dominant norm and sometimes you don't. Fluidity (changeability) was a concept that recurred.
Temporality
A noticeable observation was that during many exercises, participants' past and childhood memories came up for discussion at various times. Sometimes there was a clear prompt or direction for this, such as the use of the "I am from..." - poem. But at other times as well, such as explaining which Pillars are the most distant from you or what still feels the least easy, past life experiences of the participants were linked.
Recognizing old (childhood and family) patterns in teaching practice and being able to work through resistance in a "pleasant" way provided a shift to being able to work more intuitively: "I'm glad to have found a little more access to a more emotional and intuitive world again without being weighed down by it." Also named was (the joy of) being able to be surprised by one's own assumptions, and what was emphasized several times throughout the module: the need to continue to see humanity, no matter how difficult a situation can sometimes be.
Transformation
During the program, all participants experienced a shift (transformation), with the insight that the human connection actually has to be central to make learning possible at all.
The IPT utilizes exercises that call for embracing and feeling with the body. The participants had varied backgrounds, with some being more experienced with physical work than others. This was not always as easy, for example as we experienced with the I am from-poem or by creating their own exercise in which I really asked them to implement the body and non-verbal communication. But all participants really took on this assignment and were able to apply it to their own practice. This provided a variety of work formats, deepening of learning and a better relationship with students. In a final reflection participant 3 wrote: "In the various IPT exercises that we entered into together and that I developed myself, I was lovingly forced to take off my cognitive coat, quiet my head and thereby 'detoxify' from that which feels so familiar to me but by which I also leave behind another part of myself, namely my body."
The other exercises, such as the Story Circle (which one participant also immediately implemented in their own practice) and the evaluation tool also contributed to the transition from head to body. Participant 3 during this exercise: “It touches me emotionally to see you standing there, and that I am standing here by myself.” The evaluation tool is inspired by the game Twister and design thinking. You create a playing field in the form of a circle and you literally take your body into the field. The evaluation tool was also used by one of the participants as inspiration to further develop their own tool.
So, transformation took place on different levels, but most important was the discovery that working from inclusion and a social justice perspective, asks for an internal transformation by continuously placing yourself in context and relationality within and outside of your direct environment. 
Cost
One participant felt attracted to the Pillar of cost related to power dynamics: how do you avoid constantly overstepping your own boundaries when it comes to inclusive work, keeping yourself from burning out? How do you make sure, from the dominant perspective, that by setting boundaries you do not exclude? And from the other perspective stand up for yourself enough?
The question underneath arose during group discussions and shared stories: To what extent (cost) can you stay true to yourself when work conditions are so contrary to what you want to do and what you stand for? Instead of using rigid rules and boundaries, we used the IPT's guidelines as an inspiration to create an environment in which all participants (students, co-workers) can join and in which you can allow yourself to continue an inclusive way of working and keeping yourself from burning out at the same time.
Joy
It became clear by the start of the program that there was a desire to also work from (the Pillar) joy, which was described in the reflective assignments as positivity, fun, air and breath. For example, participant 1 wrote afterwards:
[I tried to] ... engage in the module as judgment-free as possible. What was a surprising payoff was that it brought me joy. The Pillar I have the most trouble with because I don't really know how to fit that in. I always feared it would come at the expense of content. But during the module I found that applying joy in educational methods can actually add value to the content or have it land in a totally different way.
The Pillar of joy was one of the Pillars longed for, but also one of the most difficult Pillars to embrace organically in the work environment of the participants. It really took focus to implement this Pillar, but when focused, it did come back in for example exercises created. During the evaluation with the IPT Evaluation Tool, it was expressed by some participants that they want to specifically include the Pillar of joy more into their work practices.
Lessons Learned | For Continuous Work
From the perspective of the teacher-researcher, lessons learned to continue the work is to dive more into the internal transformations needed and how to get to this transformation. From the data-analysis a number of recommendations were also revealed from the participants.
Time for space and process
Because the meetings were now very full, the feedback was that a few more sessions would be nice, so that there would be more time for the process during the meetings. The period could also be spread out longer (e.g. over six months), in order to let the material sink in and have more room to try out. The time indicated for the module may not be realistic, depending on the participant. This is not so much about the exercises or assignments; these could fit into the time set for it. It’s more about the learning process that starts with working in this module. Participant 3 mentioned: “... the module has set things in motion for me. Both in my role as a teacher, and the lesson content and learning activities I engage in.” The module turns on so many mechanisms that it is difficult to have a ‘time out’ from this learning process, and so (unconsciously) more time goes into it. Even though time for reflection was implemented in the module, participants shared that they really need process time after every session. This is an important observation to take into account when setting up a continuation of this module.
The place to gather the assignment is the platform Leerpodium. Participants gave back that this platform brings limitations in designing the reflection and assignments because of the set format. If there were more possibilities to change the format, it would be more of an invitation to involve other (art)disciplines as part of the reflection/assignment.
Framework and tools for (co-)creating
The jump to creating and implementing a self IPT-inspired exercise could have been built up more step-by-step. In the original plan this was also the intention, due to the small number of participants one session was taken out of the program, which included this intermediate step. A participant also suggested that a tighter framework around the assignments would be nice, for more grip, because now 'basically everything was good'. From this we can see how conditioned we are as humans and not used to the fact that within educational programs something can just be good, an important lesson to take away. A desire was also expressed to be given more tools to apply in practice. With more participants in the group, this is already partly overcome because then more exercises can be created, contributed and shared as part of the module and peer-to-peer learning.
In Conclusion
In summary, there are several learning outcomes. In particular, carrying out different forms of work, getting out of the 'cognitive' and establishing a stronger relationship with one's work practice through physical assignments is something the module has provided.
Participants have both sharpened and (re)discovered their didactic and pedagogical skills in order to transform them and use them to create a more inclusive working and learning environment for the participants themselves, their students and colleagues. This is related to gaining new knowledge, skills, insights and tools. They’ve for example (1) showed this in their classes and work environment by creating and executing their own IPT inspired exercise, (2) used the Story Circle in class and (3) one participant also used the IPT Evaluation Tool to create a new grading system to implement in its own classes.
Another important learning benefit is the importance of peer-to-peer learning. Despite or thanks to the small group of participants, much was shared about (tedious) practical experiences in the workplace. Learning, testing and creating IPT exercises together created connectedness and, at the same time, a wealth of inspiration and a sense of collectivity: working together on something to bring about 'change' (as named in participants' motivations) - no matter how small-scale or 'slow'.
Carefully, there can be concluded that a learning benefit is that the module has contributed to a different attitude, such as daring to be flexible, following intuition, hearing the learner's voice and involving the learner in the lesson material and program. The participants gained more (self)confidence to act when it comes to creating inclusive work environments by going through internal transformations.