Page |1
GETTING STARTED: OBSERVATIONS AND OBSERVATION REPORTS By Lonnie Rowell, Ph.D.
As Sagor (1992) presents it, collaborative action research unfolds through five sequential steps: 1. Problem formulation 2. Data collection 3. Data analysis 4. Reporting of results 5. Action planning In the USD model, we take what might seem like extra steps in problem formulation. This paper addresses the first steps for problem formulation as practiced in USD collaborative action research projects.
In the USD model, because the majority of graduate student team members have not had work experience within the culture of K-12 education, we start the action research process with observations at the site or sites that will be the focus of the project. The observation visits most often take place very soon after a preliminary meeting between the graduate student team and the practitioner partner or partners. The observations also provide an opportunity to practice a research skill – observation – that is part of the curriculum in COUN 508.
Introductory meeting with practitioner partners For the initial, introductory meeting with the practitioner do all you can to have the entire team present. It can be harmful to the collaboration to have a member missing from the introductory meeting. Absence from this first meeting sends the wrong signal to our practitioner partners and to teammates.
The initial meeting may be quite brief, from 20-40 minutes, or may last as long as an hour. The team leader should take the lead in setting up the meeting with the practitioner(s) and in organizing an agenda for the introductory meeting. The agenda is most often pretty simple: introductions are made; contact information is exchanged; dates for observation(s) are set; a post-observation meeting date is set. In our experience, the team leader should facilitate this first
© L. Rowell, 2006
Page |2 meeting, making every effort to include everyone in the conversation. This should include each person at the meeting having an opportunity to speak.
In working out a time to meet with the practitioner, be aware that this is not always an easy thing to do. Our practitioner partners are all quite busy. Most school counselors in San Diego County have anywhere from 400 to 1,000 pupils assigned to them. Do not mistake their frequently rushed demeanor as a sign that they do not care about the action research project. Our experience is that being patient yet persistent in establishing contact with the practitioner partner is the best way to proceed. They understand that graduate students are busy also, so efficient and friendly meetings are appreciated by everyone involved.
Observation & Observation Report The first major step after the graduate student team has organized itself involves connecting with the assigned site and practitioner partner and practicing observation skills. After the introductory meeting between the graduate student team and the practitioner partner or partners, student members of the action research team visit their action research project site and observe the practice of school counseling at the site. Careful observation helps students notice details about the culture of a school and the practice of school counseling and strengthens their awareness of issues that impact the practice of school counseling at the site. Each team submits an Observation Report based on the initial visit to the site. Completing an observation. Each team has a site or sites to visit. The objective of the visits is to become familiar with the culture of the school and the school counseling practices and issues at the site. The steps taken in completing an observation include: 1. Prepare for completing at least six hours of observation. This can be completed in one visit by the entire team or in several visits arranged to fit the schedules of team members and the practitioner partner. 2. Arrive for the visit equipped to take notes. These field notes are your written observations of what you see taking place at the school in relationship to the practice of school counseling. As Johnson (2005) puts it, “beginning action researchers are often unsure of what they should record. My advice is to stop thinking and just write what you see. Once you start recording, you begin to see
Š L. Rowell, 2006
Page |3 things that are interesting or important. In this way field notes help you notice details you might not otherwise have noticed� (p. 63). Be sure to save all your notes. a. So, what do you observe? Consider these points: i. Your interest should be focused in large part on observing the behaviors of staff and students as they occur naturally in the setting. This constitutes the human and social environment of the school. What do the counselors seem to be doing? What do you notice about patterns of interaction and communication among staff and between staff and students? ii. Also observe the physical setting of the school and the program setting for school setting. What is the physical environment of the school like? What is the physical environment within which school counseling services are provided? How would you describe these physical settings? How is space used? How are people organized in the space? iii. Do your best to observe program activities associated with school counseling. What does counseling have to do with what you observe taking place at this school? What specific activities are initiated by school counselors? Who is involved in the work done by counselors? iv. You also should be on the lookout for informal interactions and unplanned activities. This is simply part of gathering descriptive information regarding what people, in particular the school counselors, do at the site and how they do it. What do you notice about body language and nonverbal cues in interactions? v. Pay attention to the exact language of people at the site regarding what they do. When you look over your notes after an observation you should begin to notice patterns of word usage related to the counseling program at the school.
Š L. Rowell, 2006
Page |4 vi. Finally, observe what does not happen. As Patton (1990) puts it, “when your basic experience with a program suggests that the absence of some particular activity or factor is noteworthy, be sure to note what did not happen” (in Mertens, 1998, p. 320). b. For this initial visit you are a non-participant observer. As the action research project develops you will transition from this role to a peripheralmember-researcher who observes and interacts closely enough with the school counseling practitioners and others at the site to formulate an insider’s perspective (Adler & Adler, 1994). Over time, some of you will become participant-observers and will be participating in projects at school sites while also systematically gathering data regarding the action research project. 3. When you are conducting the observations, you do not have to meet with the counselor/practitioner partner. You can simply show up at the scheduled time, check in according to the school’s policies, and conduct your observation. Discuss the protocol for the observation with your practitioner partner during your introductory meeting. 4. Meet as a team to discuss and cross-check your observations. Here, you want to identify patterns in your observational data and distinguish between accurate and inaccurate interpretations of school counseling practice at the site. Take note of differences in the times of day and the days of the week that observations took place. 5. Complete the Observation Report.
On note taking. Action research project team members take notes during interviews and focus groups and as a part of observations conducted in classrooms, at meetings, and during informal interactions around the site of a project. This section presents a framework for taking notes and keeping notes organized. A section that follows addresses coding notes to identify themes and patterns in observational data.
© L. Rowell, 2006
Page |5 Action research requires careful and descriptive note taking. It is much better to have too many field notes than too few. Field notes are the raw data of descriptive social research, and although you can shorten descriptive writing through coding and summarizing, adding to the raw data after the fact is questionable research practice and should be avoided. If field notes are too abbreviated, furthermore, it becomes difficult to capture the meaning in the words.
In essence, the notes you take are word-pictures of the setting, the people who work and study and interact in the setting, and the actions and conversations of these people. This descriptive content represents our best efforts to objectively record details of what took place in the field. We want to capture slices of life at the sites where we conduct our projects in concrete terms rather than through abstractions. As Bogdan and Biklen (1982) describe it, "do not, for example, say that the teacher was in front of the room 'teaching.' What was he or she actually doing and saying?" (p. 85). Bogdan and Biklen describe six areas that are encompassed by the descriptive aspects of fieldnotes: 1. Portraits of the subjects. Physical appearance, dress, mannerisms, style of talking and acting. 2. Reconstruction of dialogue. Capture as much of it as you can. Indicate direct quotes. Write down words and phrases that may be unique to Monarch or have a special use in the school and its' programs. 3. Description of the physical setting. Can use drawings as well as verbal sketches. What images of the place stand out to you? 4. Accounts of particular events. Who was involved, what was the event, what actions took place? 5. Depiction of activities. Similar to above. Record behaviors of those involved. Who did what? What was the sequence of particular acts? 6. The observer's behavior. Treat yourself as an object of inquiry. As the instrument of data collection, you need to attend to your own behavior, assumptions, and subjective responses to what you see and hear and participate in. Try and account for how, if at all, your presence may have impacted activities and events at the school.
Š L. Rowell, 2006
Page |6 Managing condensed accounts No matter how fast we are able to write, we cannot capture all that is being said, all that is going on around us in the field. Like it or not, our notes are condensed accounts of a complex reality. Social scientists have developed a variety of techniques for managing this process of condensing reality. One of the most important techniques to be practiced in collaborative action research is keeping track of language. In the process of completing field notes we are writing down key words, phrases, and seemingly unconnected sentences. It may seem to blur together at times. Later, we look at our notes and wonder what we have. Keeping careful track of language can help us sort out the themes and patterns. Here are some tips based on Spradley's (1979) classic, The Ethnographic Interview: •
Identify language used for each entry in the field notes o What are the language differences in the setting?
Older students
Younger students
Parents
Teachers
Mentors
Administrators
Volunteers
Board members
Funding source representatives
Cultural/linguistic differences
o Identify speakers by name and position •
Try for a verbatim record of things said o Distinguish between native terms and observer terms
We also can establish simple field notes writing practices that will aid us in managing the written accounts. One easy practice to adopt is to write using paragraphs based on a topic. As Bogdan and Biklen put it, "every time a change occurs - in the topic of a conversation, when a new person enters the setting, or whatever - start a new paragraph. When in doubt, start a new paragraph" (p. 89). Also, you can take field notes that leave large margins on the left-hand side of the page. Later, this space can be used to "code" the paragraphs.
© L. Rowell, 2006
Page |7 Finally, there is no set length for the notes taken during observations, so do not be surprised if the team quickly accumulates reams of notes. However, because our action research projects develop relatively quickly, field notes for USD projects tend to be more focused than they might be if we were doing predominantly ethnographic or phenomenological studies. Again, in general more notes will serve your project far better than fewer notes. Some rambling in note taking is understandable and expected, particularly at the start of observations.
The reflective element in field notes Field notes attempt to capture both objective and subjective elements associated with an action research project. The objective elements involve accurate depictions based on, for example, the six areas presented above from Bogden and Biklen (1982). Here, we attempt to keep our emotions out of it. We strive for an accurate, honest, clear description of events, people, interactions, settings, etc. We use reflection in responding to both the objective and subjective elements in the field.
When recording our more subjective responses to the setting and the people in it we want to give vent, so to speak, to the full range of our emotional responses to the situation. Here, however, it is crucial that we know how to separate one from the other. If we pretend, let us say, that we are objectively noting something when we actually are writing from a deeply subjective place, we lose validity in our work. It is alright, in other words, to have a subjective response to a person – be it positive or negative – but it is very important that we know how to distinguish between a feeling and an objective situation. In our projects, we identify the subjective comments by prefacing them with "O.C." (for “observer's comment”). These O.C. notations take a variety of forms, from a description of a strong impression of some sort that you become aware of while observing at a site, to a reflection on an observation experience, to speculations about what is happening in relation to particular people or particular situations at the site. Categorizing reflective elements of field notes. According to Bogdan and Biklen (1982), five categories of reflection need to be considered: 1. Reflections on analysis. What are you learning? What themes seem to be emerging? How do the pieces of data seem to connect together?
© L. Rowell, 2006
Page |8 2. Reflections on method. Thinking about the methods being used, any problems with them and ideas for addressing the problems. 3. Reflections of ethical dilemmas and conflicts. Yes, this too. How might these dilemmas and conflicts relate to the development of a student support system for Monarch? 4. Reflections on the observer's frame of mind. Any preconceptions, biases, assumptions that begin to surface? Did you take some in with you? How are you managing the shifts in your experience of the setting and the people in it? 5. Points of clarification. As you go along you will have sudden insights into what is going on, how things work, and how a student support system might fit into the ebb and flow of Monarch on a day-to-day basis. Write these down. Clarify issues that have come up for you regarding how the work of our project is progressing.
Procedure for category elicitation Initially, your efforts in the field are directed at “capturing� the thoughts, viewpoints, opinions and descriptions of the people you have interacted with to better understand some phenomenon. If you have been successful, you have captured pages and pages of this descriptive data. What do you then do with all this data? You have to analyze it, and to do that you need some kind of systematic procedure. Data analysis using category elicitation (Hong et al, 2000) is a good way to proceed. The procedure includes (a) listing and compiling (all interview and field notes are transcribed and compiled into a computer file), (b) category elicitation (based on labeling interview responses and observation notes and inspecting labels in search of common categories and subcategories), and (c) mapping (responses are mapped onto tentative categories with subcategories added and/or revised as needed). Finally, categories and subcategories are analyzed to identify the main, over-arching themes for each audit element. In the USD model, we have found the following steps helpful in eliciting categories: (A) Each sentence of a set of notes is to be judged and tentatively labeled. (1) Read a sentence and find a theme of the sentence. Sometimes, you can conjure up a theme from more than one sentence.
Š L. Rowell, 2006
Page |9 (2) The theme is the tentative "category" and the theme is also the “label” at this time. For example, if one talked about needing more counselors, you may tentatively put down in a notebook or computer as "needing more counselors," or perhaps, in the context of the observation, it would be better labeled as “need for resources – counselors.” Clearly, there will be more than one way to label a category. It is okay at this stage that the label seems to be too specific. You may find many tentative categories at this stage. Simply label each one and set it aside for now. (3) As you are reading over notes and adding tentative categories, you will find that several teammates have mentioned the same theme in various sentences. So, there is a certain quantification involved with coding as well. For example, imagine that “need more counselors” was mentioned in four of five sets of notes. You will want to keep track of this dimension as well. (4) After you finish going through your notes the tentative labels are to be inspected to determine if there are common categories and subcategories that can be elicited. For example, a category might be "needing more personnel" and subcategories for this category would be "campus supervisor," "counselors," and "classroom teachers." (B) Tentative mapping (1) After you finish eliciting common categories and subcategories, all participants' responses are mapped onto the tentative categories. This activity is in a sense a "repeat" of the above procedure, but is essential for this type of research for exhausting all participants' statements you might have missed at the first go-around and also for refining the categories tentatively elicited earlier in the process. (2) Add more categories or subcategories, combine categories, or revise categories as needed when you are going through the notes the second time. (3) After going thru the statements the second time, categories are to be inspected one more time to see if further revisions are needed. Most likely, while the main categories will remain relatively the same, the subcategories may need to be revised.
© L. Rowell, 2006
P a g e | 10 Completing the Observation Report Each student submits a 5-6 page report based on the observation plus a copy of all notes taken at the site. The total number of pages for this assignment (the report plus the notes) generally varies between 20-23 pages. The notes can be typed or handwritten. The report includes the following: 1. A description of the action research project setting that conveys the look, sound, and feel of the site as well as basic demographics. 2. A summary of the observation(s) completed. What was observed regarding the human and social environment within which school counseling takes place, counseling program activities and services, informal interactions and unplanned activities among staff and between students and staff, the native language of school counseling at the site, and nonverbal communication among students, teachers, and the rest of the school staff? Lastly, what did not happen that stood out as significant? 3. A brief discussion of how your observations might relate to the upcoming action research project. This is what might be called a first approximation concerning the focus of the project, and as such you should not worry about having the “right answer.” The writing should simply convey some possible hypotheses concerning the relationship between what you all observed and what the practitioner partner has indicated is the area of interest for improving school counseling practice at this site.
Conclusion The observation and observation report are crucial first steps in engaging with real world research. These steps provide early practice in “seeking to say something sensible about a complex, relatively poorly controlled and generally ‘messy’ situation” (Robson, 2002, p. 4). Conducting an observation serves a dual purpose: it provides practice in an actual research method; it helps the collaborative action research team understand the site and develop ideas for formulating a research question. As Robson puts it, observation methods are “commonly used in an exploratory phase, typically in an unstructured form, to seek out what is going on in a situation as a precursor to subsequent testing out of the insights obtained” (pp. 311-312). In the
© L. Rowell, 2006
P a g e | 11 USD model, observation is an important part of the exploratory phase of collaborative action research.
Š L. Rowell, 2006