Development of Training Themes for Joint, Interagency,..(2010) Ross, K. US Army Research by lauren cat

U.S. Army Research Institute For the Behavioral and Social Sciences

Research Note ####

Development of Training Themes for Joint, Interagency, Intergovernmental and Multinational (JIIM) Operations Karol G. Ross Cognitive Performance Group Anna Grome Klein Associates Division, ARA Meagan C. Arrastia Cognitive Performance Group Brooke Schaab US Army Research Institute James Ong Stottler Henke Associates, Inc. David Spangler Global Innovation and Design, Inc.

May 2010

Approved for public release; distribution is unlimited.

U.S. Army Research Institute for the Behavioral and Social Sciences A Directorate of the Department of the Army Deputy Chief of Staff, G1 Authorized and approved for distribution:

MICHELLE SAMS, Ph.D. Director

Technical review by

NOTICES

REPORT DOCUMENTATION PAGE

Form Approved OMB No. 0704-0188

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188), Washington, DC 20503.

1. AGENCY USE ONLY (Leave blank)

2. REPORT DATE February 2010

3. REPORT TYPE AND DATES COVERED October 15, 2006 through October 31, 2009

4. TITLE AND SUBTITLE

5. FUNDING NUMBERS

Development of Training Themes for Joint, Interagency, Intergovernmental and Multinational (JIIM) Operations

W91WAW-07-C-0067

6. AUTHORS - Karol G. Ross, Anna Grome, Meagan C. Arrastia, Brooke Schaab, James Ong, David Spangler 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Stottler Henke Associates, Inc. 951 Mariner’s Island Blvd., Suite 360 San Mateo, CA 94404

8. PERFORMING ORGANIZATION REPORT NUMBER

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) ERDC Contracting Champaign Office PO Box 9005 Champaign, IL. 61826

10. SPONSORING/MONITORING AGENCY REPORT NUMBER

Phase II Research Report No. 345

11. SUPPLEMENTARY NOTES NONE 12a. DISTRIBUTION/AVAILABILITY STATEMENT Distribution authorized to US Government only; Proprietary Information (DFARS – SBIR Data Rights). Other requests for this document shall be referred to the DoD Controlling Office or the DoD SBIR Program Office.

12b. DISTRIBUTION CODE

13. ABSTRACT (Maximum 200 words)

The contemporary operating environment requires a new emphasis on collaboration across services, across government agencies and with non-governmental agencies and host nations to create global stability in the interest of national security. This report documents an effort to capture the expertise of experienced military and non-military players, combined with operational lessons learned and previous research to insure that training and education will prepare our forces for Joint, Interagency, Intergovernmental and Multinational (JIIM) Operations. First, we reviewed documents that reflected lessons learned and insights into these operations. Second, we conducted interviews with experts in collaboration between the military and other organizations, as well as with host nation officials and citizens. Third, we analyzed interview transcripts to derive themes that expressed the high-level cognitive skills in these operations. We identified these themes: Understand the situation within its historical, regional, and cultural context; Understand the other participants; Shift perspective; Establish and maintain common ground; Build capability to affect the situation; Visualize the operation; Support information exchange; and Maintain flexibility. We describe our six-step thematic analysis method and provide analysis tables to clarify the process. Finally, we administered surveys and held focus groups to validate the themes with experienced Civil Affairs officers. 14. SUBJECT TERMS Stability Operations, Joint Interagency Intergovernmental and Multinational Training, Training Themes, HASE, JIIM, Thematic Analysis

15. NUMBER OF PAGES 186 16. PRICE CODE

17. SECURITY CLASSIFICATION OF REPORT UNCLASSIFIED

NSN 7540-01-280-5500

18. SECURITY CLASSIFICATION OF THIS PAGE UNCLASSIFIED

19. SECURITY CLASSIFICATION OF ABSTRACT UNCLASSIFIED

Computer Generated

20. LIMITATION OF ABSTRACT UNLIMITED

STANDARD FORM 298 (Rev 2-89)

Research Note ####

Development of Training Themes for Joint, Interagency, Intergovernmental and Multinational (JIIM) Operations

Karol G. Ross Cognitive Performance Group Anna Grome Klein Associates Division, ARA Meagan C. Arrastia Cognitive Performance Group Brooke Schaab US Army Research Institute James Ong Stottler Henke Associates, Inc. David Spangler Global Innovation and Design, Inc.

Fort Leavenworth Research Unit Stanley M. Halpin, Chief U.S. Army Research Institute for the Behavioral and Social Sciences 2511 Jefferson Davis Highway, Arlington, Virginia 22202-3926

May 2010

DEVELOPMENT OF TRAINING THEMES FOR JOINT, INTERAGENCY, INTERGOVERMENTAL AND MULTINATIONAL (JIIM) OPERATIONS

ACKNOWLEDGEMENTS The authors would like to thank Partners International Foundation who organized the Health African Scenarios Exercise workshop and shared workshop data with us. We also thank faculty at Joint Forces Staff College, members of the 91st Civil Affairs Battalion at Ft. Bragg, the 350th Civil Affairs Command and others who shared their expertise and perspectives on Joint, Interagency, Intergovernmental, and Multinational stability and support operations.

iii

DEVELOPMENT OF TRAINING THEMES FOR JOINT, INTERAGENCY, INTERGOVERMENTAL AND MULTINATIONAL (JIIM) OPERATIONS

EXECUTIVE SUMMARY Research Requirement: The contemporary operating environment for U.S. military forces requires a new emphasis on collaboration across services, across government agencies, with nongovernmental agencies, and with host nations in order to create global stability in the interest of our national security. While operations are driven by U.S. national objectives, our interests are increasingly dependent on defining common goals with others to support the security and stability of a wide range of regions and nations in which we currently are engaged or soon will find ourselves conducting operations. The skills to determine relevant objectives and take effective actions in this collaborative environment have long been a part of U.S. military capabilities, but these skills are now more in the foreground of operations and are required of a wider range of personnel from the tactical to the strategic level. New and emerging doctrine directs the nature of these interactions for the U.S. military. While doctrine provides guidance and structure, it does not capture the expertise required to successfully apply that doctrine. The purpose of this research effort was to identify the high-level cognitive skills required for effective performance in Joint, Interagency, Intergovernmental and Multinational (JIIM) environments to serve as themes to support training development. Procedure: The approach to theme development included three steps. First, we identified and analyzed relevant documents that reflected lessons learned or other insights into operations in JIIM environments. Second, we conducted a series of interviews with subject matter experts whom we had identified as experienced in operations that required collaboration among the military and other organizations, as well as host nation officials and citizens. Third, we analyzed the interview transcripts to derive themes that expressed the high-level cognitive skills evident in the experiences. To represent our findings, we combined interview findings with insights gained from the documents to produce a matrix of themes. The method for theme validation included a quantitative and qualitative approach. First a survey instrument consisting of 54 items was developed from the previous interviews with JIIM operators in order to explore the degree of consensus among a different set of SMEs on the value of the eight themes identified. The items are statements describing the skills that make up each theme with the answer choices on a 5point Likert scale ranging from very important to mission success to very unimportant to mission success. In addition to the survey, a focus group was conducted to discuss the themes with a subset of the individuals that have actually had to employ such skills in

JIIM environments. Feedback about the themes in their own words, as well as examples of each theme, added richness to the data that allowed the themes to be further explored. Findings: A total of eight themes emerged from the data. The themes are listed below. Each theme is presented as a cognitive act that the practitioner in the domain must perform. In addition, the set of themes was integrated into a descriptive model to show their interrelationships and application over the course of an operation. 1. 2. 3. 4. 5. 6. 7. 8.

Understand the situation within its historical, regional, and cultural context Understand the other participants Shift perspective Establish and maintain common ground Build capability to affect the situation Visualize the operation Support information exchange Maintain flexibility

The themes were validated as important to the vast majority of the SMEs surveyed. They recommended few changes to the wording of the definitions and accompanying cognitive challenges. Utilization and Dissemination of Findings: This research produced a set of high-level cognitive skills in the form of themes that will be used to build a training product. The scenario-based tutorials in the product will support the development of a common understanding of successful collaboration in JIIM environments in a training audience that varies in levels of expertise. Use of the product will prepare participants to engage in more difficult training and exercises based on the understanding of the doctrine, processes and skills needed for successful collaboration which they will acquire.

DEVELOPMENT OF TRAINING THEMES FOR JOINT, INTERAGENCY, INTERNATIONAL AND MULTINATIONAL (JIIM) OPERATIONS

CONTENTS INTRODUCTION .............................................................................................................. 1 DEVELOPMENT OF TRAINING THEMES.................................................................... 2 Method ............................................................................................................................ 3 Research Question ...................................................................................................... 3 Participants .................................................................................................................. 3 Data Collection Procedure .......................................................................................... 4 Analysis Plan .............................................................................................................. 4 Results ............................................................................................................................. 5 VALIDATION OF THEMES............................................................................................. 8 Method ............................................................................................................................ 8 Research Question ...................................................................................................... 8 Participants .................................................................................................................. 8 Data Collection Procedure .......................................................................................... 8 Analysis Plan .............................................................................................................. 9 Results ............................................................................................................................. 9 Survey Results ............................................................................................................ 9 Focus Group Results ................................................................................................. 10 DISCUSSION ................................................................................................................... 10 REFERENCES ................................................................................................................. 12 APPENDIX A: LIST OF PARTICIPANTS AND DOCUMENTS ............................... A-1 APPENDIX B: EXAMPLES OF DATA ANALYSIS TABLES ................................... B-1 APPENDIX C: THEMES FOR OPERATIONS IN JIIM ENVIRONMENTS .............. C-1 APPENDIX D: THEME VALIDATION SURVEY ...................................................... D-1 APPENDIX E: THEME VALIDATION FOCUS GROUP GUIDE...............................E-1 APPENDIX F: THEME VALIDATION SURVEY RESULTS ..................................... F-1

LIST OF FIGURES Figure 1. Model of High-Level Cognitive Skills Required for Joint, Interagency, Intergovernmental and Multinational Operations ............................................................... 7

LIST OF TABLES Table 1. Summary of Survey Findings for Validation of Themes .................................... 10

vii

DEVELOPMENT OF TRAINING THEMES FOR JOINT, INTERAGENCY, INTERGOVERMENTAL AND MULTINATIONAL (JIIM) OPERATIONS INTRODUCTION The contemporary operating environment for U.S. military forces requires a new emphasis on collaboration across services, across government agencies, with nongovernmental agencies, and with host nations in order to create global stability in the interest of our national security. While operations are driven by U.S. national objectives, our interests are increasingly dependent on defining common goals with others to support the security and stability of a wide range of regions and nations in which we currently are engaged or soon will find ourselves conducting operations. The skills to determine relevant objectives and take effective actions in this collaborative environment have long been a part of U.S. military capabilities, but these skills are now more in the foreground of operations and are required of a wider range of personnel from the tactical to the strategic level. New and emerging doctrine directs the nature of these interactions for the U.S. military.1 While doctrine provides guidance and structure, it does not capture the expertise required to successfully apply that doctrine. The purpose of this research effort was to identify the high-level cognitive skills required for effective performance in Joint, Interagency, Intergovernmental and Multinational (JIIM) environments. We addressed this requirement by identifying the expertise of experienced members of the military and non-military players in these settings. We combined that understanding of expertise with operational lessons learned and previous research in order to build an integrated set of themes that reflect the skills required for collaboration in the JIIM environment. Expertise in operations in JIIM environments is concentrated within certain elements of the military forces such as Civil Affairs, combat arms units who have successfully engaged in counterinsurgency operations, members of Provincial Reconstruction Teams, and those who have served in Joint or State Department positions. Additionally, insights into expert performance are also found in non-government organizations that are part of the collaborative efforts these operations require. Our approach to understanding expert performance was to conduct interviews across this range of experienced people using a critical incident approach and then to identify themes that can inform the focus of training and education. The purpose of this report is to document the results which will be used for the development of a training product intended to build an understanding of this domain across a diverse audience and to improve collaboration in the JIIM environment. The process by which we turned the expertise into training themes and then validated the themes is described here in detail to stimulate a consistent approach to developing themes as the basis of training development in complex domains. We present the themes in both a matrix and integrated into a model that indicates the application of the themes in operation. 1

See for example U.S. Army Field Manual 3-07 Stability Operations, October 2008; Joint Publication 307.3, Peace Operations, 17 Oct 07; and DoD Directive 3000.05, Military Support for Stability, Security, Transition, and Reconstruction (SSTR) Operations, 28 Nov 05.

DEVELOPMENT OF TRAINING THEMES The concept of using themes to guide cognitive development through situated learning was introduced in the research literature as part of the constructivist approach to instruction. The goal of theme-based instruction is to go beyond superficial familiarity with concepts and facts to mastering the conceptual complexity of an area. Thematic instruction aids the learner in going over the same problem from different viewpoints. This instructional strategy is said to promote cognitive flexibility in a domain of practice and avoids counter productive training, i.e., training that inhibits transfer of knowledge to field performance. Examples of counter-productive training include oversimplification of concepts, linear presentation of material which ignores the inter-related nature of concepts, using one exemplar leading to a student perception of one right answer for complex problems, and using simple analogies for complex systems (Spiro et al, 1992). Theme-based instruction when applied as a method to help the student explore situations from different perspectives produces a more cognitively complex understanding of a domain more quickly, as well as more flexible transfer of concepts to field performance. Theme-based training of cognitive skills has since been successfully implemented for the military. Spiro and Jehng (1990) provided a model of Cognitive Flexibility to guide learning which also proposed the concept of a deeper level of cognitive insight gained by theme-based exploration of situations or scenarios. They asserted that this method of learning supported the ability to spontaneously restructure the knowledge gained in adaptive response to changing situations. Spiro, Feltovich, Jacobson, & Coulson, (1992) demonstrated this approach with adult learners in the area of tactical thinking. They used one case study, the battle of Chancellorsville—a case study often used in U.S. Army tactical education—to support theme-based learning. Their efforts were focused on demonstrating that a deeper conceptual understanding of tactics could be gained and generalized to new settings by using a theme-based structure to examine a scenario or case in depth. During the development of the ―Think like a Commander‖ (TLAC) training environment, a number of high-level cognitive skills common across expert tacticians were summarized to guide deliberate practice in this domain. The approach of TLAC is to explore a situation from the multiple perspectives. In line with the earlier examples of this type of instructional strategy cited above, the high-level constructs were dubbed ―themes‖ (Ross, & Lussier, 1999). This successful theme-based approach to training thinking skills (Lussier, Shadrick, & Prevou, 2003) has since been used to produce training for military crisis management thinking skills in the Red Cape tool (Shadrick, Schaefer, & Beaubien, 2007). Theme-based training as an approach to situated or scenario-based training and education is dependent on generating acceptable themes that reflect the high-level cognitive skills in a domain. While a theme-based approach has been successful in military training in limited applications, the approach for identifying relevant and useful themes to guide such training has not been documented. Thematic analysis is generally used to pull meaning from the data in order to understand a phenomenon or specific aspects of a phenomenon that are psychological (such as the experience of choosing cosmetic surgery or choosing treatment during 2

serious illness) or social in nature (such as patterns of drug abuse in a particular segment of society). Our use of thematic analysis is phenomenological in nature as we seek to understand particular situations from the point of view of subject matter experts, but is aimed specifically at understanding their high-level cognition for the purpose of supporting learning that will go beyond the procedural. In the qualitative research literature, â&#x20AC;&#x2022;[t]hematic analysis is widely used, but there is no clear agreement about what thematic analysis is and how you go about doing itâ&#x20AC;&#x2013; (Braun, & Clarke, 2006, p. 79). There are a number of methods that can generally be thought of as thematic analysis (such as conversation analysis, interpretative phenomenological analysis, discourse analysis, and narrative analysis). Our approach is detailed below. Method Our approach included three steps. First, we identified and analyzed relevant documents that reflected lessons learned or other insights into JIIM operations. Second, we conducted a series of interviews with subject matter experts whom we had identified as experienced in operations that required collaboration among the military and other organizations, as well as host nation officials and citizens. Third, we analyzed interview transcripts to derive themes that expressed the high-level cognitive skills evident in the experiences. To represent our findings, we combined interview findings with insights gained from the documents we had reviewed to produce a matrix of themes. Research Question The focus of this research effort is on the high-level cognitive skills that underlie successful performance in JIIM operations. Our intention was to reflect the findings of critical incident accounts and the perceptions of the individual as they pertained to successful performance. Participants We recruited interview participants with experience in some aspect of JIIM environments. Our goal was to compare multiple perspectives at tactical and operations levels. Participants in the military or associated with military organizations were volunteers. Participants from other organizations were compensated for their time as subject matter experts (SMEs). We interviewed a total of 15 military or military-related SMEs; three African nationals, one affiliated with a non-government organization and two with a government health organization; and five SMEs with Department of State experience for a total of 23 interviews. Data from one additional military interviewee was not used. Prior to the interviews we reviewed a number of relevant documents that capture the skills required of participants in JIIM operations. (See Appendix A for a complete list.) We reviewed 1) JIIM lessons learned (Agrait, & Loughran, 2007); 2) transcripts of three days of meetings of the HASE (Healthy Africa Scenarios Exercise) Workshop in Ghana, Africa held January 21-25, 2008; 3) a briefing on the Joint, Interagency, and

Multinational Plannerâ&#x20AC;&#x2DC;s Course from the Joint Forces Staff College; 4) a report on cognitive challenges in operations other than war (Miller et al., 2003); 5) a report that reviewed the cognitive challenges of working across military organizations (Oâ&#x20AC;&#x2DC;Dea et al., 2006); 6) a report on the elements of cross-cultural competence required for military operations (Ross, 2008); and 7) a report on modeling cross-cultural competence in the U.S. Army (McCloskey, Grandjean, & Ross, in publication). Data Collection Procedure A semi-structured interview protocol was used that was based on the Critical Decision Method (CDM; Crandall, Klein, & Hoffman, 2006; Hoffman, Crandall, & Shadbolt, 1998). All interviews were digitally recorded, and all interviews were transcribed for analysis. Analysis Plan The guiding principle in our analysis was to find the elements of performance that are key to successful planning and operations in JIIM environments. Key aspects were not determined by a simple prevalence count, i.e., how many times certain performance elements were mentioned, but by the emphasis placed on how the operations are accomplished or how they fail, i.e., expert strategies. Existing documents that had previously examined operations in terms of expertise or lessons learned helped us determine the emphasis to place on elements of performance. Our approach was inductive, i.e., the themes identified are linked strongly to the data gathered specifically for this project or to reports that are based on data about performance in JIIM environments. We developed questions to help us indirectly assess the elements of expertise by gathering rich examples of performance and insights and strategies that drove performance. In this approach, the themes may bear little relation to the actual questions asked in the interviews. One assumption of the research is that as expertise develops, people are not generally able to articulate the how and why of their cognitive performance. Therefore, the analysis is grounded in the specifics of the data. The data has been collected so as to expose the cognitive challenges and strategies inherent in situations without directly asking participants why they did things the way they did. The inductive process does not try to fit the data into a pre-conceived framework, but creates the framework from the data. The process also provides a rich description of the cognitive challenges and strategies as opposed to a description of the procedures involved in planning and operations. Analysis requires interpretation of specific data to general themes. Specifics are coded into categories first within each individual data item (interview or document) and then summarized across the data set. The first step in the process was to read through the data set to immerse the analysis team in the findings. The research question informs how reading proceeds. Both analysts made notes of interesting ideas in the data and documented those that might be converted into coding categories. Review of the selected documents was interspersed with reviews of the interview transcripts. The selected documents were not theoretical background, but

were previous findings based on interviews that had been analyzed in a thematic manner and lessons learned generated by experienced participants. Two analysts experienced in thematic analysis for cognitive performance and experienced in the subject area conducted the first step. The outcome was a set of preliminary themes from each analyst and notes to connect each potential theme to the interviews and documents. During this step more themes were generated than are retained during the final analysis. The second step was to generate initial themes of expert high-level cognitive skills. The two analysts independently reviewed each otherâ&#x20AC;&#x2DC;s themes and rationale for each and then discussed the overlap and wording for each theme. The outcome of this step was an agreed upon set of codes that were to serve as preliminary themes. The third step was to convert the preliminary themes into a representation to help the team understand the nature of each theme. A three-column table was constructed to present an initial name for each theme, a one-paragraph definition of the theme, and a list of the cognitive challenges associated with the theme. An example of this can be seen in Appendix B. The fourth step was to review the themes more exhaustively against the data set to identify areas needing refinement. In this step, both analysts re-read the data set. A third analyst was added to the team and also reviewed each data item (report or interview). Each item was coded according to the themes that had been developed, and an additional field was created for â&#x20AC;&#x2022;otherâ&#x20AC;&#x2013; themes or interesting elements that emerged during the complete review. The number of instances of a theme found in the data set, or prevalence, does not necessarily mean a theme is more crucial. Our inclusion of other documents in the data set allowed us to examine the nature of performance in JIIM environments in general across a wide range of data and previous analysis in order to make judgments about the criticality of any one aspect of performance and judge whether to retain inclusion of a theme in the representation. The fifth step was to refine the name of each theme and the corresponding definition and cognitive challenges for each. Our goal was to have a name for each theme that was easy to remember and to create definitions and explanations of cognitive challenges that were drawn from the data, i.e., using the words of the experienced interview participants or findings in the documentation to the extent possible to define and explain the themes. The sixth step was to create a model of expertise in JIIM planning and operations that reflected the high-level cognitive skills of experienced practitioners as opposed to a procedural representation. Our goal was to reflect the flow of cognitive performance and inter-relationship among the themes. Results The interview participants were grouped into military (15), Africa Nationals (3), and Department of State personnel (4). In the interviews, we attempted to concentrate on experiences that were based in relief, development, or reconstruction operations. Some experiences also contained aspects of security operations. Our primary focus was on 5

understanding military performance in JIIM operations which accounted for the majority of interview participants. Our analysis resulted in a total of eight themes. The themes are listed below. Each theme is presented as a cognitive act that the practitioner in the domain must perform. 9. Understand the situation within its historical, regional, and cultural context 10. Understand the other participants 11. Shift perspective 12. Establish and maintain common ground 13. Build capability to affect the situation 14. Visualize the operation 15. Support information exchange 16. Maintain flexibility The themes are presented in a table format including the cognitive challenges for each at Appendix C. The construction of our representation was heavily influenced by our previous work with Klein‘s macrocognition framework for understanding high-level cognitive performance (Klein et al. 2003). The elements of macrocognition were used to help describe the cognitive challenges we discovered in this domain. In the cognitive challenges column we entered macrocognitive labels for the overarching abilities we believed were associated with each theme or we used elements found to be important to cross-cultural competence (e.g., sensemaking, decision making or willingness to engage) as documented in McCloskey et al, in publication. We constructed items for each challenge that are specific manifestations of the macrocognitive or cultural elements in terms of skills and challenges in this specific domain (e.g., ―Understand local/regional and organizational power structures‖ or ―See the situation from another‘s point of view to influence and predict behavior‖). We indicated the high-level skills in the order in which they occur and labeled their relationships in performance. The themes representation also indicates a progression in complexity from initial understanding of the situation to maintaining flexibility during operations. We created a graphic descriptive model of the themes as a high-level depiction of the cognitive skills that support effective performance in the JIIM environment. The model captures how the high-level skills occur in progression as planning and implementation proceed, but also are mutually supportive and require reiteration throughout operations. In addition, some actions are precursors of others in that they set the stage for success in applying the subsequent skill. See Figure 1 below.

Figure 1. Model of High-Level Cognitive Skills Required for Joint, Interagency, Intergovernmental and Multinational Operations 8. Maintain flexibility across interactions to understand and adapt new approaches and to be resilient when attempts to understand others or implement plans are not immediately successful.

1. Understanding the situation within its historical, regional, and cultural context – Learn to read these types of situations. Provide knowledge to

3. Shift perspective – Understand how others see specific aspects of the situation and see your organization.

Provide knowledge to

Must be implemented as you

2. Understanding the other participants in the specific situation – Who are they? How do they do business? What has their role been in this situation?

4. Establish and maintain common ground – What goals can you work on together that are mutually beneficial?

Is the basis for effective

6. Visualize the operation including 2d and 3d order consequences

Provide knowledge to

5. Build the capacity to affect the situation by using all assets and expertise across organizations

7. Support of information exchange – understand communication preferences

VALIDATION OF THEMES Method The method for this phase of the project had two steps, one quantitative and one qualitative. The intent was to investigate the validity of the eight themes for successfully operating in a JIIM environment. First a survey instrument consisting of 54 items was developed from the previous interviews with participants in JIIM environments in order to explore the degree of consensus among a different set of SMEs on the value of the eight themes identified. See Appendix D for a complete list of the survey items. The items are statements describing the skills that make up each theme with the answer choices on a 5-point Likert scale ranging from very important to mission success to very unimportant to mission success. In addition to the survey, a focus group was conducted to discuss the themes with a subset of individuals that have actually had to employ such skills in JIIM environments. Feedback about the themes in their own words, as well as examples of each theme, added richness to the data that allowed the themes to be further explored. The survey data was analyzed in order to find out whether or not the themes are meaningful to experts who have worked in JIIM environments and if there was consensus about the importance of the themes to operations. The focus group transcripts were analyzed for the denial or affirmation of the themes‘ utility as descriptions of effective performance in JIIM. Research Question Our research question was ―Do these themes have utility as descriptions of effective performance in the JIIM environment?‖ This question was addressed with the survey described above. The focus group was intended to provide us with incidents in which the skills described were crucial to mission success. Participants We recruited Civil Affairs personnel with experience in many JIIM environments. Eighteen reservist Civil Affairs officers and NCOs were surveyed. The mean age of the sample was 47 years and the average number of years in service was 23. The sample consisted of three Colonels, eight Lieutenant Colonels, two Majors, three Sergeants First Class, one Master Sergeant, and one Staff Sergeant with 72% of the sample being officers. Of the 18 participants, six were asked to volunteer to participate in a focus group based on the extent of their experience working in JIIM environments. Data Collection Procedure All participants signed the informed consent form, which were collected before the administration of the surveys, as well as the beginning of the focus groups. A semistructured interview protocol was used for the focus groups which consisted of first describing the themes followed by probing questions to determine the importance of each

theme and the relevance of the statements used to describe theme. Critical incidents in which the themes were crucial to the success of missions were also elicited. The focus groups were digitally recorded and transcribed for analysis. Analysis Plan The survey data was analyzed to determine the degree of consensus among the respondents on the value of the eight themes. This analysis consisted of five steps: 1. The data was entered into SPSS, data-analysis software. 2. The data was cleaned and checked for any errors during data entry. 3. The mean answer choice and standard deviation for each question was calculated. A table was constructed listing the frequency of each answer choice, the mean, and the standard deviation of each question. 4. The individual questions were combined into their thematic groups. Once this was done the themes could be reported on instead of the individual questions. 5. The mean answer choice and standard deviation for each theme was calculated. This indicated the general consensus of the importance of each theme to mission success within a JIIM environment. See Table 2 below. The six participants who subsequently formed the focus group were asked to identify incidents that illustrated one or more themes. After the focus group recording was transcribed: 1. The themes were read through to re-familiarize the researcher. 2. The transcripts of the focus groups were read through to familiarize the researcher with the data. 3. The transcripts were analyzed for each theme, one at a time. Incidents mentioned were coded and catalogued under each theme they described. Results Survey Results Each of the eight themes was validated with the use of the 54 items in the survey. Each item was rated on a scale from 1 to 6 with 1 being very important and 6 being very unimportant. Each item was rated as important on average with means ranging from 1.1 to 2.1. See Appendix F for a full listing of means and standard deviations. The rate of importance for each theme ranged from 91-100% with Shifting Perspective being reported as important or very important by an average of 91% of the sample and Maintaining Flexibility being reported as important or very important by 100% of the sample. The percentage of participants that found each theme important or very important is listed in Table 2 below. Although the sample size does not allow for generalization of the results, the findings provide some confidence that these themes are applicable for JIIM training.

Table 1. Summary of Survey Findings for Validation of Themes Theme

Related Item Numbers

Total Items

Understand the situation within its historical, regional, and cultural context Understand the other participants Shift perspective Establish and maintain common ground Build capacity to affect the situation Visualize the operation Support information exchange Maintain Flexibility

1-9

% of Participants that Found the Theme ―Very Important‖ or ―Important‖ for JIIM Operations 95.1%

10-16 17-24 25-31

7 8 7

96.8% 91.0% 99.2%

32-38 39-44 45-49 50-54

7 6 5 5

95.2% 94.4% 91.1% 100%

Focus Group Results In order to further investigate the validity of the JIIM themes and improve them we went over the phrasing of the themes and other data in the matrix, as well as gathered example. Our analysis of the focus groups‘ transcripts confirmed that the challenges and training needs reflected in the themes were meaningful to experienced practitioners. We also revised the wording, but generally the wording was accepted by the focus group members. The eight themes were accepted by all the members of the Civil Affairs sample and the participants of the focus groups expanded on each theme by relating an example from their experience in theater. They agreed that in order to better work in JIIM environments in the future, training needs to be targeted towards specific issues that fall within these different themes. DISCUSSION The themes we derived from interviews with experienced personnel and documents in the area of JIIM operations are appropriate for guiding training development relating to the high-level cognitive skills needed for effective performance in a collaborative environment. Themes are used in situated learning designs to increase the learner‘s ability to see a situation from multiple perspectives. Secondly, they are used to reinforce habits of thought that are consistent with expert performance to guide deliberate practice of these thought processes to accelerate expertise. While theme-based situated learning has been used successfully to teach advanced concepts, we believe that the potential for accelerating the acquisition of entry-level knowledge is equally as important. Too often entry-level knowledge is taught in a manner that is boring and decontextualized. These methods result in a lack of retention and a lack in the student‘s ability to generalize

knowledge to new settings and to higher-level learning, essentially requiring the student to re-learn introductory information later in the education or training process. The themes will be used to support the development of scenario-based tutorials which will support learning for a training audience of mixed levels of experts. Building scenarios around the themes will help learners at different levels understand the basis for successful implementation of doctrine and processes.

REFERENCES Agrait, R. & Loughran, J. J. (2007). Lessons Learned from Joint, Interagency International and Multinational (JIIM) Workshops and Papers. Vienna, VA: ThoughtLink, Inc. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77-101. Crandall, B., Klein, G., & Hoffman, R. R. (2006). Working minds: A practitionerâ&#x20AC;&#x2122;s guide to cognitive task analysis. Cambridge, MA: The MIT Press. Hoffman, R. R., Crandall, B. W., & Shadbolt, N. R. (1998). Use of the critical decision method to elicit expert knowledge: A case study in cognitive task analysis methodology. Human Factors, 40(2), 254-276. Klein, G., Ross, K. G., Moon, B., Klein, D. E., Hoffman, R. R., Hollnagel, E. (May/June 2003). Macrocognition. IEEE Intelligent Systems, 81-85. Lussier, J. W., Shadrick, S. B., & Prevou, M. I. (2003). Think Like a Commander prototype: Instructorâ&#x20AC;&#x2122;s guide to adaptive thinking. (Research Product 2003-02). Alexandria, VA: U. S. Army Research Institute for the Behavioral and Social Sciences. McCloskey, M., Grandjean, A., & Ross, K. (in publication). Assessing learning and development in Army cross-cultural competence. (Phase 1 SBIR Report.) Alexandria, VA: U. S. Army Research Institute for the Behavioral and Social Sciences. Miller, T. E., Phillips, J. K., Battaglia, D. A., Wiggins, S. L., Baxter, H., Mills, J. A., & Ross, K. G. (2003). Collaborative online training for operations other than war (OOTW) (Final Technical Report for Contract #F33165-00-C-6003 Air Force Research Laboratory, Mesa, AZ). Fairborn, OH: Klein Associates Inc. O'Dea, A., Ross, K.G., McHugh, A, Phillips, J.K., Throne, M.H., & McCloskey, M. & Mills, J.A (2006). Global teams: Enhancing the performance of multinational staffs through collaborative online training (Research Report 1849, AD A469425). .Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences Ross, K. G. (2008, May). Toward an operational definition of cross-cultural competence from interview data. Patrick AFB, FL: Defense Equal Opportunity Management Institute (DEOMI). (Available from the Cognitive Performance Group (karol@cognitiveperformancegroup.com).

Ross, K. G., & Lussier J. W. (1999). A Training Solution for Adaptive Battlefield Performance. Proceedings of the Interservice/Industry Training, Simulation & Education Conference (I/ITSEC), 1999. Orlando FL. Shadrick, S. B., Schaefer, P. S., & Beaubien, J. (2007). Development and Content Validation of Crisis Response Training Package Red Cape: Crisis Action Planning and Execution (Research Report 1875). Alexandria, VA: U. S. Army Research Institute for the Behavioral and Social Sciences. Spiro, R. J., Feltovich, P. J., Jacobson, M. J., & Coulson, R. L. (1992). Cognitive Flexibility, constructivism, and hypertext: Random access instruction for advanced knowledge acquisition in ill-structured domains. In T. Duffy, & D. Jonassen (Eds.). Constructivism and the Technology of Instruction: A Conversation (pp.57-75). Hillsdale, NJ: Lawrence Erlbaum Associates. Spiro, R.J., Feltovich, P.J., Coulson, R.L., Jacobson, M.J., Durgunoglu, A., Ravlne, S., & Jehng, J. (1992). Knowledge acquisition for application: Cognitive flexibility and transfer of training in ill-structured domains (ARI Research Note 92-21). .Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.

APPENDIX A: LIST OF PARTICIPANTS AND DOCUMENTS

A-1

Experience Base Iraq, Afghanistan

Mozambique

Organization Dutch Marines NATO US Army

Afghanistan

US Army

4 5

Iraq Afghanistan

US Army US Army

Iraq, Afghanistan

US Marines

Ethiopia, Kenya

US Army

Kenya

US Army

Djibouti, Ethiopia, Yemen, Philippines Afghanistan, Niger, Kenya, Iraq Iraq, Chad

Iraq, Afghanistan

US Army

Iraq, Kenya, Afganistán, Senegal, Cameron Afghanistan, HOA

Afghanistan, Iraq

US Army

Honduras

US Army Corps of Eng.

Point of View Multinat‘l Ops; Mentoring/Training of Iraqi Ministry of Interior/Defense Leadership; Gov‘t infrastructure building, Working in Mozambique with the local NGO and host nation to build HIV/AIDS laboratories. Embassy Affairs; Security Cooperation; NGO/IGO Coordination; Reconstruction Project Mgmt. Studied police reform in Afghanistan through interagency interaction. Worked as Chief of Transportation in Iraq. Worked with the UN and the host nation to stabilize the area.

CAO working with the host nations in Ethiopia and Kenya to provide humanitarian assistance. Working in Kenya with the embassy and host nation to fix water sanitation. CAO worked with local tribes to provide humanitarian assistance.

US Army

COA working in Afghanistan with host nation to provide humanitarian assistance. CAO working in Chad and Iraq with DOD and the State Department. CAO working in Iraq and Afghanistan with NGOs. COA working in Afghanistan with host nation to provide humanitarian assistance.

US Army

A-2

In Afghanistan, he worked with a local NGO to provide a medical clinic. In the HOA, he worked with the State Dept. and the Dept. of Defense. Worked with a local NGO in Afghanistan to provide for Pakistan refugee camps and educate villagers on irrigation and agriculture. Worked with another NGO in Iraq to provide aid in Baghdad. Reconstruction and Stabilization Project Mgmt.

1 2 3 4 5 6 7

Dutch Marines, NL MOD State Dept.

Multinat‘l Strategic Ops and Intel; Int‘l Political Advisement

UN , State Dept. Representative State Dept. Representative State Dept. Representative

Worked with the UN on environmental missions in Haiti and Iraq.

Afghanistan

2 African Nations, Central and S. America Haiti, Iraq

Central Asia

Iraq

Lithuania

US Embassy

Rwanda

Ghana

International NonGovernment Organization African Health Research Center International NonGovernment Organization

State Dept representative that met to develop COA with military personnel.

Worked with the host nations on environmental missions in Central Asia. State Dept. Representative on a multinational committee to exchange information in the Green Zone of Iraq. Worked with the host nation and the State Department. Worked in a political-military organization with local NGOs to obtain humanitarian assistance in Rwanda during the genocide of 1994. African national working with NGOs and US Army in health research laboratory.

African national working with US Army in health research laboratory.

Other Materials ―Lessons Learned from Joint, International, and Multinational (JIIM) Workshops and Papers‖– ThoughtLink, Inc. (Produced as part of this project) Transcript of the Healthy Africa Scenarios Exercise Workshops January 21-25, 2008 JIMPC General Overview Briefing – Joint Forces Staff College ―Global Teams‖ Project Report (interview-based themes) ―Operations Other Than War (OOTW)‖ Project Report (interview-based themes) Cross-Cultural Competence Operational Definition Report (interview-based competence modeling) Cross-Cultural Competence Assessment Report (interview-based competence modeling)

A-3

APPENDIX B: EXAMPLES OF DATA ANALYSIS TABLES

B-1

Interview with Military Personnel Interview with Military Personnel Experience in Iraq Theme Title 1. Understand the situation within its historical, regional, and cultural context 2. Understand the other participants

 When in Iraq… ―The first reaction you have as a civilian walking into that is how do you find your way around? In my case, having not been a diplomat and not having come out of the State Dept., not having NOT been a part of an embassy country team before, and I had a steep learning curve just on the diplomatic side, on the State Dept., country team side to figure out how all those processes and people worked and interacted, etc. On top of that, though, I had to look at MNFI, and MNCI, and MNDB, and all of those other HQ that are layered there and figure out how to effectively integrate my staff with that group of people.‖ (analyst note: this is about understanding the other players, your own role, and how you fit into the mix)  ―Some of these guys have very, very distinct impressions on whether the civilians are of any use at all in a battle space. And so culture is a big deal. We need to get these guys to seek out the civilians, find and know what their capabilities are and exploit those capabilities in the battle space so they don‘t have to do those things that the civilians can do. And then collaborate…..So anything that we can do that gives the military guys a sense of a positive contribution civilians can do that helps them understand that they‘re a different culture, that it helps them understand that we have different communications processes and capabilities, and that our op tempos are different.‖ (analyst note: This overlaps with ―building capability: theme – e.g., identifying and leveraging expertise, using all available assets)

3. Shift Perspective 4. Establish and maintain common ground

 ―I found frequently, that the civilian side of the equation was competing with the military side. There was lack of common objective and understanding of the long-term desired end state. A specific example, front page of the Washington Post here 6, 8, 10 months ago, DoD‘s Current State, Undersecretary Brinkley is in Iraq, came in and briefed us and said we want a stand-up, state-owned enterprise—state-owned enterprise that‘s a failed legacy of Saddam‘s socialist sort of approach to economics—lots of employees, very little output, very poor quality, and the embassy and the State dept. had established as an end state, stamp out

B-2

Interview with Military Personnel

Theme Title all state-owned enterprises, privatize and make it a competitive economy that can merge into the international arena. So you have specific assignment of civil affairs officers, under Gen. Carelli, working under Brinkley with millions of dollars of funding, to stamp out state-owned enterprise, ostensibly you get people off the streets, get them employed because that reduces the number of potential candidates for insurgents. And the other side you have a bunch of civilians beating themselves to death, trying to stamp out state-owned enterprise and to create privatized business to build the economy.‖ (analyst note: This is an example of lack of common ground between civilian and military players)  Plane crash incident: ―…it was a civilian airplane, with six people on board, German registered, one Iraqi, five Germans, the airplane disappeared, it was overdue for fuel, couldn‘t find it, and the question then is ‗well who is responsible for search and rescue?‘ And MNFI and MNCI maintained that they were not. There was no good answer for then who was…..there was no acceptance and no knowledge of who was responsible for search and rescue in that country where we owned the airspace. My initial reaction was one of disbelief and frustration that MNFI, which owned the airspace, and which had air assets available didn‘t acknowledge any role or responsibility….Then sort of aghast that there wasn‘t some contingency plan that would have prescribed the process and steps that a person would take. So we created it on the fly….and it took so long…And the process was not visible to me. So part of the challenge is that the military, the MNFI processes, are not really visible to a large population of the civilian leadership and planners who have to depend on it. It was not collaborative; it was two separate stove pipes coming in and trying to work together. …finally the FRAGO got disseminated out to MNCI, which then resulted in aircraft availability. Which in the end we didn‘t use because __ in Iraq had already taken off and gone. We lost face with the government of Iraq. It set us back in terms of our ability to work with them because they felt we intentionally didn‘t respond, that we intentionally didn‘t give them the information, that we wanted them to look bad, etc…‖ (analyst note: This is an example about lack of common ground, as far as who is responsible for what. This overlaps with ―knowing the other participants‖ and their roles and responsibilities. Following the crash, there was no shared knowledge of who is responsible for search and

B-3

Interview with Military Personnel

Theme Title rescue. No understanding of who‘s in charge and who owns that role/responsibility).  ―If I were to walk in that office again…I‘d ask what‘s our plan? What are our capabilities? Whose responsibility is it? What are the assets that are dedicated to that response? And what is the communication network that is employed to make that work smoothly?‖ (analyst note: This is about building capability, common ground, knowing participants and roles/responsibilities) 5. Build capability to affect the situation

6. Visualize the operation

7. Support information exchange

 ―One of the things I fought for, was to get them to identify transportation as a critical, or essential… I think they called it an Essential Services—oil, water, electricity, they were all designated essential and transportation was not. Well transportation infrastructure is critical and you can‘t distribute oil in Iraq without transportation, you can‘t create autonomy without transportation, you can‘t move guns…all the other things for war, much less a civil economy without highways and rail. And so it should have been that determination should have been made early that transportation was critical. And then how would you leave out the senior person in transportation from the campaign planning? (analyst note: this is about making your own role and expertise clear to others. Convincing other stakeholders of the importance of your input)  ―…there‘s no continuity from incumbent to incumbent, there are no archives, and no records in the civilian side, and which weren‘t while I was there. I inherited no records, I had no overlap from my predecessor, very, very minimal plan existence so there‘s no continuity.‖ (analyst note: This is about a lack of continuity. No visioning of how what you do (or don‘t do) as far as record-keeping and continuity impacts the future of the operation)  Re Gen. Casey‘s Campaign Plan…‖I hadn‘t seen it…. you would think that if transportation infrastructure is critical to our success in Iraq, you would have thought that there would have been an overt attempt to get Transportation and Secretary of Transportation‘s fingerprints on that campaign plan. There had been no effort that I‘m aware of to do any of that. And so I got that plan and I sat down and I wrote a significant number of comments specifically related to this footprint issue. And I tried to run that up channel, they never got their copy of the plan back either, when I left it was probably still in the safe. But, because I had not seen it, and had not been given the opportunity to coordinate on it, to provide input, and I am suspicious that very few people on the civilian side had a chance to act on it.…This is a clear case of two stovepipes B-4

Interview with Military Personnel

Theme Title competing instead of collaborating. I didn‘t see the campaign plan until very late. My ability to provide input to it was greatly constrained, and I‘m not even sure today, how many of my comments got to anybody who would‘ve read them and been able to do anything with them…But the bigger issue is how do you coordinate serious documents, serious plans across that spectrum of all those people, all the relative stakeholders.‖ (analyst note: This is an example of poor info exchange. He had no involvement in Casey‘s campaign plan, no input.)  ―We can‘t collaborate if everything is done on SIPR because the civilians don‘t have access. So we need to look at ways to do… I mean, first of all, how do you share information? And what are the tools for sharing that information? And what are the impediments for sharing that information? And I said SIPR is one of them. Over classification and then not having a common IT or… the other side of that is that you have the military guys who are very far advanced, IWS and all those other tools. They don‘t interface with the civilians on that.‖ (analyst note: this is about recognizing impediments to information exchange with certain players)

B-5

APPENDIX C: THEMES FOR OPERATIONS IN JIIM ENVIRONMENTS

C-1

TITLE 1. Understand the situation within its historical, regional, and cultural context

DEFINITION Do your homework. Understand the context and history of the situation and the participants. Identify drivers/causes of the current situation. Recognize that chaos is typical of relief situations and some development situations. A clear framework for seeing the situation provides a stable basis for assessment and decision making. Learn how to read situations from others who can see more in the situation than you can.

COGNITIVE CHALLENGES Declarative knowledge, development of a general mental model of relief/development situations, sensemaking  Understand and differentiate long-term and short-term issues (for example, know and use the Capital Analysis and Performance Strategy (CAPS) framework).  Know and apply the Tactical Conflict Assessment Framework (TCAF) when conflict is a part of the situation.  Understand the timeline up to the current situation.  Identify and understand critical events on the timeline.  Identify and understand regional and cultural factors that have influenced the current situation.  ―Read‖ novel situations to focus on priority indicators (leading up to and during operations).

2. Understand the other participants

Know the types of organizations and entities that can be involved and identify those who are involved in the specific situation. Be aware of their historical roles and relationships, as well as their priorities and their willingness to collaborate. Also recognize there may be differences in how they do business. There can be differences in work pace, work hours; integration into the community; maintenance of clear cut roles and responsibilities versus diffuse roles; communication styles; metrics for progress; comfort

Declarative knowledge, development and application of a mental model of relief/development operations, cultural competence, willingness to engage others to understand their organization  Consider both organizational culture and culture of host nation to understand how others conduct business.  Understand local/regional and organizational power structures.  Anticipate impacts of

C-2

TITLE

DEFINITION with and availability of technology; authority structures; and whether they concentrate on tasks or relationships to do business. Anticipate how these differences can affect goal focus, information sharing, planning, and decision making in order to work more effectively together. Understand local power structures to anticipate how decisions will be made, approved and supported in the host nation/local region.

COGNITIVE CHALLENGES cultural and organizational values and differences on decision making and action.  Understand the language used in other organizations to describe situations and priorities.  Understand Interagency Management System designed for post conflict/stabilization situations.

3. Shift Perspective

Shift your perspective to see the situation from another person's point of view. Shift Perspective to understand, predict, and influence behavior and foster communication. Be aware of how you and your organization appear to other organizations and the host nation/region. Be able to explain your organization's position. Problems are multi-dimensional and aspects of the situation that are salient to you may be background to others and vice versa. Be aware of the tendency to interpret others‘ decisions and actions based on your own experience base, beliefs, assumptions, and value system. When seeking to understand others‘ rationale for decision-making, consider their intent, their priorities, patterns of living, and long-term goals.

Cultural competence, prediction, persuasion, negotiation, collaboration, sensemaking, rapport building, relationship building  See yourself as others see you.  Present yourself and your organization in a way that is meaningful to others.  See the situation from another‘s point of view to influence and predict behavior.  Anticipate how your own actions and decisions will be interpreted by host nation in order to mitigate negative fallout.

4. Establish and maintain common ground

Establish common ground as a basis for common goals and unity of effort. There are usually many diverse organizations and countries with which we interact in noncombat operations, each with their own goals and cultures. Building in the time necessary to find common C-3

Collaboration, rapport building, relationship building, problem identification, solution identification  Be willing to engage others and build relationships.  Maintain relationships over the course of the operation.

TITLE

DEFINITION ground, and actually discovering it becomes a significant challenge. Establishing common ground requires a willingness to engage and build relationships with people who may be very different from you. Building and maintaining these relationships over the course of the operation takes time. Self-regulation to maintain control and openness when dealing with others takes practice. The resulting relationships create the basis for establishing common goals and coordinated actions.

COGNITIVE CHALLENGES  Engage in self-regulation.  Discover common interests in the situation.  Negotiate goals and solutions.

5. Build capability to affect the situation

Capability is gained by knowing and using all assets to address common goals. It is difficult to anticipate who will be involved in a situation and what assets will be available. Different functional cells of the military organization may have to self-organize in response to the situation in the early stages of operation to leverage assets and opportunities. The diverse set of resources and expertise that may exist in your own organization and other participant organizations is often not explicit. They can go untapped if not deliberately sought out. Explore who has expertise around different issues or problems. Recognize the military can bring technical skill and disciplined decision making to the situation and how that can best be leveraged without "shutting out" others who have ownership in the situation and specific expertise the military does not. Understand own boundaries for action and manage expectations. Contribute to the creation of a solution process to which multiple players can contribute.

Declarative knowledge, envisioning asset employment, recognizing and leveraging options  Identify available expertise.  Know and use all assets; understand how resources are accessed and deployed.  Match assets to the ―right‖ problems, not just those most salient to your organization.  Develop workarounds where resource constraints or barriers to access exist.  Manage expectations.

C-4

TITLE 6. Visualize the operation

DEFINITION Anticipate the need to transition to the next phase including the resources needed for different players. Realize your phase of operations may be different than those of others. A well-developed exit strategy up front is crucial. It guides both large-scale and smaller, incremental decision making. Visualize the transition from relief to development. Recognize the risks to different players in the situation in relation to a decision or action. Visualize risks and mitigating actions in a dynamic manner. Some issues develop rapidly and some over time. Crisis situations are likely to contain both kinds of problems. Some solutions can be affected rapidly and some can only be accomplished over the long-term. Know the difference and don't let short-term solutions overcome longterm possibilities by ignoring second and third order consequences.

COGNITIVE CHALLENGES Mental simulation, synchronization, identification of leverage points, prediction, contingency thinking, decision making, cultural competence  Understand how the dynamic unfolding of the operations may interact with the region‘s historical and local interests.  Anticipate resource needs for the long-term; recognize conflicts between shortterm and long-term issues.  Recognize different phases of operation of different players.  Develop a transition strategy early to guide decisions and actions.  Anticipate 2nd and 3rd order effects of decisions and actions to create lasting solutions.

7. Support information exchange

Recognize different organizations‘ information needs, priorities, and sense of urgency for informationsharing, and be prepared to articulate your own. Understand that the methods and channels of communication differ across organizations such as method of presentation; who shares information; how authority for information sharing is managed; how complete information must be to support decisions; central access to information versus local access for all involved. Recognize different communication styles--flexible versus standardized, embedded in relationships versus "all business" focus. Recognize adherence to a

Uncertainty management, collaboration, coordination, negotiation, persuasion, perspective taking  Identify and respond to different informational needs of your collaborators.  Recognize that different agencies may have different priorities, and as a result, different senses of urgency for information.  Share your own need for information and rationale with collaborative partners.  Respect the different communication channels and methods for sharing information.

C-5

TITLE

8. Maintain flexibility

DEFINITION formal chain of command; degree of autonomy allowed to individuals; likelihood of junior individuals to speak out. Opportunities and limits to information exchange can vary. Create access to information sharing in terms of where and how you conduct business. Understand own planning process and coordination requirements for the situation.

COGNITIVE CHALLENGES

Recognize when your approach or stance in a situation isn‘t working, and be willing to adapt it. Recognize and apply behavior that is most likely to be successful in each situation to successfully communicate with others. Be aware that there are viable approaches and solutions to problems that might be outside the realm of the way you‘ve approached them in the past. Be willing to adapt your tried and true approaches. Recognize that the first approach or attempt may not be successful, and that is not the same as failure.

Problem detection, resilience, adaptation, re-planning, observation, flexible problem solving, situation assessment, decision making  Be reflective and aware of your own and your organization‘s biases.  Refrain from force-fitting a certain approach.  Put more ―mental energy‖ into understanding the situation than in choosing a solution.  Be aware of when it‘s time to shift your approach and try something different.  Recognize when your style is not effectively communicating and have other methods to substitute.

C-6

APPENDIX D: THEME VALIDATION SURVEY

D-1

Survey of Cognitive Training Themes for Joint, Interagency, Intergovernmental and Multinational (JIIM) Operating Environments: The purpose of the questionnaire is to learn what you believe are important skills and abilities that Service Members need in order to adapt to cognitive challenges in the context of Joint, Interagency, Intergovernmental, and Multinational Operations. Your responses will only be used within the JTF-ITS project team. This is a two-part questionnaire: Part 1 Personal Information (4 items) Part 2 Rating the Performance Abilities (55 items) The entire survey will take approximately 60 minutes to complete. Please rate each item according to the instructions provided for that section. If you do not understand these instructions or the items to be rated, the facilitator will provide clarification. Thank you for your participation. Your input is valuable for understanding how to manage the cognitive challenges that can be found in Joint, Interagency, Intergovernmental and Multi-national (JIIM) operating environments.

Joint Task Force, Intelligent Tutoring System (JTF-ITS), SBIR Ph. 2 Dr. Brooke Schaab, Research Sponsor, (Army Research Institute) 757-203-3306 Mr. Jim Ong, Principal Investigator, (Stottler Henke Assoc., Inc) 650-931-2700 Dr. Karol Ross (Cognitive Performance Group) 407-282-4433

D-2

Part 1 Personal Information ______ __________ Please tell us about your background and experiences by completing this section. Your responses on this questionnaire will remain confidential, and all questionnaire results will be reported without attribution to any of the respondents. 1. Identification: Respondent‘s ID Number _______ Age: ______

Gender:

Years of Service: _______

M F

Rank: ___________ MOS/Branch ________

Service: (Circle one)

Army, Air Force, Navy, USMC

Current Duty Position: ___________________________________________________ Date and Location of Last Deployment: ______________________________________

Please circle one answer for the following questions. a. Have you worked with Government Agencies or Departments as part of your civilian occupation? YES

Don‘t Know

b. Have you ever been assigned to a Joint Task Force staff? YES NO Don‘t Know c. Have your military duties required you to work within a JIIM environment? YES

Don‘t Know

d. Have your military duties required you to work with an Allied or Host Nation military unit? YES

Don‘t Know

2. Most Recent Deployments: (in a Joint, Interagency, Intergovernmental, Multinational Operation) Location (Country or Region)

Duty Position

3. Cultural Training and Education: D-3

Date (Month/Year) From… To…

Please list any training or education that prepared you for a Joint, Interagency, Intergovernmental or Multinational command or staff assignment:

Type of Training or Education

Location

Date (Month/Year)

4. Other Information: Please provide other information regarding your background, operational experience, or education/training that you believe to be relevant working with another culture.

D-4

Part 2 Rating Performance Requirements Based on your experience working within a Joint Interagency, Intergovernmental, Multinational (JIIM) environment, rate the importance of the following abilities. Abilities

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.

13. 14.

Very Important 1

Doing homework to understand the historical context of the situation and the participants before arrive Identifying the drivers or causes of the current situation. Having a clear framework to assess the situation. Having an assessing framework when the situation is chaotic Learning to read a situation from others who understand the situation better than I do Differentiating long-term and short-team issues when assessing a situation Establishing a timeline leading to the current situation. Identifying and focusing on key indicators in a situation Understanding of regional factors Knowing what types of organizations and entities could be involved in the situation Understanding the role of each agency before I arrive to a situation Understanding the willingness of different organizations to collaborate Understanding how the agencies involved conduct business, such as, work hours, formality of organizational structure, pace of work, goals, and communication styles Understanding local power structures to understand how decisions will be made, approved and supported D-5

Important 2

Not Sure 3

Unimportant 4

Very Unimportant 5

Based on your experience working within a Joint Interagency, Intergovernmental, Multinational (JIIM) environment, rate the importance of the following abilities. Abilities

15. 16. 17. 18. 19. 20. 21. 22.

Very Important 1

I need to understand the effect of culture on goals and priorities. It is important to understand the Interagency Management System, to conduct a successful mission. Being able to understand the situation from another personâ&#x20AC;&#x2DC;s point of view Taking another personâ&#x20AC;&#x2DC;s perspective, to better predict and influence their behavior Taking another personâ&#x20AC;&#x2DC;s perspective so that I can communicate with them better Being aware of how the US Military appears to other players in the situation Presenting the role of the US Military in a way that is meaningful to others Understanding that my experiences, values, and beliefs are not the only way to interpret the situation Seeing myself as others see me.

23. 24.

Anticipating how my actions and the actions of the US Military will be perceived to avoid negative consequences Unifying efforts by establishing common goals

25. 26. 27. 28.

Taking the time to find common ground with other players in the situation Building and maintain relationships Maintaining emotional self control D-6

Important 2

Not Sure 3

Unimportant 4

Very Unimportant 5

Based on your experience working within a Joint Interagency, Intergovernmental, Multinational (JIIM) environment, rate the importance of the following abilities. Abilities

29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39.

Very Important 1

Maintaining openness to others Being willing to engage with others Negotiating goals and solutions Increasing our capacity by sharing resources across organizations. Understanding and using all assets in the situation Identifying who has expertise about different problems across all players Various players having ownership and contribute assets Understanding the boundaries of US Military actions Managing expectations about US Military actions Recognizing that your phase of operation may be different from the phase of operations of other players Anticipating the resources needed for all the different players to transition to the next phase of operations A well-developed exist strategy

40. 41. 42.

Visualizing the transition from addressing short-term emergency issues to long-term stability issues Understanding when long-term and short-term solutions conflict

D-7

Important 2

Not Sure 3

Unimportant 4

Very Unimportant 5

Based on your experience working within a Joint Interagency, Intergovernmental, Multinational (JIIM) environment, rate the importance of the following abilities. Abilities

43. 44. 45. 46. 47. 48.

49. 50. 51. 52. 53. 54. 55.

Very Important 1

Creating or at least not interfering with long-term solutions when responding to a crisis Visualizing an operation to avoid potential negative consequences Recognizing that different organizations have different needs, priorities, and urgencies for sharing information Understanding that different organizations have different methods and lines of authority for sharing information Creating access for information sharing by deciding where and how to conduct business Understanding the militaryâ&#x20AC;&#x2DC;s planning and process when sharing information with other organizations Sharing information about the military planning and coordination process with other players when trying to share information with other organizations Recognizing when my approach isnâ&#x20AC;&#x2DC;t working and adapting it Understanding the methods I have tried before may not work a new situation Being resilient and trying new approaches when the first attempt does not work Understanding a situation in order to create flexible solutions Having more than one style of communicating Understanding doctrine 3-07 D-8

Important 2

Not Sure 3

Unimportant 4

Very Unimportant 5

Based on your training and experience, what other performance factors are important in a JIIM environment?

Thank you for your responses. This completes the JTF-ITS survey. Please turn in this your responses to the Facilitator.

D-9

APPENDIX E: THEME VALIDATION FOCUS GROUP GUIDE

E-1

Personal Information

______ __________

Please tell us about your background and experiences by completing this section. Your responses on this questionnaire will remain confidential, and all questionnaire results will be reported without attribution to any of the respondents. 5. Identification: Respondent‘s ID Number _______ Age: ______

Gender:

Years of Service: _______

M F

Rank: ___________ MOS/Branch ________

Service: (Circle one)

Army, Air Force, Navy, USMC

Current Duty Position: ___________________________________________________ Date and Location of Last Deployment: ______________________________________

Please circle one answer for the following questions. e. Have you worked with Government Agencies or Departments as part of your civilian occupation? YES

Don‘t Know

f. Have you ever been assigned to a Joint Task Force staff? YES NO Don‘t Know g. Have your military duties required you to work within a JIIM environment? YES

Don‘t Know

h. Have your military duties required you to work with an Allied or Host Nation military unit? YES

Don‘t Know

6. Most Recent Deployments: (in a Joint, Interagency, Intergovernmental, Multinational Operation) Location (Country or Region)

Duty Position

E-2

Date (Month/Year) From… To…

7. Cultural Training and Education: Please list any training or education that prepared you for a Joint, Interagency, Intergovernmental or Multinational command or staff assignment:

Type of Training or Education

Location

Date (Month/Year)

8. Other Information: Please provide other information regarding your background, operational experience, or education/training that you believe to be relevant working with another culture.

The matrix of themes was attached for discussion to gather examples of each theme.

E-3

APPENDIX F: THEME VALIDATION SURVEY RESULTS

F-1

Item

1 Very Important

1. Doing my homework to understand the historical context of 88.9% the situation and the (16) participants before arriving 2. Identifying the 77.8% drivers or causes of (14) the current situation 3. Having a clear 61.1% framework to assess (11) the situation 4. Having an assessing 50% framework when the (9) situation is chaotic 5. Learning to read a situation from others 50% who understand the (9) situation better than I do 6. Differentiating longterm and short-team 50% issues when assessing (9) a situation 7. Establishing a 22.2% timeline leading to (4) the current situation 8. Identifying and focusing on key 50% (9) indicators in a situation 72.2% 9. Understanding of regional factors (13) 10. Knowing what types of organizations and 83.3% entities could be (15) involved in the situation 11. Understanding the role of each agency 55.6 (10) before I arrive to a

5 Very Important Not Sure Unimportant Unimportant

Mean

11.1% (2)

1.1

16.7% (3)

1.2

33.3% (6)

5.6% (1)

1.4

38.9% 11.1% (7) (2)

1.6

50% (9)

1.5

50% (9)

1.5

66.7% (12)

5.6% (1)

1.9

38.9% 11.1 (2) (7)

1.6

27.8% (5)

1.3

16.7% (3)

1.2

44.4% (8)

1.4

F-2

Item situation 12. Understanding the willingness of different organizations to collaborate 13. Understanding how the agencies involved conduct business, such as, work hours, formality of organizational structure, pace of work, goals, and communication styles 14. Understanding local power structures to understand how decisions will be made, approved and supported 15. Understanding the effect of culture on goals and priorities 16. Understanding the Interagency Management System, to conduct a successful mission 17. Being able to understand the situation from another personâ&#x20AC;&#x2DC;s point of view 18. Taking another personâ&#x20AC;&#x2DC;s perspective, to better predict and influence their behavior 19. Taking another personâ&#x20AC;&#x2DC;s perspective so that I can

1 Very Important

5 Very Important Not Sure Unimportant Unimportant

38.9% (7)

61.1% (11)

Mean

1.6

38.9% 5.6% 55.6 (10) (7) (1)

1.7

66.7% (12)

33.3% (6)

1.3

66.7% (12)

33.3% (6)

1.3

22.2% (4)

61.1% 11.1% (11) (2)

5.6% (1)

33.3% (6)

61.1% (11)

5.6% (1)

1.7

33.3% (6)

50% (9)

16.7% (3)

1.8

33.3% (6)

55.6% (10)

5.6% (1)

1.8

F-3

Item

20.

21.

22.

23.

communicate with them better Being aware of how the US Military appears to other players in the situation Presenting the role of the US Military in a way that is meaningful to others Understanding that my experiences, values, and beliefs are not the only way to interpret the situation Seeing myself as others see me

1 Very Important

5 Very Important Not Sure Unimportant Unimportant

55.6% (10)

38.9% (7)

5.6% (1)

72.2% (13)

22.2% (4)

5.6% (1)

66.7% (12)

33.3% (6)

44.4% (8)

24. Anticipating how my actions and the actions of the US 72.2% Military will be (13) perceived to avoid negative consequences 25. Unifying efforts by establishing common 44.4% goals (8) 26. Taking the time to find common ground 50% with other players in (9) the situation 27. Building and 77.8% maintaining (14) relationships 28. Maintaining 61.1% emotional self control (11) 29. Maintaining openness 38.9%

Mean

1.6

1.3

33.3% 11.1% (6) (2)

11.1% (2)

1.9

22.2% (4)

5.6% (1)

1.4

55.6% (10)

1.6

50% (9)

1.5

22.2% (4)

1.2

38.9% (7)

1.4

55.6%

5.6%

1.7

F-4

Item

1 Very Important

5 Very Important Not Sure Unimportant Unimportant

to others (7) (10) 61.1% 38.9% 30. Being willing to engage with others (11) (7) 31. Negotiating goals and 61.1% 38.9% solutions (11) (7) 32. Increasing our capacity by sharing 16.7% 77.8% resources across (3) (14) organizations 33. Understanding and 55.6% 44.4% using all assets in the (10) (8) situation 34. Identifying who has expertise about 61.1% 38.9% different problems (11) (7) across all players 35. Various players having ownership 50% 38.9% and contributing (9) (7) assets 36. Understanding the 66.7% 27.8% boundaries of the US (12) (5) Militaryâ&#x20AC;&#x2DC;s actions 37. Managing expectations about 55.6% 33.3% the US Militaryâ&#x20AC;&#x2DC;s (10) (6) actions 38. Recognizing that your phase of operation may be 38.9% 61.1% different from the (7) (11) phase of operations of other players 39. Anticipating the resources needed for all the different 27.8% 66.7 (12) players to transition (5) to the next phase of operation 40. A well developed 27.8% exist-strategy 66.7 (12) (5)

F-5

Mean

(1) 0%

1.4

5.6% (1)

1.9

1.4

11.1% (2)

1.6

5.6% (1)

1.4

5.6% (1)

1.6

5.6% (1)

1.8

5.6% (1)

1.4

Item 41. Visualizing the transition from addressing short-term emergency issues to long-term stability issues 42. Understanding when long-term and shortterm solutions conflict 43. Creating or at least not interfering with long-term solutions when responding to a crisis 44. Visualizing an operation to avoid potential negative consequences 45. Recognizing that different organizations have different needs, priorities, and urgencies for sharing information 46. Understanding that different organizations have different methods and lines of authority for sharing information 47. Creating access for information sharing by deciding where and how to conduct business 48. Understanding the militaryâ&#x20AC;&#x2DC;s planning and process when sharing information with other organizations

1 Very Important

5 Very Important Not Sure Unimportant Unimportant

22.2% (4)

72.2% (13)

22.2% (4)

61.1% 11.1 (2) (11)

27.8% (5)

72.2% (13)

44.4% (8)

Mean

1.8

5.6% (1)

1.7

55.6% (10)

1.6

33.3% (6)

66.7% (12)

1.7

22.2% (4)

77.8% (14)

1.8

33.3% (6)

50% (9)

16.7% (3)

1.8

44.4% (8)

55.6% (10)

1.6

F-6

Item 49. Sharing information about the military planning and coordination process with other players when trying to share information 50. Recognizing when my approach isnâ&#x20AC;&#x2DC;t working and adapting it 51. Understanding the methods I have tried before may not work in a new situation 52. Being resilient and trying new approaches when the first attempt does not work 53. Understanding a situation in order to create flexible solutions 54. Having more than one style of communicating

1 Very Important

5 Very Important Not Sure Unimportant Unimportant

27.8% (5)

44.4% 16.7% (8) (3)

66.7% (12)

33.3% (6)

44.4% (8)

Mean

11.1% (2)

2.1

1.3

55.6% (10)

1.6

61.1% (11)

38.9% (7)

1.4

55.6% (10)

44.4% (8)

1.4

55.6% (10)

44.4% (8)

1.4

F-7