Project Delivery Toward the Future Phase 1: literature review and framework design
Document prepared by Amy Ennen
Acknowledgements This research is for one of four pilot research internships for the new Master of Science in Research Practices (MSRP) degree program at the University of Minnesota. The MSRP is a program that was created by Renee Cheng while she was Head of the School of Architecture and is the only one of its kind in architectural academia. Amy Ennen was identified as the student to conduct this particular research based on her previous experience in business and developing work flows. The research was guided by academic advisors, Renee Cheng and Andrea Johnson. The research would not have developed without their wisdom, patience and direction. Griff Davenport, DLR Group, and Robert Nartonis, Mortenson Construction, were also instrumental in the success of this research. Additionally, the list of special thanks is endless, however, it is important to name the people and institutions that substantively contributed to moving this research forward or in some cases, a new direction. Those names follow: Derek Cunz, Mortenson Construction Chris Gibbs, DLR Group Eman Alahawaf, University of Minnesota Dennis Bane, DLR Group Barbara Bryson, Rice University Kevin Collins, Victor O. Schinnerer & Co., Inc. Greg Donofrio, Univeristy of Minnesota Billie Faircloth, Kieran Timberlake Dean Thomas Fisher, University of Minnesota Kate Golden, Mortenson Construction Patrick O’Connor, Faegre Baker Daniels LLP Alisdair McGregor, ARUP Faaiza Rashid, Harvard Business School Adam St. Cyr, DLR Group American Institute of Architects College of Design, University of Minnesota Construction Industry Institute Design Future’s Council
Abstract The following report contains a summary of the project delivery research conducted, fall 2013, at the University of Minnesota, in conjunction with Mortenson Construction, DLR Group and the College of Design, Consortium for Research Practices. The purpose is to determine measurable metrics that will allow future projects in the architecture/engineering/construction industry to predict results and drive towards a higher level of success. The research methodology is a mixed methods approach and the survey engine is Qualtrics. This is a robust research engine used by the University of Minnesota that includes a capacity for complex surveys and enhanced data analysis tools. The research will operate out of a live lab and span 24-36 months. The following report outlines the establishment of the purpose, the metrics, the survey questions used to measure the metrics and the overall frame work of the research.
Table of Contents Introduction Research Methodology Literature Review Metrics Projects Surveys Live Lab Work Flow Conclusion Bibliography
Introduction Research has shown that there is an opportunity to improve the project delivery process. A market research firm in Australia found that most clients surveyed believe that traditional contractual forms often lead to adversarial situations between parties which negatively impact the project (ACA). In the United States, a new approach to project delivery called Integrated Project Delivery (IPD) is being tested. The American Institute of Architects, California Council, has defined Integrated Project Delivery as an approach “that integrates people, systems, business structures and practices into a process that collaboratively harnesses the talents and insights of all participants to reduce waste and optimize efficiency through all phases of design, fabrication and construction.� The California Council went on to define the principles of an integrated project. The principles are: mutual trust; mutual benefit; early goal definition; enhanced communication; clearly defined open standards; appropriate technology; high performance and leadership. The larger ideas and principles set forth around integrated project delivery provide a solid ideology. The California Council even recommends how to build an integrated project delivery team, but how does it really work? The initial questions that arose during this research were focused on testing integrated project delivery. As with any large system, it is necessary to deconstruct it, and find the correct metrics to measure. It is also important to understand the existing seminal studies and literature. The first six weeks of the 15 week term were spent conducting a literature review. The review looked to past case studies, theoretical papers and literature around each component of project delivery. For example, seminal studies like the Penn State Study of 1997 were reviewed, as well as, white papers solely on BIM. Additional initial research was conducted by way of informal conversations with many experts in the field. During this phase, metrics were extracted from the literature and conversations. Also during this phase, it became clear that the research needed to remove the formality of Integrated Project Delivery and focus on the metrics that may or may not make up a successful project. The definition of a successful project seemed to be the same regardless of the project delivery method used. Generally speaking, it included saving time and money and avoiding adversarial conditions. This being said the ideas of integrated project delivery and collaboration were ones that could resurface - as the metrics had developed with them in mind - they were just no longer a constraint of the research.
The working thesis at this point became: We are studying project delivery because we want to find out where, and how, time, energy and money are gained/lost, during the process, in order to better understand how we can make changes and improvements that lend themselves to a more sustainable world. Energy relates to performance of both the team and the building. These metrics were vetted by leaders in each of the industries. Most of this information was contained within spreadsheets and examples of this can be seen further along in this document. It was decided that the framework of this research should operate as a live lab and extend over 24-36 months. This decision was made in an effort to avoid potential inaccuracies that can occur when surveys are conducted after a project is completed. The live lab scenario would allow the research to track projects in real time and make appropriate adjustments along the way. The decision to conduct this research in a live lab also paired well with the research methodology. The research methodology options were also researched. Research has assumptions inherent in it that drive how and what is learned so it is important to make an informed decision when selecting methodology. A shorter literature review around research methodology was conducted in order to gain perspective regarding the options that existed. A mixed methods approach was selected - see Research Methodology section of this report for more details.
How can we best choreograph linear and non linear processes?
The projects identified for participation came from both DLR Group and Mortenson Construction and encompassed most of the United States and a broad range of typologies and project costs. There is a map illustrating their approximate location in the Projects section of this report. Specifics will be revealed at a later date if all parties agree. There are over 20 projects being tested. Having more than 15 projects was crucial because this allows more advanced statistical modeling to be used during the analysis phase. The University of Minnesota is a renowned research university with robust research tools available to students and faculty. Approval was required prior to receiving clearance to use Qualtrics, a comprehensive survey and analytics tool. It was determined by the Institutional Review Board and the Human Research Protection Program that this research was exempt from further approvals and Qualtrics usage was granted. This report contains a few screen shots of the Qualtrics tool. There are also screen shots of the survey questions developed, however, not all of that information can be shared in a document of this nature. A letter to the owner was also drafted during this term - requesting that each owner opt out if he/she were not interested in participating. Additionally, this report establishes the framework for the live lab spanning the 24-36 months with regard to staffing and tasks to be completed on a semester by semester basis. The framework is intended to be flexible so that it can adjust to unforeseen changes to the overall research project and accept additional projects from other contractors and architecture firms. This report is meant to provide an idea of the work that was done during the first semester of this project. Naturally, not every spreadsheet could be shared in its entirety, but please follow the project and feel free to contact anyone involved should you have questions. At this juncture the next research intern, Matthew Tierney, has begun to administer the initial surveys. Amy Ennen is still involved, but working less hours. Tierney is also instrumental in developing dynamic graphics that will respond to the data as it is collected.
Research Methodology When selecting a research methodology it is important to consider the knowledge claim, strategies and methods. The following pages examine the options that were considered and show the choice that was made.
Procedures of research or strategies.
What constitutes the knowledge? Knowledge claim.
Procedures of data collection or methods.
Knowledge Claim Post-Positive (Scientific Method/Quantitative/Positivist/Empirical Science) Deterministic in that causes determine outcomes. Reduces ideas into discrete set of ideas to test. Knowledge is developed through careful observation and measurement. Researcher begins with idea (assumption), collects data to support or refute the assumption and makes necessary revisions prior to conducting additional tests. 1. Knowledge is conjectural and there is no absolute truth. 2. Research is process of making claims and refining or abandoning them for other claims. 3. Data, evidence, and rational considerations shape knowledge. 4. Research seeks to develop relevant true statements to explain situation or describe causal relationships. 5. Objectivity is essential to competent inquiry. Continually examining for biases. Constructivism People seek meaning and develop subjective meanings of their experiences typically toward certain objects or things. Researcher seeks complexity of views. Broad and general questions so participant can construct meaning. Often contextual and historically influenced. Do not tend to begin with an assumption or theory, but typically induce a theory or see a pattern. 1. Meanings are constructed by human beings. Researchers use open ended questions. 2. Humans make sense of their world based on historical and social perspective. Researcher must understand the context via visiting site personally. 3. The basic generation of meaning is always social, arising in and out of interaction with a human community. Advocacy/Participatory For marginalized individuals and groups. Believed post-positivists were too structured and that constructivists did not go far enough in advocating for action. Researchers believe that inquiry needs to be intertwined with political agenda. Typically addresses important social issues like inequality, suppression and alienation. Collaborative. Participants help form questions, collect data and analyze. 1. Participatory action focused on bringing change to practice. 2. Meant to help individuals free themselves from constraints. 3. Meant to ignite political debate and change. 4. Practical and collaborative because done together instead of to someone.
Pragmatic Arises out of actions, situations and consequences. The concern is with what works and is solution oriented. Methods are less important and the problem/solution is most important. 1. Not committed to one system of philosophy and/or reality. Mixed methods research used. 2. Researcher is free to choose the methods that best suit their needs and purposes. 3. Do not see world as absolute unity. Look at multiple methods of collecting and analyzing data. 4. Truth is what works at the time and not dualistic. 5. Researchers look at what and how to research based on consequences - where they want to go with it. 6. Believe research is contextual.
Strategies Quantitative is often aligned with post-positivist and can include: true experiments, quasi-experiments, correlational studies, more complex experiments with many factors using factorial design and repeated measure design. Surveys are also a method that can include cross sectional and longitudinal studies using questionnaires or structures interviews for data collection with the intent of scaling the outcome. Often uses cause and effect thinking. Qualitative is often aligned with constructivism and advocacy and can include: ethnographies, grounded theory, case studies, phenomenological research and narrative research. Often uses open ended questions and looks for themes in emerging data. Researcher brings personal values to the study and positions him/herself.
Mixed Methods often aligns with pragmatism and can include anything - with the idea that the biases will triangulate and therefore be eliminated or reduced. Often includes simultaneous or sequential data collection. Develops a rationale for mixing and integrating data at different stages of inquiry. It fits a researcher who likes structure of quantitative and flexibility of qualitative inquiry. This is our strategy. We will use small focus groups to vet ambiguous metrics and literature. We will combine observations and interviews with traditional surveys. We will nest milestone interviews within traditional surveys, along a timeline, that occur at the end of each indicated phase. We will collect data from all relevant parties per that phase of design and construction. We will collect as much data as possible on our own about each project.
Methods Sequential and nested. Closed ended measures and open ended observations. We are using this method because on one hand we are trying to understand the best predictors of outcomes (quantitative) and on the other it is a new topic so we must remain exploratory (qualitative). It also lends itself to the feedback loop that produces layered and rich research.
Define Scope Literature Review (qualitative/emergent/observational/coded). Conversations around findings based on key players knowledge and experiences (biased by each individuals constructs/emergent/observational). Projects selected based on timeframe (quantitative/specific). Milestone interviews and focus groups (qualitative/emergent) nested along a trajectory of traditional surveys (specified in advance). Analysis
Literature Review Most of the literature included in the review was publish during or after 2000 - aside from a few select works.
Literature Review sample spreadsheet
Metrics What are collaborative characteristics? Can we measure them? How? These were some of the initial questions that were asked while reading the literature. There was a constant attention to what these characteristics might be and how to measure them. Mutual respect Integrated close out Early goal definition
No blame culture Decisions made by most able - best for project. Roles not clearly defined
Interoperability
Transparent data structure
Open data exchange
Shared risk/reward
Disciplined data structure BIM Clear standards
Transparent business exchange
Insights from all valued
Integrated team
Enhanced communication open, straight and direct
Early contribution of knowledge and experience
Early buy-in
Potential characteristics were then further broken down. Questions were generated based on potential ways to measure.
Does that mean everyone emails everything?
Does that mean more informal conversations offline? Enhanced communication open, straight and direct
Does everyone get copied on every email?
Are techniques like mirroring or paraphrasing best?
The metrics were collected in a spreadsheet.
Metrics example
The questions devised to measure the metrics and the metrics were paired and organized by category. The categories were the different areas of project delivery - for example: insurance, money, time or technology.
DATA WILL DRIVE THE RESULTS
Metrics reorganized and measurements developed
Metrics reorganized temporally
This diagram was made to illustrate how the metrics were culled from the literature, organized and reorganized and eventually matched with the phases of design. This means that metrics and measurement questions were grouped according to the phase of design in which they would need to be known. The research was taking on strong temporal characteristics that worked well with the established 24-36 month live lab.
First pass at scope + Literature Review Vetted by experts
Metrics Emerged
Categories made based on topic
Categories (re)made based on time
One survey per time period + follow up interviews face to face
S0 General Information S1 Pre-Design S2 Schematic Design S3 Design Development S4 Construction Development S5 Bid/Negotiation S6 Construction Administration S7 Post Occupancy
Projects
Identified projects. Design Start Date: Late 2013 - Early 2014 Project Cost: $12M - $250M Market Sectors: Justice/Civic, Higher Education, Health Care, Higher Education/Sports, Hospitality, Corporate, Education K-12
University of Minnesota College of Design 145 Rapson Hall 89 Church Street SE Minneapolis, MN 55455 January 17, 2014 Owner Title Address Dear Owner: I am writing to you on behalf of the Research Practices Consortium at the University of Minnesota to request your permission to include one of your projects in our current research. I am working with Professor Renée Cheng, AIA, to conduct research on project delivery, developing predictable metrics on project and team outcomes. We have identified 20 projects to study, and your project, XYZ, was selected as a good candidate for inclusion by our consortium partner Mortenson Construction/DLR Group. All information will be coded and aggregated; no projects will be identified by name. Project team members, including owner or owner’s representatives, will be asked to participate in short questionnaires taking no more than 10-15 minutes each. We anticipate that participation will not hinder normal project operations and will provide a benefit for any future projects. We will share the results with you when the research is complete. The Research Practices Consortium was founded in 2012 with the intention of connecting the university to the profession. The profession brings current issues to the table and the university conducts research then communicates the results, creating a viable feedback loop. More information can be found at www.rp.design.umn.edu. We would like to begin contacting project teams by mid-January and follow projects for eighteen months or until completion whichever comes first. Please contact me if you do not want your project included in this study, if we don’t hear from you by February 5, we will assume that you have no objection to including your project in the study: enne0007@ umn.edu. Thank you in advance for your consideration. Sincerely, Amy Ennen Graduate Student
Surveys
R
Ma
Mc
O
PMa
PMc
T
24Q
27Q
25Q
17Q
84Q
91Q
9Q
The graphic above quickly shows the number of questions estimated for each survey. The letter indicates to whom the survey wold be administered - for example, O is owner and PMa is Project Manager (architect side). The form to the right shows the Institutional Review Board’s determination of exempt. This means the University of Minnesota endorses the use of Qualtrics for this research. Examples of Qualtrics are on the following page.
Route this form to: Human Research Protection Program
DETERMINATION OF HUMAN SUBJECT RESEARCH
U Wide Form Aug 2013
Version 1.1 Updated August 2013, go http://www.irb.umn.edu for the latest version This form is used to help researchers determine if a project requires IRB review. If evidence that IRB review is not required, this form may be used to document that the IRB has reviewed the project description and issued a determination. Please allow two (2) business days for review and response.
FOR IRB USE ONLY:
Section 1
Project Title
Provide the project title below If the project is funded or if funding is pending, the project title must match grant title.
Based on the informaƟon provided, this project does not meet the regulatory deĮniƟon of human subjects research. IRB approval IS NOT required.
Project Delivery - Toward a New Future
Section 2 Contact Information Name (Last name, First name MI):
Ennen, Amy E Email:
Affiliation: U of M Fairview Gillette U of M x.500 ID (ex. smith001):
enne0007@umn.edu
enne0007
323-854-9571
College of Design, Architecture
Phone Number:
Other
U of M Department
Section 3 Project Description 1. Provide a brief description of your project. Include a description of what any participants will be asked to do and a description of the data accessed and/or collected (1,000 character limit).
I am working with the Consortsium for Research Practices at the College of Design. I am a third year graduate student conducting research on project delivery methods. We are hoping to discover predictable metrics that will help save time and money and reduce conflict. We need to administer a set of short surveys to participants to collect data on the projects we have identified as viable for this study. It is existing data. We are using a mixed methods approach so we do not have a specific research question or hypothesis, per se. Much of the data is publically available, but not all of it.
Section 4 Is it Human Subjects Research as Defined by Federal Regulations? The University of Minnesota is an equal opportunity educator & employer. ¤ 2006 by the Regents of the University of Minnesota.
1
Surveys written using Qualtrics
Survey preview sample - Univeristy of Minnesota brand endorsement
As the survey questions developed they were coded back to a category, a variable and a sub-variable. The categories are: context and waste. The variables are: time, money, building and process. The sub-variables are: schedule, team, technology, experience, project scope, leadership, delivery method, communication, context and risk/reward. The generic interaction of these can be seen on the following page.
Coded survey sample
Money
Time
Schedule
Team
Waste
Context
Technology
Process
Leaders
Experience
Project Scope
Building
Communication
Delivery Method
Risk/Reward
Context
Team: relates to cohesion, collaboration, synchronicity Schedule: relates to time Risk/Reward: relates to incentives, liability, sharing risk/reward (#incentives:#months duration project) Project Scope: relates to energy, efficiency of building, program Communication: relates to how and frequency of communication Experience: relates to impacts experience or lack of experience has on outcomes Leadership: relates to how leadership or lack of leadership impacts variables Delivery Method: relates to components embedded in methodology Technology: relates to enhancing or encumbering collaboration with regard to technology Context: relates to characteristics like location
Outcomes of savings or losses (the variables) (the sub variable) Team cohesiveness and synchronicity, +/- disputes, impacts time and money
Contributing factors (the questions) The diagram above shows the flow from contributing factor(s) to outcome(s). The diagram to the right is an example of this idea.
RISK/REWARD
Pa st ex pe Lia rie bil ity n wa ces Fol w ive low ith r s thr te oug am hb Clea y le r com ade mun icati o rs n
Complexity of project et provided g d u b d e il ta e D d enti vize tract c n i s e n Milesto in contract t i c i l p con ve es ex n n i o t s t i nti Mile plic e x c e s n in g ti ve o n e Inc ti n g ugh n o r u h n wt cco nti o a l l u k Fo oo acco b en ox b Op k ac l B
Live Lab Work Flow The remainder of this report outlines the basic work flow frame work in terms of staffing and primary tasks to be completed during each phase. It also shows the that there is an element of overlapping.
S0 General Information S1 Pre-Design S2 Schematic Design S3 Design Development S4 Construction Development S5 Bid/Negotiation S6 Construction Administration S7 Post Occupancy
Set Objectives
Design Study
Staffing needs. One graduate student at 25 hours a week. Additional student to co-conduct interviews at 5-10 hours a week. Tasks. Administer electronic survey to marketing coordinator. Schedule and conduct interviews. General Time F rame. Spring 2014.
Prepare Instrument
Administer Survey
Manage Survey
Analyze Survey
Report Results
S0 General Information S1 Pre-Design S2 Schematic Design S3 Design Development S4 Construction Development S5 Bid/Negotiation S6 Construction Administration S7 Post Occupancy
Set Objectives
Design Study
Staffing needs. One graduate student at 25 hours a week. Additional student to co-conduct interviews 5-10 hours a week. Tasks. Populate database with Survey 1 feedback. Initiate analysis of Survey 1. Administer Survey 2 in some instances. General Time Frame. Spring 2014.
Prepare Instrument
Administer Survey
Manage Survey
Analyze Survey
Report Results
S0 General Information S1 Pre-Design S2 Schematic Design S3 Design Development S4 Construction Development S5 Bid/Negotiation S6 Construction Administration S7 Post Occupancy
Set Objectives
Design Study
Staffing needs. One graduate student at 25 hours a week. Additional student to co-conduct interviews at 5-10 hours a week. Tasks. Populate database with Survey 2 feedback. Initiate analysis of Survey 2. Administer Survey 3 in some instances. General Time Frame. Spring 2014.
Prepare Instrument
Administer Survey
Manage Survey
Analyze Survey
Report Results
S0 General Information S1 Pre-Design S2 Schematic Design S3 Design Development S4 Construction Development S5 Bid/Negotiation S6 Construction Administration S7 Post Occupancy
Set Objectives
Design Study
Staffing needs. One graduate student at 25 hours a week. Additional student to co-conduct interviews at 5-10 hours a week. Tasks. Populate database with Survey 2 feedback. Initiate analysis of Survey 2. Administer Survey 3 in some instances. General Time Frame. Summer 2014.
Prepare Instrument
Administer Survey
Manage Survey
Analyze Survey
Report Results
S0 General Information S1 Pre-Design S2 Schematic Design S3 Design Development S4 Construction Development S5 Bid/Negotiation S6 Construction Administration S7 Post Occupancy
Set Objectives
Design Study
Staffing needs. One graduate student at 25 hours a week. Additional student to co-conduct interviews at 5-10 hours a week. Tasks. Populate database with Survey 3 feedback. Initiate analysis of Survey 3. Administer Survey 4 in some instances. General Time Frame. Summer /Fall 2014.
Prepare Instrument
Administer Survey
Manage Survey
Analyze Survey
Report Results
S0 General Information S1 Pre-Design S2 Schematic Design S3 Design Development S4 Construction Development S5 Bid/Negotiation S6 Construction Administration S7 Post Occupancy
Set Objectives
Design Study
Staffing needs. One graduate student at 25 hours a week. Additional student to co-conduct interviews at 5-10 hours a week. Tasks. Populate database with Survey 4 feedback. Initiate analysis of Survey 4. Administer Survey 5 in some instances. General Time Frame. Fall 2014.
Prepare Instrument
Administer Survey
Manage Survey
Analyze Survey
Report Results
S0 General Information S1 Pre-Design S2 Schematic Design S3 Design Development S4 Construction Development S5 Bid/Negotiation S6 Construction Administration S7 Post Occupancy
Set Objectives
Design Study
Staffing needs. One graduate student at 25 hours a week. Additional student to co-conduct interviews at 5-10 hours a week. Tasks. Populate database with Survey 5 feedback. Initiate analysis of Survey 5. Administer Survey 6 in some instances. General Time Frame. Spring 2015.
Prepare Instrument
Administer Survey
Manage Survey
Analyze Survey
Report Results
S0 General Information S1 Pre-Design S2 Schematic Design S3 Design Development S4 Construction Development S5 Bid/Negotiation S6 Construction Administration S7 Post Occupancy
Set Objectives
Design Study
Staffing needs. One graduate student at 25 hours a week. Additional student to co-conduct interviews at 5-10 hours a week. Tasks. Populate database with Survey 6 feedback. Initiate analysis of Survey 6. Administer Survey 7 in some instances. General Time Frame. Summer/Fall 2015.
Prepare Instrument
Administer Survey
Manage Survey
Analyze Survey
Report Results
S0 General Information S1 Pre-Design
Staffing needs. One graduate student at 25 hours a week.
S2 Schematic Design
Tasks. Follow up on trailing projects as necessary. Comprehensive analysis and reporting.
S3 Design Development S4 Construction Development S5 Bid/Negotiation S6 Construction Administration S7 Post Occupancy
Set Objectives
Design Study
General Time Frame. Spring 2016.
Prepare Instrument
Administer Survey
Manage Survey
Analyze Survey
Report Results
Nicollet Mall
Group Wall
Fifth Street
Working Wall
Live Lab
Eliminate
DLR to host Live Lab space
Conclusion This research project by the very nature of its length will accommodate several research internships. Each intern will add something valuable to the on-going research - helping to create rich results where the sum of their parts are truly greater then the whole. As it stands, Amy Ennen will remain involved and co-conduct interviews. She will be one of the constants in the research. The initial steps taken to establish this frame work were intimidating. The creation of a live lab meant that all participants would need to trust the process and have patience for the results. It could not have happened without the collaboration between contractor, architect and academia.