A Social Platform for Collaborative Open Design by Thomas VB

© Copyright by K.U.Leuven Without written permission of the promotors and the authors it is forbidden to reproduce or adapt in any form or by any means any part of this publication. Requests for obtaining the right to reproduce or utilize parts of this bpublication should be addressed to K.U.Leuven, Faculty of Engineering – Kasteelpark Arenberg 1, B-3001 Heverlee (België). Telefoon +32-16-32 13 50 & Fax. +32-16-32 19 88. A written permission of the promotor is also required to use the methods, products, schematics and programs described in this work for industrial or commercial use, and for submitting this publication in scientific contests.

table of contents Table of contents Abstract 1. Introduction 1.1 Open strategies & social production: a new approach for architectural and urban design? 1.2 Challenges of open collaboration in architectural and urban design 1.2.1 Architecture does not allow for trial and error 1.2.2 The issue of chunking 1.2.3 Should every stage of design be open? 1.3 Motivation 1.4 Objectives 1.5 Hypotheses 1.6 Significance

p.05 p.09 p.11 p.11

2. Background 2.1 Prior research on open collaborative production of architecture and urbanism 2.1.1 Phase(X) 2.1.2 Studio Wikitecture 2.1.3 OpenArchitectureNetwork.org 2.1.4 OpenIdeo.com 2.1.5 OpenSimSim.net & Openingdesign.com 2.2 Prior research on the impact of social visualisation 2.2.1 CodeSaw 2.2.2 Netscan 2.2.3 Influencing Group Participation with a shared Display 2.3 Conclusion

p.17 p.17

3. Implementation 3.1 Defining the concept 3.2 Usage scenarios 3.2.1 Closed Hierarchical 3.2.2 Closed Flat 3.2.3 Open Hierarchical

p.25 p.25 p.27 p.27 p.28 p.28

-5-

p.11 p.12 p.12 p.12 p.13 p.13 p.14 p.14

p.17 p.17 p.18 p.18 p.21 p.21 p.21 p.22 p.22 p.22

3.2.4 Open Flat 3.3 Archibrain.org: building a prototype 3.3.1 Pre-implementation: website and project-tree mock-up designs 3.3.2 Implementing the website with Drupal & Processing 3.3.2.1 Drupal Background 3.3.2.2 Creating the prototypeâ&#x20AC;&#x2122;s basic functionality 3.3.2.3 Editing the layout 3.3.2.4 Defining user roles 3.3.2.5 Website layout and functionality changes during implementation 3.3.2.6 Project-tree layout and functionality changes during implementation 3.3.3 Class diagram of final project-tree 3.4 An overview of the final prototype

p.28 p.29 p.29

4. Evaluation study 4.1 Methodology 4.2 Known restrictions of the evaluation 4.3 Pilot-test 4.3.1 Encountered issues during the pilot-test 4.3.2 Adjustments to the evaluation method

p.71 p.71 p.73 p.74 p.74 p.74

5. Results & Discussion 5.1 Report on evaluation proceedings 5.1.1 Participants: background 5.1.2 Overview of the design process 5.1.2.1 Analysis visualisations 5.1.2.2 Session 1 5.1.2.3 Session 2 5.1.2.4 Session 3 5.2 Hypotheses: results & discussion 5.2.1 Impact on Motivation 5.2.1.1 Ideology 5.2.1.2 Career 5.2.1.3 Challenge 5.2.1.4 Social 5.2.1.5 Fun 5.2.1.6 Reward

p.79 p.79 p.79 p.80 p.81 p.82 p.86 p.90 p.92 p.92 p.92 p.92 p.93 p.94 p.94 p.94

-6-

p.37 p.37 p.38 p.39 p.39 p.39 p.49 p.63 p.68

5.2.1.7 Recognition 5.2.1.8 Duty 5.2.2 Impact on the design process 5.2.2.1 “The project-tree forms a correct represent- tion of the design process, and helps users keep an overview of the design process“ 5.2.2.2 “Rather than dealing with one design prop- sal at a time, the community takes the liberty to work on different possibilities at the same time” 5.2.3 Impact on social dynamics of collaborative design 5.2.3.1 “Our platform provides effective means for a design-community to give each other feed back, and visualises this feedback effectively in the project-tree to highlight important design proposals.” 5.2.3.2 “Users feel comfortable editing other designs and having their designs edited” 5.2.3.3 “The project-tree gives users a good impression of the role they and other users played in the design process” 5.3 Efficacy of the evaluation

p.95 p.96 p.96 p.96

6. Conclusion

p.105

Appendix I: Designstudio Arenberg: Tutorial-tree & Design Brief Appendix II: Survey questions & focus-group topics II.a Full overview II.b Reliability of surveys Appendix III: Survey results Appendix IV: Focus group transcription

p.109 p.117 p.117 p.120 p.121 p.133

Acknowledgments References Bibliography Master’s Thesis File

p.141 p.143 p.145 p.147

-7-

p.97

p.97 p.97

p.101 p.101

p.103

-8-

abstract In recent years, new trends like social media, crowd sourcing and open source strategies have been used to accumulate knowledge and creativity from the bottom up. Various fields have created easily accessible web-platforms to allow anyone to connect, share and collaborate. One field that could greatly take advantage of this is architectural and urban design, which in all its complexity and versatility could flourish in an open and collaborative design environment. We approached this topic in a practical way by creating an online platform where a network of people can collaborate asynchronously on a given design assignment. This platform allows people to share design proposals and their corresponding CAD-files in an open manner, allowing the network to give feedback trough comments and ratings and giving them the ability to adapt and evolve each others proposals in an open source manner. The main feature of the platform is the project-tree a visual way of browsing through the different design proposals. This visualization organizes the proposals in a diagram that illustrates the relations between them, and visualises community feedback like the number of comments and the rating a proposal has received. By doing this it gives users a clear overview of the design process as a whole, and helps them decide which proposals are relevant to work on. We tested the efficacy of this platform by conducting a short design assignment with a group of students using the website as the main medium of communication, and tested the results of this evaluation to a set of hypotheses concerning the platformâ&#x20AC;&#x2122;s impact on user motivation, the design process and the social dynamics between users. This experiment aims to enforce the feasibility and relevance of a bottom-up approach to architectural design; opening up architecture to the ideas, creativity and experiences of the many.

-9-

- 10 -

introduction

The network effects of the digital age have made possible a whole new platform for sharing and producing information, knowledge and culture, accessible to anyone with a computer and a basic Internet connection. Yochai Benkler named this new mode of collaboration “Social Production’:

“(...) the networked environment makes possible a new modality of organizing production: radically decentralized, collaborative, and non-proprietary; based on sharing resources and outputs among widely distributed, loosely connected individuals who cooperate with each other without relying on either market signals or managerial commands.” (Benkler, 2006) New phenomena like social media, blogging, crowd sourcing and open source software have shown that it is now easier than ever for an individual or a group of individuals to connect and collaborate with others to produce meaningful content from the bottom up. Their accessibility have helped make many passive Internet users become active participants in a variety of fields. Various fields have already created platforms where users can contribute their knowledge and collaborate to create something that is greater than the sum of their individual contributions. These communities often show a form of collective intelligence: a “large number of people contributing to a single project exhibiting intelligent behaviour.” (Maher et al. 2010) A well known example of this is Wikipedia, an online encylcopedia created by the individual contributions of a vast amount of volunteers, which now comes close to Encyclopedia Britannica in therms of accuracy. (Giles 2005) Notable is that many content contributors are often spatially distributed volunteers, people acting out of social and psychological motivations, who are simply interested enough in the subject matter to feel compelled to contribute their time and knowledge to it. In what ways could the field of architectural and urban design delve into the knowledge and creative power of online communities and benefit from these new forms of participation and collaboration?

1.1 Open strategies & social production: a new approach for architectural and urban design? Contemporary architectural and urban design problems often pose complex multidisciplinary challenges that need to be solved by a groups of people with a variety of backgrounds. The built solutions to these problems can have a profound effect on the built environment and its inhabitants. The challenge architects and planners often face is to address the varied, sometimes conflicting needs of the different user groups with a homogeneous design that solves their needs as well as possible. Should finding answers to such complex problems be limited to a private team of experts if their solution will have a remaining impact on our urban fabric for many years to come? What if this complex problem could be opened up to a vast amount of architects, designers, sociologists, engineers, citizens, and others who want to contribute knowledge and ideas, simply motivated by a desire to improve their built environment? What if these design problems did not have to come from a local government or a client who wants to make a profit, but were defined by locals and developed by architects who want to help them improve their neighbourhood? In other words: can social production and open strategies form a viable approach for architectural and urban design?

1.2 Challenges of open collaboration in architectural and urban design Maher et al. (2010) state that “collective design is possible because we can easily communicate and share ideas, digital models and files on the Internet”. Indeed, during the last two decades, the architectural design and planning process has become almost entirely digitalised, making distributed collaboration easier through sharing CAADfiles over the Internet.

- 11 -

Maher et al. (2010) further state that a conceptual space for collective design should be built around three considerations: • Communication How do people communicate during the design process? (e-mail, blog, etc...) • Representation What part of the design is developed between collaborators (design problems, solutions, …) and how is the collaborative design represented? (sketches, 3D models, …) • Motivation In what way are user motivated to contribute on the platform, and how is their participation structured? To apply these considerations surrounding collective design on the architectural and urban design process we need to consider certain restrictions inherent to these types of problems.

1.2.1 Architecture does not allow for trial and error “If a writer makes a bad book people don’t read it. But if you make bad architecture, you impose ugliness on a place for a hundred years.” (Renzo Piano, 2011) The first thing to consider is that sharing and collaborating on architectural and urban design problems via a web platform is limited to digital representations (2D drawings, renderings) and virtual models of the design (which can be shared with many) and not the actual ‘built’ design. This means that an important feedback loop based on the design’s real world performance is missing: quick iterations of a design cannot be tested in real life to test its functional efficacy. Feedback is restricted to the subjective opinions of contributors who base themselves on personal knowledge and previous experience. This is for example different with open source software: when contributors test versions of a piece of software, they are testing its actual functionality rather than just speculating whether a piece of code would provide the desired functionality or not. Additionally, when a first version of a program is released, it can still be improved based on userfeedback. This is not the case in architectural and

urban design: once built, fundamental changes to it are often hard or expensive to make. What this issue implies is that the final design needs to be as good as it can be: all critical design innovations necessarily need to happen during the development of the design, as there is no possibility for trial and error once a design is realised. An open approach to architectural an urban design needs to embrace that design-community feedback concerning a design’s performance will be based on subjective experience, and therefore needs a good way of organising the various opinions in discussion during the design process so they can form the basis of socially responsible design decisions that reflect the design community’s wishes. In other words, the communication dimension needs to take a central role in the design process.

1.2.2 The issue of chunking The second thing to consider is the issue of ‘chunking’ (Benkler, 2006). In the case of open source software, functionality can often be ‘chunked’ into separately solvable little pieces that can be recombined at a later stage. Solving a single piece usually does not require complete knowledge of the overall project, which makes it easier to find motivated volunteers to solve them. Chunking is less evident in the case of architecture and urban design, since the various aspects of a design are often ill-defined and have a tendency to be highly dependent on each other. Though designers may focus on more detailed aspects from time to time, the consistency with the overarching concept of a design always needs to be considered. In other words, contributors will always need a good overview of the design process to know where their contributions fit in with the big picture. For this, the representation dimension should go beyond the mere representation of individual design proposals, and make an effort to clearly representing the design process as a whole and the place individual design proposals take in it.

1.2.3 Should every stage of design be open? The issue of consistency also introduces the question whether every stage of the design pro-

- 12 -

cess should be completely open: during the early conceptual stages of the design an open strategy can be an great way to acquire many different ideas and perspectives on the given design problem. During late stages of development, however, a completely open strategy could make it hard to keep the necessary level of consistency in the design. Participation on a platform for collaborative architectural design should provide a certain flexibility of how open it can be, ranging from a completely open form of participation to a closed collaboration between a few participants.

• The platform needs to provide a means of managing how users can participate, since not every design stage would benefit from a completely open approach.

1.4 Objectives The aim of this research is to define a concept for a social(4) platform(1) for collaborative(2) open(3) design. By this we mean the following: 1. Platform for design We aim to make a platform where people can come into contact with each other to discuss, share and collaborate on architectural and urban design problems. The goal is not to share finished designs, but rather the design process itself. This platform will be implemented via a website. Using the web as medium allows for a lower barrier of entry to the platform, making it accessible anywhere to anyone who has access to the Internet.

1.3 Motivation Today’s multidisciplinary and complex architectural and urban design problems could benefit enormously from new open forms of distributed collaboration and participation via the Internet, since it has the potential to bring in a wealth of knowledge and creativity to the design process and helps include the needs and opinions of different user groups.

2. Collaborative Sharing the design process online can open the creative process of architecture and urbanism up to a whole new level of distributed collaboration and participation.

Though there has been previous research and discussion about the possibilities of open collaboration in architectural and urban design, the creation of an effective online platform that lets people start up their own architecture and urban design projects and that helps them organise distributed collaboration with others to develop these projects is still in its infancy. Some currently active platforms fall short of our expectations concerning their functionality (cfr. ch. 2, p.17), while other platforms which might come closer to what we expect are, at the time of writing, still in development. In parallel with these current developments we will present our own conceptual platform for open collaborative architectural and urban design, addressing possible solutions for the complexities inherent to the design process by focusing on the following elements:

3. Open This not only refers to the ability for anyone to participate on the platform, but also to freedom for anyone to decide for themselves what problems are relevant to work on. Anyone can define their own design assignments and choose who can or cannot collaborate on them. 4. Social The platform will help users keep an overview over the complex creative interactions that occur between designers within the collective design process. This will be done trough a visualisation which we call the project-tree that displays the various design proposals, the relation between them and the feedback they receive from the design community. This visualisation will be used to navigate through the various design proposals and forms the central interface of the platform.

• The platform needs to effectively organise discussions so a variety of diverse opinions can be taken into account during the design process. • The platform should give a clear overview of the collective design process so individual collaborators can keep the overarching design in mind when making contributions to the design.

We aim to evaluate the efficacy of this concept, and especially the role of the project-tree, by partially implementing it into a prototype website on

- 13 -

which we will organise a collaborative design. The results of this evaluation will be tested to a set of hypotheses about the desired impact of the platform on user motivation, the design process and the social dynamics of collaborative design.

1.5 Hypotheses

b. Users feel comfortable with editing other designs and having their designs edited.

1. Impact on user motivation Our hypotheses about the impact of our platform on user motivation are based on 8 categories of motivation, as defined in previous research by Maher et al. (2010). e. Ideology: participating on this platform gives users the feeling they are contributing to a larger cause f. Career: participating in a collective design provides experience that can help a user advance his career g. Challenge: participation provides a sense of personal achievement through acquiring additional knowledge or skill. h. Social: the platform provides a shared experience that people appreciate i. Fun: the platform provides a pass-time where people participate for their own enjoyment j. Reward: by displaying the contributions and their importance to the design process, the visualisation can be visually rewarding for participants

c. The project-tree gives users a good impression of the role they and other users play in the design process.

1.6 Significance The results of our evaluation will provide an insight into the efficacy of our proposed strategies for dealing with the complexities inherent to distributed collaborative architectural and urban design. The results concerning our project-tree visualisation will provide insights on how the collective design process can be effectively summarised visually, and how the visualisation of contributor feedback can be incorporated into such an overview. We will also learn what impact such a visualisation can have on user motivation and the collective design process. Current and future web platforms for open architectural and urban design collaboration can use these results

k. Recognition: participants feel motivated because of the feedback they get for their contributions l. Duty: participants feel a personal desire to contribute 2. Impact on the design process a. The project-tree forms a correct representation of the collective design process, and helps users keep an overview of the design process. b. Rather than dealing with one design proposal at a time, the community takes the liberty to work on different possibilities at the same time

- 14 -

- 15 -

fig. 2.1: TOP: An overview of the projects made during the first four phases of phase(x). The lines connect projects from an earlier phase with their adaptations. fig. 2.2: MIDDLE:The wiki-tree inside Second Life fig. 2.3: LEFT: Each â&#x20AC;&#x2DC;leafâ&#x20AC;&#x2122; represents a design submission, which can be viewed on the viewing parcel, or edited and resubmitted as a new leaf on the tree.

- 16 -

BACKGROUND

Our background study consists of two parts. We first give an overview of previous research on open collaborative strategies for architectural and urban design, including an overview of web platforms that are currently in use or being developed for the same goal. We will evaluate their approach to organising collaborative design and compare them to the goals we hope to achieve with our own platform. Secondly a brief literature review on social visualisation and the way it influences collaboration will be given, and will be discussed as background for the impact of our own projecttree visualisation.

2.1 Prior research on open collaborative production of architecture and urbanism 2.1.1 Phase(X) (Hirschberg, 2001)

2.1.2 Studio Wikitecture (Chase et al., 2008)

Phase(X) was a course in computer aided design taught at ETH Zurich in 1996, 1997 and 1999, and is one of the first experiments with open distributed collaborative design. Their approach was to have the design organised in phases: “after every phase the works of all authors are stored in a common database and become visible to all participants. In the subsequent phase, each author has to select the design of one of their colleagues to develop it further. The same design can be selected by many different authors, but they are not allowed to continue working on their own designs.” (Hirschberg, 2001)

Studio Wikitecture is an experiment concerning distributed collaborative design via virtual worlds like Second Life. All designs are created and shared within the second-life platform. For this an interface was developed within Second Life with the ‘wiki-tree’ as its main feature. This wiki-tree has a canopy of colored spheres, each representing a different design submission. This canopy gives an overview of the evolution of the designs by linking the proposals that have built upon each other. The colors of the spheres indicate the popularity of the design proposals, which is based on a thumbs up/down voting system where each participant can cast a total of three positive and three negative votes. Two parcels in Second Life allowed participants to view the designs stored in the wiki-tree, and build new designs by editing previous design proposals.

This approach to collaboration bases itself on the principle of meme-theory (Dawkins, 1976): ideas (in this case designs) are like genes, and survive by replicating themselves. The ‘fittest’ design is developed the most and thus survives in the next phase. The most interesting aspect of this is that phase(X) forms an objective self-rating system, where “...the success of any work in the system depends on how many offspring it gets and how successful those are in turn.” (Hirschberg, 2001)

The wiki-tree concept gives an interesting insight in how the various design proposals made in a collaborative design process and their respective popularity can be visualised in a way to give participants a clear overview of the design process as a whole.

- 17 -

2.1.3 OpenArchitectureNetwork.org The Open Architecture Network, a website developed by Architecture for Humanity1 and launched in 2007, describes itself as “... an online, open source community dedicated to improving living conditions through innovative and sustainable design.” Their platform allows users to share detailed information on their projects together with their construction plans. By default the project’s users uploads are ‘public’, allowing anyone to view, review and adapt it. This also means that once a design is uploaded, anyone can use the plans for free to build it. A project-team can however also choose to keep their content private. The website also hosts design competitions as a way of crowd-sourcing innovative design solutions to improve living standards in the developing world. Though the website does allow users to openly share designs, it does not seem to have any builtin mechanics to organise collaboration between users. We used the websites contact-form to ask how their platform organises collaboration and interaction between users and if they focused mainly on the competitions as a way of generating projects. Mike McCaffrey, a member of Architecture for Humanity’s web team responded:

“Right now the collaboration on the site is mostly between members of a project team. There are also the competitions, where the organizers and judges interact with the entry projects. We are exploring other methods of interaction in the future.”

users upload projects and references that might help solve the problem at hand. During the ‘concepting’ phase, users can post their solutions or adapt each others solutions. In the ‘evaluation’ phase, the community rates and comments on the concepts they think solve the problem the best The highest rated concepts win and become available for development by the challenge sponsor. The design phases are not absolute, and new phases can be introduced, for example a ‘refinement’ phase that allows preliminary winners to further develop their concepts before the final evaluation phase. Another novel element of this website is the ‘design quotient’, a visualisation that displays how much a user participated in each phase and collaborated with others.This visualisation serves as a way of providing users with a form of feedback and recognition concerning their contributions. Which challenges are tackled on the website is moderated by the platform administrators. At the time of writing the platform is only supporting a handfull of challenges at the same time. Of interest to us is also the collaboration app: an interactive visualisation that connects together inspirations and concepts that have informed and built upon each other. Though it does allow for a novel way of browsing trough various inspirations and concepts, it does not yet give a clear overview of the creative process as a whole. The challenges on the platform focus mainly on social issues, rather than architectural or urban design problems. Nonetheless, it forms a successful example of a platform that manages to coordinate creative collaboration throughout every step of the design process.

Up to this point, more than 30.000 members have uploaded over 6000 projects, of which 85 have been realised. Though this website does not introduce novel ways of organising collaboration between designers, it does prove that an online design community can create valuable contributions that have a real-world impact.

2.1.4 OpenIdeo.com OpenIdeo.com, launched in 2010, is an online community that tackles design challenges by dividing the problem-solving process into phases: all projects start with an ‘inspiration’ phase, where 1 Architecture for humanity is a non-profit design services firm dedicated to building a more sustainable future through the power of professional design.

fig. 2.6: The design quotiënt displays how much a user participated in each design phase and collaborated with others.

- 18 -

fig. 2.4: Home page of OpenArchitectureNetwork.org

fig. 2.5: Home page of OpenIdeo.com

- 19 -

fig. 2.7: A ‘challenge’ page on OpenIdeo.com. Challenges are devided into different design phases: the inspiration phase where people share inspiring references, the concepting phase where they post possible solutions and the evaluation phase where a winner is chosen by the design community.

fig. 2.8: OpenIdeo’s ‘collaboration map’ is an interactive visualisation that connects together inspirations and concepts that have informed and built upon each other.

- 20 -

2.1.5 OpenSimSim.net & Openingdesign.com These two websites both aim to introduce web platforms that allow for a more open and collaborative approach to architectural design. However, at the time of writing, these two platforms are still in development and have little information available on how they plan to organise open design collaboration on their websites. Though we contacted both websites via available contact-forms to try and find out more about their approach, the feedback we received was limited. Both websites do have a short video explaining their intentions. OpenSimSim.net suggests to devide the design process into an idea phase (cfr. openideo), a project phase and prototype phase. During each phase, the development process is opened up to certain user groups (designers, engineers, web 2.0 communities, final user groups, etc.). The final product is open for anyone to build (cfr. openarchitecturenetwork.org). OpeningDesign.com aims to create a network where architects, engineers and designers can share design knowledge, exchange ideas and give feedback on each others designs. They

aim to create a set of tools that aid collaboration between members. What these tools do and how they will work will be revealed during later stages of development.

2.2 Prior research on the impact of social visualisation 2.2.1 CodeSaw (Gilbert et al., 2008) Codesaw is a social visualisation of open source software development that displays both the code-contributions and the project communication (e-mail) on a timeline and colors them by author. With it you can see in what manner and frequency someone contributed work tot an open source software project. The aim of this visualisation is to visualise the group dynamics of open source software developers that would otherwise be buried in textual databases. The evaluation study of the visualisation revealed that it held the potential to motivate contributors, as it visually showed their and other peopleâ&#x20AC;&#x2122;s impact on the overall software development process.

fig. 2.9: The CodeSaw visualisation. Code contributions are displayed above the timeline and project communication are displayed below. Each contribution is colored by author and represented on a seperate timeline below. Two or more authors can be compared by dragging them into the â&#x20AC;&#x2DC;investigation areaâ&#x20AC;&#x2122; at the top.

- 21 -

2.2.2 Netscan (Smith et al., 2000) Netscan is a set of tools that visualise the structure of threaded conversations and the patterns of participation within these conversations. With this the authors hope to bring more insight into activity, structure, interconnection and content of a thread user group, and at the same time reinforce socially beneficial behaviour. The elements that make up this tool are shown and explained in fig. 2.10. The main element that interests us is their threadtree visualisation, which gives a clear overview of the reply patterns in a certain threaded conversation. Each new reply is illustrated by a box in the tree, which is drawn chronologically from top to bottom. Multiple replies on the same post result in multiple branches. Clicking on a box displays the reply in a message display box. The tree also visualises the most prolific posters (half-shaded boxes) and highlights a currently selected authorâ&#x20AC;&#x2122;s contributions (red boxes). Although the subject matter being visualised is vastly different from architectural design, this method of visualising linked content to create overview and reveal structure and social dynamics serves as an important inspiration for our own project-tree visualisation.

2.2.3 Influencing Group Participation with a shared Display (DiMicco, 2004) This paper addresses whether a shared display with information on user performance can influence the behaviour of a group during a collaborative task. The author evaluates this by organising group discussions with and without a shared display. This display visualised how long each participant had talked (fig 2.11) and indicated if someone was over-participating or under-participating. The results of their evaluation show that the visualisation successfully influenced participant behaviour in the case of over or under participation. Relevant for us is the indication that a shared display of a collaborative effort can help influence participant motivation.

2.3 Conclusion Concerning current platforms for open collaborative design, both OpenIdeo and the Open Architecture Network fall short concerning the expectations we have for such a platform. OpenArchitectureNetwork.org is currently focusing mainly on crowd-sourcing strategies, which often encourage designers to submit finished designs rather than collaboration with others during the design phase. OpenIdeo.com introduces a structured and easy-to-learn way of collaborating with others during the design process, but in our opinion still lacks a clear overview of the design process as a whole. Both platforms encourage users to contribute creativity and ideas to projects selected by the platform administrators, but seem to lack the means for users to put forward and organise design problems themselves. Our background study concerning social visualisation shows the impact a visualisation of the collaborative process can have on user motivation, a userâ&#x20AC;&#x2122;s overview of the design process, and a userâ&#x20AC;&#x2122;s insight into the social dynamics between collaborators. This solidifies our belief that our own project-tree visualisation can have a positive effect on the collaborative design process.

fig. 2.11: A shared display visualised how long each participant had talked and indicated if someone was over-participating or under-participating.

The tree-structure and visualisation of authorship and feedback in the wikitecture and Netscan trees served as an important source of inspiration for our own project-tree visualisation.

- 22 -

fig. 2.10: The Netscan dashboard

- 23 -

- 24 -

Implementation

In the first part of this chapter we describe our conceptual online platform for collaborative open design. In the second part we detail the development of the partial implementation of this concept in a prototype website, which is then used to evaluate the concept. (cfr. ch.4 p.71)

3.1 Defining the concept The main purpose of our conceptual ‘social platform for collaborative open design’ is to provide a web platform where anyone can start an architectural or urban design project, and can then collaborate with others to find design solutions. The platform will allow this collaboration to happen in a distributed and asynchronous manner, and focuses on organising collaboration and the social dynamics around it. The main elements the website provides to users for this are ‘proposals’ and ‘project-trees’. • Proposals are like short blogposts users make to share their designs and ideas. They contain some images of the design proposal, some text explaining it, and possibly the CAD-files that were made for it. Each proposal can receive a rating by members of the design community, and has a comments section where the community can discuss what they like or dislike about the proposal. • The project-tree provides users with an organised overview of all proposals that were made for a certain project, and acts as the main interface to browse trough them. Since design via this platform is mainly asynchronous, having a clear overview of new design proposals that have been made is vital. The tree also visualises community feedback like the number of comments and the rating a proposal has received. By doing this it aims to help users decide which proposals are relevant to work on; an important tool when the tree starts to grow large. When someone decides to create a new projecttree, he becomes the tree’s ‘administrator’. He will first have to make the project-tree’s assignment,

describing the project-brief and supplying designers with necessary information and some basic CAD-files (e.g. of the project-site). The administrator can now assign roles to users who want to work on his project. Roles limit the possible interactions a user can have with the project-tree, and allow the admin to control how ‘open’ the collaboration is: he can limit the actual design team to a select group, or open it up to anyone who wants to contribute. By default there are tree roles an admin can assign to its users: contributor, commenter and viewer (table. 3.1). The admin can also create new roles or change the permissions for a given role, thus giving him complete control over who and how people can interact with his tree. When the assignment is made and the roles are defined, contributors can start by downloading the basic CAD-files from the assignment and posting a first set of ideas and proposals. Commenters and contributors can give these proposals a rating and post comments to discuss what they like or don’t like about the suggested ideas. Contributors can download the CAD-files from each other’s proposals, edit them and re-upload

table 3.1: The three standard user-roles an admin can assign to his users. Roles limit the possible interaction a user can have with the project-tree, thus giving the admin a way to control how ‘open’ the collaboration is.

- 25 -

fig 3.1: When a user creates a project-tree he defines the project â&#x20AC;&#x2DC;assignmentâ&#x20AC;&#x2122; and chooses who can collaborate in which way by assigning roles to other users. (top) Contributors will make a first set of design proposals on which the design community can give feedback via comments and rating. (middle) Contributors can edit each otherâ&#x20AC;&#x2122;s proposals. The resulting project-tree is a diagram that shows how the collective design evolves over time. (bottom)

- 26 -

them as new proposals that expand, rethink or evolve the original proposal. The resulting projecttree is a diagram that shows how the collective design evolves over time, mutated by the ideas and discussions of the online community. We believe that the concept described above can provide a platform with a low barrier of entry where anyone can collaborate with others on the architectural and urban design problems that they themselves find relevant. By giving the users full control over who can collaborate and how, the platform can be used in the way that best fits a certain design or design-stage; be it a brain-storming session that is open to anyone, or a closed collaboration between geographically distributed architects. The project-tree visualisation is of central importance to this concept, as it helps keep an overview over the complex mechanics of distributed collaborative design and community feedback.

3.2 Usage scenarios The inclusion of user roles means that collaboration can be organised in a variety of ways, making the platform usable in different types of design scenarios. It is our opinion that having control over how ‘open’ a design is is vital to the design process, since not every design (nor every design-stage) would benefit from a completely open approach. For example, in the early conceptual stages of a design one would benefit from opening the problem up to as many collaborators as possible, to generate and explore a vast and diverse set of possible solutions. However, during the final stages of a design a large group of collaborators could hinder efficient progress and consistency. Here it would make more sense to have a few architects as ‘collaborators’ who detail the design, whilst engineers and other consultants would only be ‘commenters’ or ‘viewers’. Designers need to have a good knowledge of what type of collaboration fits what stage of a design process best, and this platform aims to give them the necessary design community-control to manage this. To illustrate the various ways in which people can collaborate via this platform we will describe four example usage scenarios based on the four modes of collaboration as described by Pisano, et al. (2008) in an article in Harvard Business Review. According to them, collaboration is defined by being either open or closed and flat or hierarchi-

cal. Their table illustrates this (table 3.2). In what follows, we apply these collaboration modes to the specific environment of our collaboration tool.

3.2.1 Closed Hierarchical An architecture office, working internally on a design, could post important steps of the design process on the platform as a way to communicate the design process to third parties. They maintain complete control over the decisions made during the design process, and choose themselves what steps to communicate via the platform. This method allows them to involve and receive feedback from clients, engineers, contractors and others during certain phases of the design process, even if these third parties are geographically distributed. Additionally, the feedback can happen in an asynchronous manner. Roles: contributors: a select group of designers and engineers from the office decide what problems to work on and manage all design decisions themselves. commenters: clients, engineers, contractors, consultants. Each party receives this role during different phases of the design process. viewers: users the office wants to commu nicate the design to, without re quiring their feedback. eg: other architects, the clients’ investors, local government, ...

table 3.2: The four modes of collaboration (Pisano, et al., 2008)

- 27 -

3.2.2 Closed Flat The platform could be used in much the same way to simplify distributed collaboration between offices. An example scenario for this would be a collaboration between offices for a design competition, in which both offices make choices on how to tackle the design brief. An added benefit here is that collaboration can happen asynchronously, which can be of benefit if both offices are located in different time zones. The project-tree will help them keep an overview of each others design proposals even though they work on it during different periods of time. Roles: contributors: commenters: viewers:

both offices make design proposals and decide between each other which ideas are the best much like the previous example, consultants, engineers and others can be called in during certain stages of the design to provide feedback and input private to everyone, at least until the design is submitted

3.2.3 Open Hierarchical An open hierarchical approach to collaboration usually corresponds to a form of crowd sourcing for solutions, like an open architectural competition. Unless architects collaborate on a competition entry like in our last example, this category often does not encourage collaboration and open sharing of ideas between architects, and is thus not suited for our platform. We can however imagine a usage scenario where an architect or urban designer would open his design process up to the input and feedback of other professionals, local residents and others, whilst still having full control over the decisions made during the design process. For example, urban design projects in Flanders often require some form of resident participation. Urban designers could post part of their design process on the platform, giving motivated residents and external professionals the chance to discuss the viability of their designs. The platform can provide the designers with a way of working closer together with residents to incorporate their feedback, and in return also gives locals a better overview of the design rationale and the decisions that needed to be

made during the design process. When the urban designers have come to an acceptable preliminary design, they will most likely benefit from changing from an open system of collaboration to a closed one (see previous two examples), so they can work more efficiently to complete the construction plans of the design. They can simply change the permissions to a more closed configuration, or create a new tree all together in which they complete the design. Roles: contributors:

the urban designers. Even though feedback can be given by any one, they still control the main dedcisions in the design process. commenters: anyone, especially inviting local residents and external professio nals to give feedback and dis cuss the design process viewers: anyone

3.2.4 Open Flat In an open flat design, design proposals and design decisions are no longer controlled by a small group of designers, but can be made by anyone. Our concept can thus provide a platform where a community can take the decision of what design problems need to be tackled into their own hands. Instead of waiting for someone else to find a solution for their problem, they can use the platform to brainstorm for possible design solutions, free from economic or governmental restraints. Of course, when one wants to make the transition from concept to reality, regulatory and economic constraints have to be taken into account to get a project realised. Nonetheless, the platform can provide a place where important design problems facing a community can come to light, and it will give them a first place to brainstorm for possible solutions. If enough people can get behind such a project, and if the created design can be proven to be a viable solution to the proposed problem, the chances of it crossing over from concept to reality can surely increase. Consider the following scenario as an example: a group of architecture students finds that they have too little space to collaborate on design studios, and take it upon themselves to design a pavilion that can provide them with extra workspace. Via

- 28 -

our platform they can open the initial design up to all students and teachers who wish to contribute concepts and ideas. They could start with a completely open project-tree to explore a variety of ideas, and then later shift to a new, more controlled project-tree to focus on developing the best designs that came out of the first tree. If the designed concept(s) are realistic enough, and students can prove the general demand and benefit of their solution(s), they can present it to the university to see what can be done to realise the project. (this scenario was partially emulated for the evaluation of this conceptual platform cfr. ch.4, p.71) Roles: contributors: anyone (students, teachers, â&#x20AC;Ś ) commenters: anyone. At a later stage they might invite University staff to provide feedback on the viability of the design viewers: anyone

layout of website and the project-tree should look like. After this we registered the domain name Archibrain.org and implemented the website using the Drupal 6 content management system, and used a Java-applet running Processing code to implement the project-tree visualisation. In the following segment we will detail the different steps and decisions we took during development.

3.3.1 Pre-implementation: website and project-tree mock-up designs The first stage of development happened trough a series of mock-up images that explored concepts for the layout and functionality of our prototype platform. The main source of feedback and critique at this stage where consultations with the promotor and co-promotor of this thesis.

3.3 Archibrain.org: building a prototype Since the conceptual platform encourages users to decide for themselves what projects to work on and with whom to collaborate, the best way to evaluate the efficacy of our platform would have been to fully implement it and to let architects and designers use it freely. Time and know-how restrictions have however restricted us to implementing a prototype, on which the main elements of our concept can be tested. The main differences of this prototype with the concept explained in 3.1 (p.25) are listed below: 1. Users can not make their own project-trees Project-trees are created and managed by the website admin (author) only 2. Users can not assign roles to others and have no control over the role they receive The roles and subsequent editing permissions of website users were also managed by the website admin. The prototype was designed and implemented over the course of two academic semesters. Developments was done in two stages. First, a series of mock-up images were made to explore how the

- 29 -

fig 3.2: mock-up 1: The first concept for the website, showing the main elements of a design proposal page: the project-tree interface, the proposal (images and text), and the comments section.

- 30 -

mock-up 1 • Proposals would be better represented as thumbnails in the project tree.

The first mock-up was made as a proof-of-concept to explain the main elements and functionality of the proposed platform. As mentioned above, design proposals are made trough short blog posts simply called ‘proposals’. Fig.3.2 shows us the main elements of such a proposal-page:

• The popularity of a proposal (represented by the surface of its sphere in the project-tree) should not only be based on a proposals rating, but also take other factors into account, like the number of comments received or the number of offspring (edits of the proposal).

• The project-tree (top) Each proposal-page has the project-tree at the top, which is used to browse trough the proposals that have been made, represented in the tree by spheres. The tree starts with an assignment at the far left, and grows from there as people add new proposals and edit each others proposals. The coloured sphere shows you what proposal you are currently viewing. The surface area of the spheres reflects the proposal’s rating and helps to quickly recognise the popular designs. The spheres contain the automatically generated name of their proposal, which are built up by a sequence of letters that form an indication of the proposals history. If someone is the second person to edit proposal AC, their edited proposal will be named ACB. If someone is the first to edit ACB, their new proposal is named ACBA and so forth. The project-tree ‘grows’ from left to right as users edit each others proposals, giving a sense of progression.

• The layout needs to be remodelled taking the general dimensions of computer monitors into account. The main elements of a proposal-page should be visible without the need to scroll down. • The concept still lacks a design for a ‘frontpage’: the page you first see when you come to the website

• The proposal (middle) A proposal consists of the automatically generated title, a screenshot of the design-proposal, a short textual explanation, and the attached cad files. Other users can vote on the proposal by clicking thumbs up or down, and edit it by clicking the ‘edit’ button. This would take you to a new page (fig.3.3) where you could download the attached cad files, edit them, and upload them as a new proposal. • Comments section (bottom) Here other users can discuss what they like/ dislike about the proposal. This first concept needed to be improved on the following points: • Users should be able to give their proposals a title, rather then having a non-descriptive name generated for them.

fig 3.3: mock-up 1: To edit a proposal, you would be taken to a new page where you could download the attached CAD-files of the proposal, as well as upload the edited version as a new proposal. (top)

- 31 -

fig 3.4: mock-up 2: This second layout oriented the proposal and the project-tree vertically, so they could both be visible without having to scroll down. The dotted line marks what part can be viewed in one time on a conventional monitor. This mockup also show a concept for two stacked bar-charts: one indicate a projects relevance (based on rating, comments and number of offspring), the other compares this relevance to that of the proposals that came before it.

- 32 -

mock-up 2 • A front-page was designed (fig.3.5) which displays the project brief (at that time we assumed the prototype would only support one projecttree), a ‘wall’ that displayed the latest activity in the project-tree, and a ‘leader-board’ displaying the top commentators, top designers and top designs. The goal of the leader-board was to not only encourage good designers, but also active commenters.

changes made: • The proposal and the project-tree are oriented vertically so they are both visible without having to scroll down. • Each proposal has two bar-charts: one indicates a project’s relevance (a percentile score based on rating, comments and number of offspring). How this relevance is calculated is not yet defined. The second is a stacked bar-chart which compares the proposals relevance to that of the proposals that came before it.

things that need improving:

• Proposals are represented by thumbnails in the project-tree. • Scrolling over a proposal in the tree reveals more information on that proposal, allowing users to browse trough proposals without each time having to load the full proposal-page.

• The vertical layout differs too much from typical web-design conventions, and leaves little space for the proposal and the comments section • Finding an algorithm that can objectively convert the number of comments, rating and number of offspring of a proposal into a score that accurately represents the relevance that a proposal had in the design process is quickly subject to mistakes, resulting in a score that misrepresents the actual relevance of a proposal.

fig 3.5: mock-up 2: This first design for the front-page contains the project brief, a ‘wall’ that displayed the latest activity in the project-tree, and a ‘leader-board’ displaying the top commentators, top designers and top designs.

- 33 -

fig 3.6: TOP: mock-up 3: The third layout reverted back to a more horizontal design, this time making it compact enough so both the project-tree and the top of the proposal can be viewed without scrolling down. The attachments can now be downloaded from the proposal-page itself, and submitting an edited version of the proposal can now be done by clicking the â&#x20AC;&#x2DC;create offspring projectâ&#x20AC;&#x2122; button in the lower right corner. Also new are the authors of previous proposals, the 5-star rating and the creative commons license the proposal is licensed under. This license defines the legal boundaries by which others can adapt a design (in this case they need to attribute the original author ans share their adaptation under the same conditions). fig 3.7: RIGHT: mock-up 3: Instead of risking a miscalculation in determining a single score that represents a proposalsâ&#x20AC;&#x2122; relevance, the rating a proposal gets and the number of comments it receives are visually represented separately in the project tree by the color of the glow around the proposals sphere (rating) and its surface area (# comments). Scrolling over a proposal now also displays the recent comments a proposal received, which gives users browsing trough the project-tree a glimpse of the feedback a proposal is getting.

- 34 -

mock-up 3 changes made:

• The project-tree now visualises a proposal’s rating by a colored glow, ranging from green (high rating) to red (low rating). The number of comments are visualised by the surface area of the sphere that represents the proposal. Both a proposal’s rating and the number of received comments can be used to evaluate the importance of a proposal, but by visualising them separately, we avoid making mistakes that would most likely occur if we tried to calculate a single score to represent a proposals relevance.

• The proposal and the project-tree are oriented horizontally again, but are this time compact enough so both the project-tree and the top of the proposal can be viewed without scrolling down • attachments can be downloaded from the proposal-page itself • submitting an edited version of the proposal is now done by clicking the ‘create offspring project’ button, which takes you to a new page where you can submit your proposal. • all proposals are now clearly marked with a Creative Commons Attribution-ShareAlike licence; this licence implies that anyone can adapt the authors’ work, as long as they attribute the original author and publishes his/ her work under the same licence. All proposals made on the website will automatically fall under this licence.

• The website now has a black background to make the colored glows of the proposals more visible. • Scrolling over proposals now also displays the recent comments a proposal received, which gives users browsing trough the project-tree a glimpse of the feedback a proposal is getting. things that need improving:

• The thumbs up/down rating is replaced by a 5-star rating system, allowing users a more qualitative way of voting.

- 35 -

• Using a color-range to represent a proposals rating is a visualisation method that is not universally interpreted in the same way. Not all cultures would immediately recognise green as positive and red as negative.

fig 3.8: This mock-up tree was coded in Processing to explore possible layouts and algorithm for drawing the project-tree

project-tree mock-up During the early stages of development, a mockup version of the project-tree was coded in Processing, an open source programming language that can be used to create interactive graphic visualisations. This mockup was built to explore possible layouts and algorithms for drawing the project-tree (fig 3.8). This mock-up did not use data from archibrain.org to draw the tree, but used a number of predetermined, locally stored images as thumbnails for the proposals. This first attempt at an algorithm to draw the tree was very inefficient. As a result, none of the code from this tree was used in the final product, so further technical details are not elaborated here.

- 36 -

3.3.2 Implementing the website with Drupal & Processing The actual implementation of archibrain.org began at the start of the second semester. During implementation we mainly based ourselves on the layout and functionality of mock-up 3 (fig.3.6 p.34). The main source of feedback during the first part of the second semester came from two alphatesters who actively used archibrain.org during development and provided valuable feedback on technical issues and issues with layout and user-friendliness. The alpha-testers were two graduating engineer-architecture students from the Katholieke Universiteit Leuven, who used the website as a way to blog the various design-steps they made during their design-studio master thesis entitled: “Fragments of a regenerated Nazareth: An urban reconnection”. Near the end of the semester we also conducted a pilot-test of our evaluation (cfr. ch.4) which also provided us with valuable feedback on technical issues and issues of user friendliness concerning both the website and the project-tree visualisation.

This gives the webmaster control over what users can view, do or create on the website. Drupal has two default roles: anonymous user (no profile or logged-out), and authenticated user (logged-in user). • Nodes A node is a piece of content on the website. Apart from some exceptions (like users), each piece of content on a drupal website is a node. Each node on Drupal has the following data by default: - a title - the author - the creation date - Body content (text-area) - Content-types - A ‘type’ of node, like a blog-post or a web-page. Although over 15 contributed modules were used to add all necessary functionality to the website, we can explain the construction of the site’s basic functionality with 4 modules:

3.3.2.1 Drupal Background The archibrain.org website was built with the Drupal content management system (http://drupal. org/), using the book ‘Using Drupal’ (Byron et al. 2009) as a guideline. Drupal is an open source content management system (CMS) which allows for the creation of complex dynamic websites, even with limited knowledge of html, css, php and other scripting languages. To explain how the site was built, we first need to define a few commonly used Drupal terms: • Modules Modules are like building-blocks that expand a website’s functionality. Drupal has a few ‘core’ modules by default, which supply the basic functionality for a website (creating content, defining user roles, …). The vast majority of functionality can be added with ‘contributed’ modules, created and maintained by the Drupal community. • Roles A webmaster can define different user-roles, and assign different permissions to each role.

- 37 -

• Content Creation Kit (CCK) When defining a new content-type, the CCK module allows you to add custom fields to it. A field is a piece of data you want your node to have (eg: a text-area, images, fileattachments, etc... ). • Node Reference URL One of the possible fields that can be added when creating a content-type is a field that references to another node (another piece of content). This module allows us to fill this field in automatically with information from the current URL. • Views The views module provides an easy way to create lists and tables of content. These lists can then be turned into separate web-pages, or added to a node as a field. • Displays Displays provides an easy way to organise and layout a node’s fields. It divides the screen into regions (top, bottom, left & right-column, content top, content bottom) into which the fields can be dragged. Additionally, displays provide an easy way to create new fields for a node that contain snippets of code.

3.3.2.2 Creating the prototype’s basic functionality We can now explain how a new project-tree is made. This is done in three steps:

Step 1: Setting up the proposal’s content-type First, we need to define what makes up the design proposal that people submit for a projecttree. We do this by creating a new content-type named ‘proposal’. When users submit a designproposal, this is the type of node they submit. By default, this content-type already had a title, author, and postdate. Drupal also provides a coremodule that sets up a standard comments section on each created node. We used the CCK module to add the fields for the content-type that were still missing: • attachments A file-field where users can upload the CADfiles that accompany their design proposal. In the case of our evaluation, the only files that were exchanged were sketchup files. • images A file-field for uploading screenshots of a proposal . • description A text-field that holds a short description of a proposal. • parent proposal A node-reference field. Using the node Reference form URL module, this field creates a link on each proposal that lets users create an edited version of the original proposal. This is the equivalent of the ‘edit’ button or ‘create offspring project’ button from the mock-up designs (cfr. ch.3.3.1). The field then automatically saves the name of the new design proposals’ parent (the parent proposal). The specific fields that make up a proposal-node were subject to change during development, but we will elaborate on these changes in ch.3.3.2.2. Each time we make a new tree, we need to create a new content-type to serve for its proposal. This enables us to distinguish one project-tree’s proposal from another.

Step 2: Setting-up the project-tree In the second step we need to organise the information of the created proposals in such a way that we can use it to create the project-tree visualisation. We use the views-module to create an XML-list that summarises all data of the created proposals (title, description, images file-paths, etc...) necessary for drawing the project-tree. This XML-list can be accessed by a unique URL. The project-tree is coded using Processing. The processing code can be embedded on the website as a java-applet. When a web page with the project-tree is opened, the applet runs and parses the XML-data, using it to draw the project-tree. Exactly how the Processing code used the data to draw the project-tree was subject to change during development, and is not relevant for understanding the basic construction of the website. We will elaborate on how the final version of the project-tree worked in ch.3.3.2.4.

Step 3: Putting it all together In this last step we use the displays module as a starting point for organising a proposal’s content on the proposal page; putting the main information (title, images, description, rating) in contenttop, additional info (author, attachments, parent proposal) in the right-column, the link for creating an edited version at the content-bottom, and the comments section at the bottom. We then use displays to create a new field containing a snippet of HTML-code that embeds our project-tree applet on the page: <script type=”text/javascript” src=”http:// www.java.com/js/deployJava.js”></script> <applet class=”sketch” code=”processing_ code” archive=”/path_to_processing_ code/processing_code.jar” width=”960” height=”400”> </applet> Again using displays, we can put this field on the top of each proposal page. With these steps, we have created the basic mechanics of adding proposals, communicating data

- 38 -

to the project-tree, and putting it all together on the proposal-page. Other modules were used to add smaller bits of functionality (eg: a five star rating module), but these are not essential to understanding how the basic construction of the site works, and will be specified in ch.3.3.2.2 where we will detail the different steps of the website’s development process.

3.3.2.3 Editing the layout Drupal has a variety of user created themes that can be applied to change the look of a website. Though the CSS, HTML and PHP files that make up the theme can be edited for full customisation, most themes also allow for some basic parameters to be changed in the Drupal interface. We applied the free Zen theme which provided us with a simple clean layout. Additional small changes to layout were made using the CSS-injector module, which allowed us to apply snippets of CSS code to certain pages.

se roles were given permissions similar to the ‘viewer’ role we defined earlier, and could only view the created proposal nodes without being able to comment on, rate, or create proposals themselves.

3.3.2.5 Website layout and functionality changes during implementation The layout and functionality of our platform has changed over the course of development, based on feedback from our alpha-testers and the pilottest. What follows is a summary of the versions the platform went trough. The development process of the project-tree will be discussed separately in the next chapter.

3.3.2.4 Defining user roles We use the Role system in Drupal to create the user-roles we have defined in our concept (cfr. table 3.1). The following roles were used: • Contributor Contributors can comment, rate, and edit each others proposal nodes. When a new tree was created, a new content-type had to be made to serves as that trees’ proposal. This meant that a new unique contributor role had to be made every time a new tree was made, to ensure contributors of one tree could not edit proposals from another tree. • Commenter People with the commenter role can comment and rate proposals. Although the role should have been unique for each new tree created (like we did with the contributor role), Drupal did not let us differentiate between comments on different trees. This meant we had no way of restricting what trees a commenter could or could not comment on. • Anonymous user & Authenticated user These are two default roles in Drupal. Both the-

- 39 -

fig 3.9: Version 1: The first implementation of the main elements that make up the proposal page, without CSS applied to it.

- 40 -

Version 1 This version shows the first full implementation of all the elements of the proposal-page. Apart from activating the Zen theme and using the displays module to organise the different fields on the screen, no CSS has been applied on it to alter the layout. In fig.3.9 we can see the following implemented elements: • the Archibrain.org logo This links back to the front-page, at this stage still an empty page. • Primary links linking to an about page, the users profile, a members page and a log-in/out page • Title This is the default Drupal title-field, which will always show up above a node’s content. It cannot be moved or removed without the use of CSS. • the Project-tree java applet • the proposal: • Title Though the default title could not be moved, we could use the displays module to place a second title in the right place under the project-tree. • Description • Five-star rating this was implemented using the ‘Fivestar’ module, which displays both the average rating and the user’s rating. This last one also provides an simple interface for a user to cast or change his vote. • Thumbnail & Images Users can upload images, but have to upload one thumbnail-image separately. The project-tree uses this thumbnail to represent the proposal in the tree. The thumbnail image is displayed on the proposal underneath the fivestar-rating. Other images a user uploads are organised in a list (made with the views module) beneath the thumbnail-image, and are represented by little thumbnails. Clicking on these thumbnails opens a simple image-viewer (the ‘Lightbox 2’ module) that is superimposed on the

- 41 -

page. • author This field links to the authors profile page. • parent proposal This field links to the proposal’s parent. • attachments Clicking the link downloads the attached file. • ‘create offspring project’ link This link takes users to a page where they can submit an edited version of this proposal. • the comments section

fig 3.10: Version 2: The proposal page, lay-outed with CSS

- 42 -

Version 2 This version is the same as version 1, but layouted with CSS (fig. 3.10). We can see that this first layout was almost completely modeled after mock-up 3 (fig.3.6, p.34). A few fields were added:

During their use of the platform, the alphatesters gave the following points of critique:

• a ‘share’ button that lets users share their proposals on social networking sites. This was implemented with the ‘AddToAny’ module. • The creative commons license which applies on all added proposals is displayed at the footer of the page. It was in this stage of development that the alphatesters started using the site. There were two project-trees: one used by the alpha-testers, and one used to try out new things while developing the site. The front page, which first only contained a small welcome text, was now expanded with two lists (made with the views module), containing the latest projects from these two trees. Clicking on any of the projects takes you to that project in the project tree.

• The images on the proposal need to be bigger. Also, viewing images in a separate image viewer like lightbox is cumbersome and disconnects the image from the textual description, since the image-viewer blocks the view of the rest of the page. • The default upload limit for any file-field (like images and attachments) in Drupal is 8Mb. Some of the CAD-files that alpha testers wanted to upload exceeded this limit. • The lists on the front page should form more of an introduction to the design that is being made on the project-tree.

fig 3.11: Version 2: The frontpage of the website contained lists of the latest proposals made on the two project-trees the site was hosting at the time.

- 43 -

fig 3.12: The layout of the proposal was rearranged to allow for larger images.Looking trough different images is now done with a pager, rather then a pop-up lightbox. The color-scheme was changed from grey to black to accentuate the images

- 44 -

Version 3 In this version we addressed the issues brought forth by the alpha-testers. The following changes were made: • The images were placed above the description which enabled us to make them larger. A viewer can now look trough the different images in the proposal-page itself using a simple pager, eliminating the need to have a superimposed image-viewer blocking the view of the rest of the page. The color of the proposal was swapped from grey to black to accentuate the images. • The upload-limitation was solved by adding a short php-snippet to our websites homedirectory: <?php upload_max_filesize = 30M post_max_size = 30M This overrides the default installation of Drupal and increased the upload-limit to 30Mb • The ‘create offspring’ link was renamed the ‘create child proposal’ link, in an effort to make its purpose more understandable. • Each project-tree was now represented on the front-page by an image, a short description of the design that was being made on the project-tree, and tree lists containing information on the projects most recent proposals, highest rated proposals and proposals with the most comments (fig.3.13) The only exception to this was the image-link created for the tutorial of the website. Version 3 was used during the pilot-test. There were two other issues that emerged: • Users are still asked to upload both a thumbnail and a set of images. This separation was made so there would be one specific image the project-tree visualisation needed to load. If users wanted the thumbnail to appear in the images, they had to upload both as a thumbnail and as an image, which was perceived as cumbersome.

- 45 -

• When users edited each others proposals, their new proposal only linked to the proposal they derived it from (the parent proposal). The presumption here is that every new design proposal is derived from only one predecessor. The pilot-testers however, often wanted to combine the ideas of several proposals into a new proposal. In other words, they needed a way to reference other proposals that were not the parent proposal.

fig 3.13: The front-page received a new layout. Each project-tree was now represented as an image, a short description of the design that was being made on the project-tree, and tree lists containing information on the projects most recent proposals, highest rated proposals and proposals with the most comments. The only exception to this is the image at the top which links to the tutorial of the website.

- 46 -

Version 4 Both issues that emerged during the pilot-test were addressed in this final version, which was the version used during the platformâ&#x20AC;&#x2122;s evaluation. â&#x20AC;˘ Uploading a separate thumbnail was made redundant by having the project-tree use the first uploaded image as a thumbnail instead. â&#x20AC;˘ When users create a child proposal, they can now also reference up to 3 other proposals by selecting them from a list.

- 47 -

fig 3.14: project-tree V1: this version implemented the main elements that were defined during the conceptual fase: spherical interconnected thumbnails represent the various proposals that have been made. They vary in size depending on their rating.

fig 3.15: project-tree V1: scrolling over the proposals reveals additional information. Check-boxes allow a user to visualise additional information, in this case the proposals that most recently received comments.

fig 3.16: project-tree V1: The â&#x20AC;&#x2DC;show ageâ&#x20AC;&#x2122; parameter makes the older proposals more transparent

- 48 -

3.3.2.6 Project-tree layout and functionality changes during implementation The element of our prototype that went trough the most changes was the project-tree. Though the code went trough a total of 65 builds before reaching its final version, we will summarize this development process by focusing on the seven most important versions. Once again, the main source of feedback came from our alpha-testers and the pilot-test.

cess of information that would make the tree unreadable. The check-boxes also form a type of legend that explains what the visualisations mean. This version contained two parameters: ‘show hot topics’ put a red glow around the proposals that recently received comments (fig. 3.15), and ‘show age’ made the older proposals more transparent. (fig. 3.16)

Project-tree V1 This first version implemented most of the concepts that were formulated during the conceptual stage of development: • All information to draw the tree is parsed from a single XML-file. • Proposals are represented by interconnected spherical thumbnails. The connections link a proposal with his child proposals; in other words, if someone edits a proposal, the new proposal is linked to the original. Clicking the a proposals thumbnail takes you to that proposals page. The surface area of the thumbnails represents the rating the proposal has received. (fig. 3.14) • Scrolling over the proposals reveals an ‘information tab’, which displays additional information concerning the proposal (fig. 3.15) • Users can pan the camera by dragging the left mouse button, and zoom-in/out by dragging the right or scrolling. Clicking a ‘center view’ button resets the tree to its default position. This functionality is implemented by two community created processing libraries: Interfascia v.3.0 Alpha (Brendan Berg, 2006) takes care of the button, and giCentre Utils v.3.2.0 (X., s.d.) which handles zooming and panning. This first version also introduced a new concept: parameters. These parameters are check-boxes that allow users to visualise extra information about the proposals which can help them decide what proposals are relevant to work on. By using check-boxes, we allow users to only display the information they want to know, avoiding an ex-

- 49 -

fig 3.17: project-tree V2: The rating visualisation is now a parameter that can be turned on and off.

fig 3.18: project-tree V2: Info tabs display the last comment a proposal received.

- 50 -

Project-tree V2 changes made: • The two libraries were removed and a custom version of them was implemented by hand, which gave us full control over their functionality and appearance, and allowed us to solve the two issues that were mentioned in the previous version. • Visualising the difference in rating was converted into a parameter (fig. 3.17). The title of this parameter’s check-box (‘visualise rating’) helps explain what is being visualised. • The proposals information tabs now also displayed the last comment a proposal received (fig. 3.18), giving the users a sense of what feedback a proposal is getting. things that need improving: • The tree takes a long time to start up because it only starts drawing after all the images have been loaded. • The red glow of the ‘visualise hot topic’ parameter gives no real indication towards how many comments were given on a proposal. • Only displaying one comment per proposal gives no real sense of the discussions that are being held in the proposals comments section.

- 51 -

fig 3.19: project-tree V3: Images are loaded separately, allowing the project-tree to be drawn first. Images then appear one by one as they are loaded.

fig 3.20: project-tree V3: Each proposal can be surrounded with up to 8 comment bubbles. Scrolling over them gives the user a glimpse of the discussion that was held in the comments section.

fig 3.21: project-tree V3: The new parameter â&#x20AC;&#x2DC;visualise recent commentsâ&#x20AC;&#x2122; colors the latest comments red.

- 52 -

Project-tree V3 changes made: • The tree now loads the thumbnails from a separate XML-page, which means that it can now be drawn while the images are loading in the background (fig. 3.19). The total loading time is still the same, but the tree structure is already visible and the proposals info-tabs can already be viewed. • A proposal’s comments are now represented by ‘comment bubbles’ that appear around it, going clockwise from new to old (fig. 3.20). The comments are also loaded in a separate XML-page. Scrolling over them gives users a glimpse of the discussion that was held in the proposal’s comments section. The number of comment bubbles was restricted to 8, since 8 bubbles would fill the proposals circumference if the ‘visualise rating’ parameter was on and the proposal had a rating of 1 star (the smallest possible sphere). • The ‘visualise hot topics’ parameter was swapped for the ‘visualise recent comments’ parameter, which colored the latest comment bubbles red (fig. 3.21). The name of this new parameter is also easier to understand. things that need improving: • The tree gives no indication to what proposal is currently being viewed. Since the tree reloads every time a new proposal is opened, determining where in the tree you are required a user to remember which proposal he clicked on. • Although the tree grows from left to right, this does not necessarily mean that the proposal on the right was added the last. In fig 3.20, for example, the last proposal to be added is the one highlighted by the mouse arrow.

- 53 -

fig. 3.22: project-tree V4: The proposal with the shadowed edges indicates what proposal is currently opened in the browser.

fig 3.23: project-tree V4: Proposals received a compacter, easier to read info tab. A new parameter ‘visualise authorship’ colors all proposals and comments according to author.

fig 3.24: project-tree V4: The ‘visualise chronology’ parameter puts a number on each proposal that shows in which order they were added.

- 54 -

Project-tree V4 changes made: • The proposal that is being viewed is indicated by a shadowed edge around the proposal’s sphere. For this to work, we needed to find a way for the browser to tell the applet what proposal was currently opened. We did this by adding a line to the embed-code: <script type=”text/javascript” src=”http://www. java.com/js/deployJava.js”></script> <applet class=”sketch” code=”processing_code” archive=”/path_to_processing_code/processing_ code.jar” width=”960” height=”400”>

The [title] field is part of a Drupal module called ‘Tokens’, which provides a simple way of adding dynamic parameters to html content. The processing code could read the parameter and use it to highlight the currently viewed proposal. • The info tab received a new layout, making it more compact and easier to read: the rating is now displayed in stars, and a short excerpt of the proposal’s description is added. • Two new parameters were added: • ‘visualise authorship’ colors all proposals and comments according to author. • ‘visualise chronology’ replaces the ‘visualise recent projects’ parameter. This parameter puts a number on each proposal that shows in which order they were added. • The ‘visualise recent comments’ parameter now highlights the latest comments by enlarging them. things that need improving: • The ‘visualise authorship’ parameter lacks a legend that shows which author is represented by which color. • The ‘visualise chronology’ parameter is hard to read, and one would expect such a parameter to be visualised in some sort of time-line progressing from left to right.

- 55 -

• The shade around proposal that is being viewed is not visible enough.

fig 3.25: project-tree V5: When the ‘authorship’ parameter is checked, a list of check-boxes appears that shows what color goes with which author.

fig 3.26: project-tree V5: These check-boxes can be turned on or off to highlight certain authors.

fig 3.27: project-tree V5: The ‘chronology’ parameter reorganizes the proposals on a chronological timeline

- 56 -

Project-tree V5 changes made:

things that need improving:

• The current proposal is highlighted more by giving it a thicker edge. • The graphic user interface (GUI) has been placed in a sidebar that can be collapsed so the project-tree can take up the entire width of the applet.

• Since the project-tree grows in a horizontal way, a vertically oriented GUI is perceived as obstructive, even if it is collapsible.

• When the parameter ‘authorship’ is checked, a list of check-boxes with the authors’ names and their respective colors appears in the sidebar (fig. 3.25). These can be turned on or off, so users can highlight the contributions of certain authors (fig. 3.26). • Clicking chronology reorganises the proposals on a chronological timeline that displays the postdate of each proposal and links proposals with their children with grey lines (fig. 3.27). Distances between proposals depend on the interval between their postdates. If this interval is large, the timeline is cut by two forward slashes. This insures that the proposals would not become insignificantly small on a large timeline.

fig 3.28: project-tree V5: The full timeline. Grey lines connect proposals with their child proposals. The GUI was collapsed in this screenshot.

- 57 -

fig 3.29: project-tree V6: This version of the project-tree used during the pilot-test had a simplified horizontal GUI.

fig 3.30: project-tree V6: The way the project-tree was drawn gave the wrong impression about what proposals are current. The highlighted proposal was the second-to last proposal to be added, but the tree makes it look as if it this branch was abandoned as a possible design-solution.

- 58 -

Project-tree V6 (Pilot-test) Only two changes were made to the project-tree for the pilot-test: • The lay-out was changed, replacing the sidebar with a compact, simplified horizontal GUI. • A ‘help’ button was added to the top-right of the screen, linking users to the website’s ‘help’ page where they can find more info on how to work with the project-tree and a ‘known-issues’ section that helps them with potential problems. The pilot-testers provided the following feedback concerning the project-tree: • Pilot-tester frequently used the ‘rating’ parameter to evaluate which proposals were important. They had to activate this parameter each time the tree loaded, while this parameter could just as well be active by default. • The way the project-tree was being drawn gave a wrong impression of what proposals were recent. This is illustrated in fig. 3.30. The highlighted proposal was the second-to last proposal to be added, but the tree makes it look as if this branch was abandoned as a possible design-solution. The designs on the right seem to be the most recent, although this is not always the case. • Even though the ‘chronological’ check box provides a way to see distinguish the recent proposals, the pilot-testers complained that the connection between proposals was unreadable in the time-line format. • The pilot-testers held little value to the visualisation of time along the X-axis, since this timeline had to be cut-up so often when the interval between proposals became too long. • Having to open each proposal to be able to get a decent look at its screenshots was perceived as cumbersome, since every time a new proposal was opened, the tree had to reload. This meant that browsing through proposals was a slow process for those who wanted to see each proposal’s screenshots.

- 59 -

• The thicker edge and shade around the currently opened proposal were still not clear enough. • Pilot-testers wanted to be able to combine the ideas of several proposals into a new proposal. In other words, they needed a way to reference other proposals that were not the parent proposal. The highlighted proposal in fig. 3.30 once again helps us illustrate this. The green author wanted to implement the ideas he developed in the top branch into the purple author’s design, but had no way of making this connection visible in the project-tree. • The application flickers a lot when viewed with Google Chrome on an Apple Macbook. No PC users had this problem with Chrome. Using Safari, the application would stop working after a while on both a Mac and a PC. Firefox proved to be the only browser that worked for both PC and Mac.

fig 3.31: project-tree V7: The way the project-tree is drawn was changed to allow the projects to be displayed in chronological order. The currently opened proposal is now indicated by a clear arrow.

fig 3.32: project-tree V7: Scrolling over a proposal highlights that proposals ‘family history’, making the other proposals more transparent. This image also demonstrates the new ‘reference’ feature, which allows people to reference other proposals as part of their inspiration. This is represented in the tree by an arrow.

fig 3.33: project-tree V7: Clicking a proposal once enlarges it so the thumbnail can be viewed properly. Pressing the right or left arrows lets you view the other images that were uploaded with the proposal. After a few second the arrow disappears so as not to obstruct the image.

- 60 -

Project-tree V7 (Final) The issues that came up in the pilot-test were addressed in this final version of the project-tree. This is the version that was used during the platform’s evaluation. changes made: • We drastically changed the way the projecttree is drawn. The new version allowed us to combine the clear relation between proposals that was present in previous designs, with a logical chronological order from left to right. The distance between proposals no longer represents the time between postdates, making the order purely chronological. This method of drawing gives a better sense of what proposals are actively being developed by the design community. This is illustrated in fig. 3.31: the proposal at the bottom has clearly been abandoned. • The currently opened proposal is now marked with a clear arrow. • In the final version of the website, users could reference additional proposals when creating a child proposal, which allows them to combine various ideas and design proposals into one. References are illustrated in the project-tree by

grey arrows, and can only be seen when you scroll over a proposal. • When you scroll over a proposal, the projecttree highlights that proposals ‘family history’ by fading non related proposals out (fig. 3.32). It also highlights any references made in the family history. This mechanic allows a user to get a quick glimpse of what proposals have led to the proposal he has scrolled over. • We addressed the issue of browsing trough proposals by making the images more viewable in the project-tree (fig. 3.33). Clicking on a proposal once enlarges it so the thumbnail can be viewed properly. Taking the mouse off the image or clicking it again makes the proposal smaller. Double clicking a proposal now takes you to that proposal-page. • Additionally, we parsed a new XML-list that let us load the proposals’ other images. When a proposal is enlarged, users can look trough the various images by pressing the arrow buttons on their keyboard. • Enlarging the proposal by clicking on it also allowed us to display more comments along the circumference of the proposal (fig. 3.34)

fig 3.34: project-tree V7: Enlarging the proposal by clicking on it also allowed us to display more comments along the circumference of the proposal.

- 61 -

fig 3.35: The class diagram of the final project-tree code. The code goes from data to visualisation in 8 steps.

- 62 -

3.3.3 Class diagram of final project-tree We want to give a bit more background on the inner workings of the final project-tree code, and explain how it goes from parsing the proposal’s data to drawing the interactive project-tree visualisation. This process is summarised in fig.3.35, a diagram that gives an overview of all the classes and the way they interact with each other.

proposals.xml: • title • author • postdate • description

STEP 1

• URL that leads to the proposal-page

Central to this code is the Main class, which stores all the data and updates and draws all the elements of the visualisation at a given frame-rate. When the applet runs, the main class starts by creating the main elements of the graphical user interface: a scroll-bar for the zoom function, a button to reset the view, and the check-boxes that visualise the project-trees parameters. Both are made by making an instance of their respective classes. These instances are stored and update in the main class.

• average rating

STEP 2 The Streamer class loads and parses all the data from the XML-pages in the background. This class is an adaptation of the Streamer class described in the book ‘Visualising Data’ (Fry, 2008). This class parses 5 pages in a specific order. The first is a list of all the proposals that have been made. The list contains the following data on each proposal :

• n° comments • parent proposal • references to proposals other then the parent Each time the data-parser finds a new author, it creates an instance of the Author class and saves it in the Main class. The reason we have a separate Author class is so we can assign a unique color to each author later on. STEP 3 The streamer creates an instance of the Nodes class for each proposal that is parsed from the first XML-page. All Nodes are stored and updated in the Main class.

fig 3.36: the project-tree during step 1

- 63 -

fig 3.37: The tree needs to make space for a total of 8 branches

fig 3.38: the project-tree after step 4

fig 3.39: the project-tree during step 5

- 64 -

its counter.

STEP 4 As soon as the nodes have been parsed, the main class uses an algorithm to calculate the position of the nodes in the project-tree. For this, the Main class needs to know 3 things: 1. Which proposal is the first proposal Since the proposals.xml list is ordered chronologically from first to last, this is the first proposal that is parsed. This first proposal is always the project’s assignment. The algorithm draws the tree from left to right and starts with the assignment. 2. The parent proposal of each proposal Because every proposal knows who his parent is (this is part of the information given in proposals.xml), the algorithm can track down the proposals that were created out of the assignment. Let’s call these proposals the ‘first generation’. Instances of the Lines class are made to visually connect the assignment and its child proposals, and the position of each child is calculated (cfr. next step). Once this is done, the algorithm does the same for all the proposals of the first generation to find and position all proposals of the ‘second generation’. This process is repeated until the tree is fully drawn. 3. The number of branches that come after each proposal in the tree The proposals are horizontally positioned in a chronological manner from left to right. To know where to vertically position each proposal, we need to know how many branches need to be drawn after it so we know how much vertical space needs to be left open to draw the proposals’ child proposals. What we mean by this is explained in fig.3.37. Proposal C, for example, needs to leave enough space for 2 branches, while proposal B needs to leave enough space for 3 branches. Proposal A (the assignment) needs space for 8 branches, which is also the total height of the tree. The number of branches is not a piece of data from the xml-page but is calculated in the code by using a recursive method. Each proposal has a counter that tracks how many branches it has. When, for example, proposal C finds out it has a second child (forming a second branch), the recursive method passes on this information to its parent (proposal B) which adds a branch to its counter, and in turn passes it on to its parent (proposal A), which also raises

Once all positions are calculated, the Main class draws the tree on screen by first drawing all the lines, and then updating and drawing each Node. All that is visible at this point are the spheres representing the proposals and the lines connecting them. STEP 5 As soon as step 2 is completed the streamer starts parsing the second XML-page, comments. xml, in the background. This XML-page contains the following data: comments.xml: • author • the comment • postdate • proposal the comment was posted on The streamer creates an instance of the Comments class for each comment that is parsed. These comments are saved and updated in the Node (the proposal) that they belong to. When step 4 is finished, the Nodes will start displaying the comments that have already been parsed. If the parser finds a new author, it creates an instance of the Author class and saves it in the Main class. This would be the case for someone who did not post any proposals but did post comments. STEP 6 As soon as step 6 is completed the streamer starts parsing the third XML-page, votes.xml. This XML-page contains the following data: votes.xml: • author

- 65 -

• vote value • time of vote • proposal the vote was cast on

fig 3.40: the project-tree after step 7

fig 3.41: the project-tree during step 8

- 66 -

The streamer creates an instance of the Votes class for each vote that is parsed. These comments are saved and updated in the Node that they belong to. These votes are not used in any part of this visualisation, but are used for certain analytical graphs made for the evaluation of this platform (cfr. ch 5), since these graphs are all based on this code. Additionally, if the parser finds a vote with an author who has not posted any proposals or comments but has voted, it creates an instance of the Author class and saves it in the Main class.

There is one class in fig.3.35 that has not been mentioned yet: the Integrator class. This class has been taken from the book ‘Visualising Data’ (Fry, 2008), and calculates all moving animations in the tree.

STEP 7 As soon as all votes have been parsed, the code knows for sure that it has created an instance of all authors that contributed proposals, comments or votes to the tree. At this point the Main class will calculate a unique color for each author, which can then be used to color the comments and proposals in the project-tree. The GUI is also expanded with a number of check-boxes, one for each author. These are visible when the ‘authorship’ check-box is checked. STEP 8 In this final step, the streamer parses the last two XML-pages which contain the information on the proposals’ thumbnails and images. These pages contain the following data: thumbnails.xml: • the thumbnail’s URL • proposal the thumbnail belongs to images.xml: • the image’s URL • proposal the images belong to These are parsed last, since loading these images is the most time-intensive. The thumbnails are loaded separately so users can see the complete tree while the proposals’ additional images load in the background.

- 67 -

3.4 An overview of the final prototype The following images provide an overview of the layout and terminology used in the final version of the website and project-tree.

fig 3.42: terminology and layout of the final version for the proposal-page

- 68 -

fig 3.43: terminology and layout of the final version for the project-tree

- 69 -

- 70 -

Evaluation study

4.1 Methodology To evaluate both the efficacy of the proposed platform and the hypotheses made in chapter 1.5 (p.14), we emulated a plausible usage scenario for our conceptual platform. We did this by organising a collaborative design studio where participants worked together via our built prototype (archibrain.org) to find solutions for a given design assignment. Participants were found by sending an e-mail to engineer-architecture students from the 2 & 3 BIRA (bachelor) and 1 & 2 MIRA (master) from the KULeuven. Four students responded to this e-mail, two of which eventually participated. Eight other participants were found by direct request, resulting in a total of ten participants. During their education, students are used to work in groups of four or less. We purposefully chose a number higher then this, so students would have to rely more on the platform to keep an overview of all design proposals. We also kept the number of participants small enough as to keep the design studio manageable enough for us. On the 5th of July, the group design was organised at the Arenberg castle in Heverlee. Our goal was to present students with a design problem that was relevant to them in such a way that the demand for the design could technically have emerged from the students themselves. The students spend most of their time working in the Arenberg Castle in Heverlee. Since there are many complaints about scarce work spaces, designing an additional workspace in the form of a stand-alone pavilion seemed like a perfect assignment. The full design brief can be found in Appendix I (p.109). Before the design started, the students were asked to create a profile on archibrain.org and complete an online tutorial introducing them to the main mechanics of contributing on the site. They then filled out a background survey, which gives us an overview over their familiarity with social media, their experience on

fig 4.1: These images were part of the â&#x20AC;&#x2DC;assignment proposalâ&#x20AC;&#x2122; presented to participants to help explain the design brief.

- 71 -

sharing information or collaborating via the web, and the way they view themselves as designers and collaborators. This background survey allowed us to compare a users’ profile with their performance. After receiving a brief introduction on the design assignment, the students were given a full afternoon to collaborate and find solutions. Every student was given the ‘collaborator’ role (cfr. ch. 3.1.2), giving him or her full permission to comment, rate and post proposals throughout the design. The assignment proposal (the start of the tree) provided the students with basic sketchup file of the site (fig. 4.2), which they could use to place their designs in context.

used to track changes in participants’ motivation. Additionally, we gather the participants’ opinion on the design up to that point, and ask them about their own performance. After the final session, a final survey was filled in, in which we focus on the participants’ overall design and collaboration experience. It also asks about their perception of the project-tree, and the impact the visualisation had on their design behaviour. Finally, the group collectively discussed the design experience during a focus group session, which explored four topics:

The design was divided in three successive design sessions, lasting about 80 minutes each. In between every session, the students could take a 10 min. break, during which they were asked to fill in a short intermediate survey. This survey was

1. Archibrain.org as a concept This topic mainly touches on the overarching concept of collaborative design via the web. Based on the experience participants had on archibrain.org and the previous design experiences they had during their education, the group discusses the viability of online collabo-

fig 4.2: Participants were provided with a sketchup file of the Castle of Arenberg and its direct surroundings; the locus of the design assignment

- 72 -

ration between architects, designers and other stakeholders . 2. Usability issues The participants discuss what they liked/disliked about the platform and make suggestions for possible improvements. 3. The design This topic explores the general knowledge participants have of the design process. Participants try to summarise the design process, and highlight what they thought where the most important designs. This perceived structure of the design process is later compared to the actual project-tree.

4.2 Known restrictions of the evaluation In the evaluation study we aim to emulate a usage scenario for the website, in this case an open distributed collaborative design. There were however a few restrictions to our evaluation method which we will have to take into account during the analysis, since they may have an influence on the results:

4. Personal design experience This topic explores how participants felt about their role and the role others played during the design process. 5. A full overview of survey questions and focus group topics can be found in Appendix II (p.117) After each design session, we made a backup of the xml-pages containing all the data used to draw the project tree, allowing us to recreate the state of the project tree after each design session. After the design studio is done, the design process and user activity (both logged in the xml-files) were analysed together with the data collected from the surveys and focus group.

1. Time Collaboration via the platform would normally be spread out over many days or weeks, and participants would not be forced to work for long periods of time on a project. However, to be able to organize and guide a design session that includes surveys and a focus group, we are forced to limit ourselves to half a day of intensive design. The pilot test pointed out that this can lead to undesirable effects of time pressure like individualistic design behaviour. 2. Space Collaboration via the web is normally distributed, which means participants are usually in different geographical locations. However, for the sake of convenience (surveys, focus group, help with technical difficulties...) we decided to do a collocated design via the web (all participants in the same room). Since we try to emulate a distributed collaboration, participants have to be forced to only communicate via the website and refrain from talking to each other. 3. Predefined assignment In social production, participants usually can choose for themselves what problems are relevant to work on. In the interest of evaluating this platform however, we were forced to predefine an assignment. We tried to compensate this restriction by defining an assignment that is relevant to all participants and that could technically have emerged from the participants themselves.

table 4.1: timetable of the design proceedings

- 73 -

4.3 Pilot-test On May 25th, a pilot-test was organised in order to find possible problems with the website and errors within the evaluation method. The test was basically a small scale version of the actual evaluation (cfr. ch. 4.1 p.71). Three architecture students collaborated on a short assignment in two short sessions of 30 minutes each. Before the test they filled out the background survey and completed a short tutorial to get acquainted with the project-tree. This tutorial consisted of adding a ‘dummy’ proposal to the project tree. In between the design sessions they filled out two intermediate surveys, and in the end they filled out the final survey and discussed the four previously defined topics in the focus group. The pilot-test thus also served as a test for the survey-questions and focus-group topics. In the opinion of the author they needed no change and were used unchanged during the actual evaluation. A short overview of the generated design proposals during the pilot-test can be seen on fig.4.3. Since main goal of the pilot-test was to root out remaining problems, and not to draw actual conclusions for this research, we will not go into detail on the results. We will however focus on the main issues that presented themselves. Some issues surrounding the project-tree visualisation were already dealt with in ch. 3.3.2.6 (p.49). In this chapter we will focus on the issues concerning the evaluation method.

4.3.1 Encountered issues during the pilot-test We encountered the following problem during the pilot-test: 1. The tutorial took too long The tutorial took place in the same project-tree as the actual assignment, so before the assignment could start, the tutorial projects had to be removed. Participants complained that the tutorial took too long and that it was annoying to have to wait until the tutorial projects were removed to be able to start designing. 2. One of the intermediate surveys could not be submitted The surveys were done using Google Docs’

- 74 -

form-documents. This issue was easily fixed by making a copy of the form document. 3. Participants complained about the lack of a clear design goal Since the pilot test was short, participants were told that the design did not really have an end point. This lack of a goal discouraged participants from collaborating. During the focus group one participant remarked:

“...we knew that we had to come up with a design together, but since there was no fixed point in time where all proposals would be evaluated, we did not feel pressured to converge the tree to one design.” 4. The tree was more divergent rather then convergent. This was in a sense more of a surprise, rather then an actual issue. Though we did feel that the actual test should aim to encourage more collaboration, one should remark that a divergent tree doesn’t necessarily mean the design session has failed. During the focus group, participants all agreed that the divergent tree could be an excellent tool for brainstorming. One participant remarked:

“...it’s fantastic to be able to freely explore design possibilities and to get instant feedback on it from other designers. (...) the barrier of entry to share and edit ideas is really low.” 5. The short time span of the design sessions meant participants felt a significant time-pressure This resulted in the participants mainly working on their own designs instead of elaborating on each others ideas, since it was too difficult for them to design new proposals whilst keeping up with the new comments and proposals the other two participants were posting. This individualistic design behaviour can clearly be seen in fig. 4.4, a screenshot of the final project-tree. Almost all branches are developed by the same participant (same color). One needs to take into account that a collaborative design on this platform would normally not be crammed into one day, but rather be spread out over several weeks/months. Though the time pressure issue would probably also occur in long-term designs, it is clear that the test’s

fig 4.3: a chronological overview of all proposalsmade during the pilot-test

- 75 -

fig 4.4: the final project-tree from the pilot-test. Notable was that most participants decided to work on their own proposals rather then elaborating on other people’s proposals.

necessary time-restriction worsens the effect. 6. Users gamed the rating system by voting on their own proposals When a users’ proposal got a low rating, they could compensate this by giving themselves 5 stars. We can assume this problem had a significant effect because there were only 3 users, meaning that a 5 star rating will significantly change the average. With more users however, this effect should be less prominent.

4.3.2 Adjustments to the evaluation method The following adjustments were made to the evaluation to address the issues encountered during the pilot test: 1. Prolonged design sessions The original idea for the evaluation study was to have four design sessions of 60 minutes. In the interest of minimizing the time-pressure problem, we turned this into three sessions of 80 minutes. (cfr. table 4.1) 2. Voting and comments session Ten minutes before the last design session, participants are asked to vote and comment on their favorite proposal. This session should help participants to decide between themselves

- 76 -

what design(s) they like best and are worth working on. A complete convergence of the three is definitely not necessary, but this should help participants to regain a common sense of direction. 3. Separate tutorial-tree The tutorial was given its own project-tree (fig 4.5), this way no time had to be spent on deleting the tutorial proposals the participants had uploaded. The tree contained some basic background information on the concept of the website, and guided participants through a simple design task to familiarize them with the mechanics of adding a proposal. After this, the participants were free to experiment with the tree as much as they wanted. 4. Social performance visualisation To encourage collaboration, a simple visualisation (cfr. ch.5 fig.5.6) was made to show each participants’ ‘social performance’, in other words: how much they interacted with other participants. The visualisation was projected on a wall during the evaluation.

fig 4.5: A screenshot of the tutorial tree page. The tutorial gives participants more information on the concept behind the website and the mechanics of adding a proposal to the project tree

- 77 -

- 78 -

Results & Discussion

In this chapter, we report and discuss the results of our evaluation in two steps. First we give an overview of the proceedings of the design studio and the participantsâ&#x20AC;&#x2122; experience with the platform, and then use this together with our survey and focus group feedback to discuss the hypotheses made in chapter 1.5 (p.14) and the platformâ&#x20AC;&#x2122;s efficacy.

5.1 Report on evaluation proceedings Despite some technical difficulties during the third design session, the evaluation followed the planned procedure described in ch.4. What follows is an in depth account of the gathered results.ners with necessary information and some basic

5.1.1 Participants: background

fig.5.1: profile pictures of the 10 participants

Of the ten participants, there were three women and seven men, all aged between 21 and 23. There was only one bachelor student (3BIRA). Seven of the master students majored in architectural design (one of 1MIRA AO, six of 2MIRA AO), One majored in urbanism (2MIRA SB) and one in construction (2MIRA BT) Participants were asked about their experience with certain websites. Many participants are frequent users of social websites, content sharing sites and search engines like Facebook, YouTube, Wikipedia, Google and Hotmail. Certain content sharing websites, like Blogger and Picasa are less popular. 9 out of 10 users had Facebook and Hotmail accounts, and posted content on Facebook monthly.

fig 5.2: Pictures taken during the evaluation. Students listened to a short presentation that introduced them to the concept behind archibrain.org (above). During the design sessions, the social performance visualisation was projected on a wall (middle). After the design sessions were over, a focus group discussion was held in the castle courtyard (bottom).

- 79 -

3 users had kept a blog before for a short period of time.

5.1.2 Overview of the design process

Finally, participants were asked to define their design and collaboration profile. In both cases they could choose from a predefined set of categories. In fig. 5.3 we can see that most participants mainly focus on context and function during design, and are least concerned with construction.

5.1.2.1

Analysis visualisations

We will use four types of visualisations to analyse the users’ activity and contributions on the website during the evaluation’s three design sessions: 1. The project-tree: proposal overview This is the project-tree as it is seen by the participants, with an added overview of all proposals made during the session. Each proposal is given a serial number (also used in other visualisations) as a way to reference them in our text.

fig 5.3: difference in designer profiles among the 10 participants

On the subject of collaboration, the participants were almost perfectly divided between the possible categories. (fig. 5.4)

fig 5.4: collaboration profile

It is clear from these figures that these participants do not form a perfectly balanced group. Our main concern is the unbalanced distribution of academic years, with only two students who are not from 2MIRA. The issue here is that these students may have less affiliation with the given design assignment, since they graduate next year and have no more need for a design studio pavilion. Another issue is that these students are not fully neutral towards this evaluation, since most of them know me (author) personally and have followed the development of Archibrain.org throughout the academic year, giving them a head start concerning their knowledge of the website’s functionality.

- 80 -

2. The project-tree: all references visible References normally only appear when a user scrolls over a proposal. To give an overview of references made, an edited version of the project-tree was made that displays all references at the same time. 3. User activity graph (fig.5.5) This graph displays all proposals, comments and votes users made during the entire design session on a timeline. This allows us to analyse the way participants are using the website. Because the graph for the entire session is very big, we will magnify certain parts for each design session. 4. Social performance visualisation (fig.5.6) This graph, which was projected on a wall during the design, allows us to visualise certain aspects of the participants’ collaborative performance during a design session. For each user it displays the number of comments they posted on other peoples proposals, the number of proposals they edited and the average rating they received from other users on their own proposals.

fig 5.5: The user-activity graph organises all proposals (big spheres with the proposal’s serial number on top), votes (small bars) & comments (little spheres) on a time-line and colors everything by author. The colored strips vary in height and represent the varying average rating of a proposal. Crosses represent the creation of a child proposal, triangles represent references.

fig. 5.6: In the social performance visualisation, each participant is represented by a colored circle. The Y-axis displays the number of proposals a participant has edited (his/her own not included), the X-axis displays the number of comments he gave on other peoples proposals (his/her own not included). The size of the spheres represents the average rating a participant received on his proposals. The more a participant collaborates giving his opinion on other designs and editing them, the more he rises to the top right. ‘Labels’ are given to people who have commented the most (‘most comments’ label), edited the most proposals (‘most edited’ label) or gotten the best average ratings on their proposals (‘highest av. rating’ label).

- 81 -

5.1.2.2

Session 1

number of proposal submitted: 10 number of comments posted: 43 number of ratings given: 16 DESIGN PROCEEDINGS The participants started by discussing what direction to take the design assignment in by posting comments on the project-tree’s assignment proposal. The main topic that emerged in this discussion is whether the pavilion should be of a temporary nature:

“Maybe we should design a structure that is only used for a few years (...) “ - user 10 “That seems like a good idea. In a few years [the university] is planning to remove architecture students from the castle... a fully realised invasive building [in the vicinity of the castle] would not be a good solution.” - user 7

There were some designs that explored different directions; one proposal aimed to keep the castle grounds free of pavilions by latching ‘a parasite pavilion’ to the castle wall (proposal 6). Another proposal expanded the assignment to a larger scale, proposing to define a circulation axis connecting the campus with the castle grounds along which various pavilions could be placed (proposal 9). In this way this proposal could incorporate various pavilion concepts made by other users; this is why the author referenced so many other proposals (fig. 5.7).

This theme of temporality became the main focus for many students. Their first concepts were very diverse, ranging from a studio on the river that could float down-town to another location (proposal 8), to a server-farm (proposal 3) that aimed to replace the physical places where students could meet and work together with a digital equivalent (like archibrain.org). One prominent proposal in this line of thought suggested a pavilion built up from a sequence of simple trusses. This was the only proposal to be adapted by various other users.

fig 5.7: Design Session 1: The project-tree with all references visible. Both the ‘rating’ and ‘authorship’ parameter are active.

- 82 -

fig 5.8: a chronological overview of the design proposals made during the first session. Each proposal is given a serial number.

- 83 -

USER ACTIVITY & PERFORMANCE Eight users posted at least one proposal and two or more comments during the first session. For the majority of this group we can notice a ‘break’ during which they do not post comments or votes for a period of 10 to 30 minutes. This happens in the time before they upload a new proposal, during which they are working ‘offline’ on their new design. Two participants were mostly inactive during this session. User 4 was completely absent since he spent the entire time designing his first proposal without commenting or voting on other ideas. User 2 quickly posted the concept he wanted to develop and then started to design it during the remainder of the session. These two can be found at the far left of the social performance display (fig.5.10). Proposals received an average response of 3.9 comments and 1.6 votes, with an average rating of 3.1 stars. User 3 (fig 5.10: top right) has an average received rating considerably above this, but this is only because he posted a new proposal right before the first session ended and received one high vote on it. He was however the most active commenter, posting a comment on nearly every proposal.

- 84 -

fig 5.9: User activity during the first design session.

fig 5.10: The social performance visualisation after the first design session. Most users posted at least one design proposal and two or more comments.

- 85 -

5.1.2.3

Session 2

number of proposal submitted: 14 number of comments posted: 61 number of ratings given: 37 DESIGN PROCEEDINGS During the second session participants mainly elaborated on three main branches, each continuing a different approach from the first design session. For the sake of convenience we will name these branches the ‘circulation axis’ branch, the ‘truss pavilion’ branch and the ‘river pavilion’ branch (fig. 5.11).

Two smaller branches were less popular with the design community (curved folding & parasite pavilion). Both of these were initiated by their original authors who elaborated their previous proposals.

The ‘truss pavilion’ branch elaborates on the idea of creating a pavilion built up out of sequence of trusses (proposal 11). The new designs explored more sculptural forms by varying the size and direction of the trusses (proposals 13, 16, 25). The ‘circulation axis’ branch elaborated on the idea of a circulation axis that connects parts of the castle park with the campus (proposal 9). Two of these proposals explored how pavilions could be distributed on this scale (proposals 18 & 23), while another explored a way to merge a shorter version of the axis idea with the truss pavilion concept (proposal 23). The ‘river’ branch combined a new pavilion design near the river (proposal 14) with the boat proposal from session 1 (proposal 8), replacing the boat with a pavilion that is partially projected over the water (proposals 19 & 24).

fig 5.11: Design Session 2: The project-tree with all references visible. The ‘‘authorship’ parameter is active. Proposals are marked with their serial number. The accolades on the right mark branches that explored different approaches to the design. The three dark accolades mark the most popular branches.

- 86 -

fig 5.12: a chronological overview of the design proposals made during the second session. Each proposal is given a serial number.

- 87 -

USER ACTIVITY & PERFORMANCE During this session, the majority of users were collaborating on one of the three main branches, each participant contributing at least one proposal, and posting an average of 6 comments and 3 votes on each others’ proposals. The user activity graph shows us that a large part of comments and votes in this session went to designs from session 1. The proposals from session 1 received an average response of 4.1 comments, while new proposals made during this session only received an average of 1.4 comments. This is mainly due to proposals 8 & 9, which formed the foundation of the ‘circulation axis’ branch and the ‘river pavilion’ branch. These proposals were posted right before the break and now received the attention of a large part of the design community. The new proposals posted in this session that received the most comments and votes were those who elaborated on proposal 8 or 9. We can see for example how the discussion around pro-

posal 9 shifts to proposal 14 & 15 once these are made. The two users who carried on their own branch from session 1 (the two less popular branches) hardly received any feedback. User 2’s proposal 12, for example, was posted at the start of this session and only received its first comment near the end. User 1 tried to combine the truss concept in his branch (proposal 17), but also received only one comment near the end of the session. Users did not seem to bother with these branches if they did not like the initial idea. This also meant, however, that the unpopularity of these branches did not manifest itself in low ratings, since no one bothered to rate these proposals. It appears to us that popularity in this session can mostly be measured with the number of people involved with a proposal (editing and commenting) rather then with the average rating it receives.

fig 5.13: The social performance visualisation after the first design session. Most users posted at least one design proposal and two or more comments.

- 88 -

fig 5.14: User activity during the second design session.

- 89 -

5.1.2.4

Session 3

number of proposal submitted: 0 number of comments posted: 31 number of ratings given: 60 DESIGN PROCEEDINGS The plan for this session was to let participants discuss and rate all proposals up to this point for a short period of time, allowing them to regain an overview of all proposals that had been made and decide amongst themselves what proposals to continue working on during the remainder of the session. Unfortunately, we were faced with some technical difficulties. A null-pointer exception in the Streamer class of the project-tree code meant the tree would draw, but the thumbnails and images would not load. This, combined with the long loading times of the large tree (+/- 30 seconds to load tree and comments) meant that browsing trough the proposals became very cumbersome. Nonetheless, participants were asked to comment and rate each others proposals for a period of half an hour. The problem was fixed after this time, but long loading times persisted. This, combined with a general feeling of fatigue amongst participants, lead to the decision to end the design session early.

Since no proposals were added, the project-tree maintained the structure it had at the end of session 2. USER ACTIVITY & PERFORMANCE The user activity graph clearly shows that user activity during the last session consisted of comments and votes. Though 18 of the 25 proposals received new feedback, most attention was given to the proposals that were added at the end of session 2. If we compare last sessionâ&#x20AC;&#x2122;s social performance visualisation to that of this sessionâ&#x20AC;&#x2122;s we see the impact of taking a break from designing to review previous proposals. The obvious change is how all users shifted to the right because of all the comments they posted. We can also see bigger differences in the average rating users received. The unpopular proposals which were previously simply ignored, now finally received ratings that corresponded to their popularity, bringing the authors average received rating down with them.

fig 5.15: The social performance visualisation after the last design session. Note that this visualisation displays the accumulated comments, proposals and rating of all three sessions. The big shift of all authors to the right is caused by the voting and commenting session.

- 90 -

fig 5.16: User activity during the third design session. Users were asked to take a moment to only comment and rate, giving them an opportunity to decide which proposals should be elaborated in this last session. Most attention was given to the proposals that were added at the end of session 2. Due to technical difficulties, no new proposals were added at all.

- 91 -

5.2 Hypotheses: results & discussion The discussion of our hypotheses is founded by the results of the design studio (see above), the responses we received on the intermediate and final surveys (cfr. Appendix III), and the focus group discussion held at the end of the evaluation (cfr. Appendix IV).

5.2.1 Impact on Motivation We claimed our platform would have a positive impact on users’ motivation to contribute ideas and designs to the platform. To analyse this we based ourselves on 8 categories of motivation, as defined in previous research by Maher et al. (2010).

an average of 3.9 (sd=0.83). A statistical T-test4 shows that this average differs significantly from the neutral point of 3 (with 90% certainty). We can assume that this result is more positive than the first question because comments and votes do not require the approval or feedback of the design community to feel relevant, as they are feedback in themselves. Either way, these results indicated that participants felt their contributions contributed to the larger cause of finding a solution to the design assignment, whether these contributions were actual designs or just the feedback they gave on other people’s proposals.

5.2.1.2 Career 5.2.1.1 Ideology We assumed the platform would motivate users by giving them the feeling they are contributing to a larger cause. In the case of this evaluation the ‘larger cause’ refers to the cause of finding a solution for the given design assignment. We checked this by asking the participants whether they felt like their contributions mattered in the overall design, When asked if they felt like their proposals helped improve the design (likert1 scale), 5 out of 10 participants responded positively, with 4 others answering neutrally (av2=3.4 sd3=0.66). The 5 positive participants were the main designers behind the three popular branches in the tree (fig.5.11). In a second question participants were also asked whether they felt that commenting and rating helped improve the design. A vast majority (8 out of 10) participants responded positively with 1- A likert scale asks respondents on a scale from 1 to 5 how much they agree or disagree with a certain statement. 2- av: average 3- sd: standard deviation. The standard deviation gives us a measure of the diversity of the data. A low standard deviation indicates that the data points tend to be very close to the average, whereas high standard deviation indicates that the data are spread out over a large range of values

Users can be motivated to work with a platform like archibrain because they feel it could help them advance their career. The platform could provide two ways of doing this. The first way is by providing offices with a medium to organise distributed design collaborations. Though this method of use was not the focus of our evaluation, we did ask participants during the focus group who they thought could benefit from this platform, and architectural offices were one of the main possible user groups they highlighted:

“I think [the platform] could be useful in an office where [some people] work part-time at home. This way they can still elaborate on designs ideas. Or it could be used by two offices in different countries who want to collaborate. They can use the website to brainstorm for ideas.” - user 7

4- A T-statistic corrects the average using the standard deviation and the number of respondents (n=10). It allows us to determine with a chosen degree of certainty (in our case 90%) if a deviation from the neutral point of 3 is a relevant result, and not just a random occurrence caused by our small number of participants. For more details on T-statistics, see Albright et. al., The t-distribution, In: Data Analysis & Decision, (2009).

- 92 -

“(...) working [asynchronously] can be a plus: if you have an architect in Belgium and an architect in Australia, one can work while the other sleeps.” -user 2 The platform could also help participants advance their career if they can reference their work on the platform in a quantifiable way. The social performance visualisation (fig. 5.6, p.81) suggested a way to award important contributors with labels (‘most comments’, ‘most edited’, ‘highest average rating’), which one could use a way to measure the importance of their contributions in a group design. Discussion during the focus group pointed out that opinions were divided over the positive effects of the social performance visualisation. One user felt it could encourage less active employees to contribute more:

“The [social performance visualisation] meant you were ‘forced’ to edit proposals or post comments to avoid [falling behind on the rest].” - user 7 Most users however showed concern that this visualisation would not encourage the right motivation with designers:

“If your pay was based on such a [performance visualisation], you’ll quickly start editing proposals without proper cause.”- user 9

scale). During the first two sessions this question resulted in a neutral average response (session 1: av=3.2 sd=1.08, session 2: av=3.1 sd=1.04). 4 out of 5 respondents who felt challenged during the first session answered the same during the second and third session. In contrast to this, 2 out of 4 people who did not feel challenged during the first session, did feel challenged during the second session. A possible reason for this rise in challenge is the increased complexity of the project-tree that users try to take into account, and an increased attempt of users during the second session to combine each other’s proposals. This is apparent in the complex network of references seen in fig.5.11 (p.86). The third session generated an average response of 4.1 (sd=0.83). A statistical T-test showed that this result differs significantly from the neutral point of 3 (with 90% certainty). We can attribute this to the technical difficulties we experienced during the third session, which made adding proposals challenging for all participants. In an open question in the final survey, participants were asked if they were introduced to new ideas, concepts or ways of working. Most answers to this questions were positive, focusing on the variety of ideas a large group of people exposed them to:

“[And you’ll do this] without taking the quality of your contribution into account.” - user 1

“Yes. Different people brought different ideas to the table. [These ideas] were discussed and then [reconnected] with older ideas, forming a better design.“ - user 6

“[The visualisation] does not take into account (...) in which way someone designs.” - user 7

“Especially the feedback was interesting” - user 10

5.2.1.3 Challenge

The variety of proposed designs was indeed apparent in our overview of the design sessions (ch.5.1.3.2 - 5.1.3.3).

We claimed that the challenge of solving design problems through participation on this platform would provide a sense of personal achievement to users, since the collaborative design process would lead them to acquiring new knowledge or skill.

As the design progressed and the tree grew large, taking other people’s proposals into account made designing more challenging. Whether users experienced this increase in challenge as positive or not is not clear to us, but they did appreciate working with this variety of designs and opinions.

In the three intermediate surveys we asked participants if they felt like creating new designs during the design session was challenging (likert

- 93 -

5.2.1.4 Social We claimed participants could be motivated by an enjoyable social experience of designing and interacting with others via the platform. We tested this trough a set of survey questions that asked how much participants appreciated feedback and collaboration on the platform. During the intermediate surveys, participants were asked whether they felt like they were part of a team. 3 participants responded positively after the first session, 5 after the second and 4 after the third. There was an average neutral response on all three sessions. The three positive responses came from the participants that worked on the â&#x20AC;&#x2DC;truss pavilionâ&#x20AC;&#x2122; concept. (fig. 5.17) A possible explanation for the increase in positive responses during the second session is the emergence of the main three branches in the tree. Each branch was elaborated on a more specific type of solution, and could have given its participants a feeling that they were working together on the design. When asked whether they took the feedback they received on their proposals into account, a vast majority responded positively during the first two sessions (session 1: av=3.4 sd=0.49, session 2: av=3.9 sd=0.54). A statistical T-test showed that these results differ significantly from the neutral point of 3 (with 90% certainty). During the last session this question was deemed irrelevant since no proposals were added in this period of time. This indicates to us that, even though users appreciate feedback from others, a real social experience of working together seems to be depen-

dent on whether designers share a similar vision for the design.

5.2.1.5 Fun Our claim was that contributing on the platform could provide a pass-time that users could enjoy. When asked during the intermediate surveys whether they enjoyed participating in the previous session, participants responded positively for all sessions save the last one. (session 1: av=4 sd=0.77, session 2: av=4 sd=0.63, session 3: av=3 sd=0.89). A statistical T-test showed that the first two results differed significantly from the neutral point of 3 (with 90% certainty). From this we can assume that in general, participants found working on the platform an enjoyable experience. The neutral response to the final session can be attributed to the technical difficulties and the general feeling of fatigue amongst participants during the last session.

5.2.1.6 Reward We claimed that the project-tree can serve as visual form of reward for participants, by displaying their contributions to the overall design process. In the final survey we asked participants if seeing their proposals visualised in the tree felt rewarding. 6 out of 10 respondents answered positively (av=3.8 sd=0.98). When they were asked the

fig 5.17: Participants working on the same branch felt like part of a team during the first session

- 94 -

same for the visualisation of their comments, 5 out of 10 responded positively (av=3.6 sd=0.92). User 7, who was one of the participants whom contributed the least, was the only one to respond negatively on both questions. A statistical T-test showed that both averages differ significantly from the neutral point of 3 (with 90% certainty). During the focus group the participants elaborated on what aspects of the visualisation felt the most rewarding: “I found it interesting to see that certain proposals in the tree had a lot of [comments] surrounding them. This made it especially interesting to see what proposals generated the most discussion.” - user 1 “I found it especially rewarding when your proposals [generate a large branch]. The size of the proposals, [based on rating], was all a bit the same” - user 7 “Indeed, all ratings seem to be situated between 2.5 and 4 stars.” The positive response to the survey questions does not count for a big majority of the participants, so we cannot conclude with certainty that the mere act of visualising their contributions was an important motivator in this design session. It appears to us that it was more through the visualisation of community feedback, especially the visualisation of comments and child proposals, that participants felt visually rewarded by the tree. The social performance graph also served as a visual way of rewarding diligent contributors. As we discussed in ch. 5.2.1.2, users’ feelings towards this visualisation motivation were ambiguous: it motivated people to keep contributing to improve

their score, but it did not do enough to reward the quality of designs or take a users’ design approach into account.

5.2.1.7 Recognition We assumed participants would feel motivated because of the feedback they get on their contributions. To check this we asked them in the intermediate surveys if they appreciated the adaptations that were made out of their proposals and if they appreciated the comments and ratings they received. In the case of adaptations, participants in general responded neutrally during the first session, with only one participant responding positively (av=3 sd=0.45). After the second session the average response was higher (av=3.5 sd=0.92), with 4 participants responding positively. This increase can be attributed to the fact that participants edited more proposals during the second session. The third session was not taken into account since no proposals were added during this period. In the case of comments and ratings, participants responded with increasing positivity, going from 5 positive responses after the first session (av=3.5 sd=0.5), to 7 after the second session (av=3.9 sd=0.7), and 8 in the third session (av=3.9 sd=0.83). The results suggest that users feel more recognised when they receive comments and ratings, rather than adaptations of their proposals. We assume this difference stems from the fact that almost all proposals did receive comments and ratings, while a majority of proposals did not receive adaptations.

fig 5.18: Participants indicated that the visualisation of received comments (left) and child proposals (right: large branches that grow out of a user’s proposals) feel more rewarding then the visualisation of ratings (size of proposals), since all proposals appear to be more or less the same size.

- 95 -

5.2.1.8 Duty Participants can be motivated by a personal desire to contribute content. In order to find enough participants, 8 of our 10 testers were directly asked to participate, rather than participating voluntarily by responding to the general e-mail. It is the author’s opinion that asking for personal motivation for contributing content would result in a biased answer in this test-group. During the intermediate surveys we did however ask the participants if they were contributing content because they had to, rather than because they wanted to. With this we wanted to poll if they were merely contributing to get the experiment over with (positive response), or if they were still engaged in the experiment (negative response). 7 out of 10 participants responded negatively during the first session (av=2.1 sd=0.7), 9 responded negatively during the second (av=1.8 sd=0.6), and only 2 during the third (av=3 sd=0.63). These results indicate that participants were generally engaged in the experiment during the first two sessions, but lost interest during the last. We can assume that this is due to the general sense of fatigue amongst participants.

5.2.2 Impact on the design process

5.2.2.1 “The project-tree forms a correct representation of the design process, and helps users keep an overview of the design process“ We claimed that our project-tree visualisation could help participants keep an overview of the different design proposals that were made and the relation between them. When asked in the final survey if they felt like they had a good overview of the proposals that had been made during the design process, 7 out of 10 participants answered positively, with an average of 3.8 (sd=0.87). A statistical T-test showed that this average differs significantly from the neutral point of 3 (with 90% certainty). This indicates that our participants generally felt like they had a clear overview over the design process. To check if the project-tree helped them with this, participants were asked whether they felt the project-tree was a correct representation of the design process

and their design contributions. A vast majority of participants responded that the project tree gave a clear view of the relations between designs. Some also said the chronological order of the tree helped make the design process more readable:

“[The project-tree] was [a clear way] to see which proposals [were] continuous and which not.” - user 5 “(...) the lines that are drawn between the designs are [correct].”- user 8 “(...) the time aspect was very important. Also the relations between the designs were clear.” - user 2 “[The project-tree] clearly represents the steps you take.” - user 1 Despite this, some users still felt that at later moments in the design, the graph, and especially the reference lines (fig.5.19), became increasingly complex:

“(...) it can get a little confusing with longer and more complex designs. Also the [reference] links could be a little messy sometimes.” - user 7 “Very interesting visualisation, but the references to other projects became unclear.” - user 1 Their answers indicate that, despite some issues with complexity during later stages of the design, the project-tree helped users maintain a clear overview over the design process. The issues people had with the complex references may be due to a design problem with the project-tree: scrolling over a proposal not only reveals that proposals references, but also the references of the proposals that came before it. We assume that it is this mechanic that makes the references hard to interpret. Only showing the highlighted proposal’s references may make the graph easier to interpret (fig.5.19). During the focus-group users were asked to give a short overview of the design process and to highlight which proposals they felt were most important. They mainly focused on the fact that there were three main branches that developed different ideas:

- 96 -

previous chapters, provide a clear indication that the project-tree allowed users to explore a variety of concepts at the same time. We should note that although we showed that the tree could encourage users to explore a diversity of solutions, we have not been able to prove that our platform can provide a way to converge these different approaches back into one design. We tried to do this by incorporating the voting and commenting session before session 3, but the technical difficulties prevented us from continuing the design from that point to see if one predominant idea would emerge.

fig 5.19: References became increasingly complex (and more difficult to interpret) in later stages of the design, since scrolling over a proposal not only reveals that proposals references but also those of the references that came before it (top). Only drawing the references of the highlighted design may reduce complexity (bottom).

“There were three [lines of thought]: the trusses, the circulation axis and the boat. [These were] three directions in which the design was heading.”- user 2 We described these three directions in which the design was going in our previous overview of the design sessions (cfr. ch. 5.1.3.3). The fact that the users also highlighted these branches indicates that well developed branches in the project-tree form a correct representation of different lines of thought in the design process.

5.2.2.2 “Rather than dealing with one design proposal at a time, the community takes the liberty to work on different possibilities at the same time” We assumed the clear overview of the projecttree would encourage users to take the liberty to explore different design possibilities at the same time, allowing users to create diverse perspectives on how to solve a design problem. Our hope was that the project-tree would serve as more than just a ‘version-control’ tool. The different branches of the project-tree (fig.5.11, p.86), which we have already often discussed in

5.2.3 Impact on social dynamics of collaborative design

5.2.3.1 “Our platform provides effective means for a design-community to give each other feedback, and visualises this feedback effectively in the project-tree to highlight important design proposals.” The platform provided participants with two means of giving feedback on each others’ proposals: a five-star rating system and a comments section. These two systems of feedback were visualised in the project-tree: a proposal’s surface area was linked to its average rating, and the number of orbs around it indicated how many comments it received. We will discuss our feedback system in two ways. First we will attempt to evaluate if this feedback system was an effective basis for assessing and visualising which contributions were more important than others in the project-tree, by analysing in what way participants combined votes and comments. Then we will report on the general feedback participants gave us concerning the feedback system. ANALYSIS Our first analysis is based on two graphs: one compares the number of comments and average rating every proposal received during the three design sessions (fig.5.20), to which we will refer as graph 1 in what follows. The other compares this average rating with the number of votes that

- 97 -

make up this rating (fig. 5.21). We will refer to this graph as graph 2. We know from our overview of the group design (cfr. ch. 5.1.3) that proposal 4 was one of the key proposals during session 1, since it was the proposal that spawned the truss pavilion branch. Due to its importance in the overall design, we would expect it to have a high rating or a large amount of commentary. In reality, the proposal only received 2 comments (cfr. graph 1), and although it did receive more votes then the average proposal during that session, its rating fell below average (cfr. graph 2). Proposal 7, a proposal in the same branch that elaborates on proposal 4, did however receive many comments as it sparked the question of where the pavilion should be located. Only 2 proposals had an average rating that significantly differed from the neutral rating of 3, but neither of these had received more than 2 votes meaning the average did not significantly reflect overall popularity.

rate all designs (as we did in session 3), but in the case of our evaluation this voting session landed the bulk of the proposals around the neutral score of tree stars, showing little variance between proposals (cfr. graph 2 session 3). The number of comments a proposal received seemed to be a good indicator of what proposals introduced the most innovative or challenging concepts, even though these proposals were not necessarily the most important proposals during a design session. (cfr. proposal 4) We should remark that during development, we did not consider the creation of offspring proposals as a type of feedback that should be highlighted in the visualisation. Although it was not part of our design goal, it appears to us that the visualisation of branches in the tree did however serve as an important indicator of importance, with key proposals of the design process generally at the root of complex branches.

The key proposals during the second session were proposals 8, 9 and 11, since they formed the basis for the three main branches in the design. In this case, both proposal 8 and 9 received many comments, and had a high average rating (cfr. graph 1). We can still question the validity of proposal 9’s high rating, as it only received 3 votes. Three proposals had a significantly higher than average rating during this session, but in all three cases, the authors of the proposals had given themselves a 5-star rating. The graph gives a first indication that the more votes one of these proposals received, the more the high score was dampened. This indication becomes fully apparent during session 3 (cfr. graph 2). During this session participants took the time to revisit all proposals and give comments and ratings. As more people voted, up to 16 proposals received 4 or more votes, making their rating a more plausible reflection of their popularity. The result of this is that almost all proposals dropped in popularity, with only proposal 8 having a distinctively higher rating than the rest.

GENERAL FEEDBACK

It appears that the rating mechanics did not give a consistently correct representation of a proposal’s popularity, since not all proposals received sufficient votes during the design for the average rating to be significant. This low rating response also meant the average was more susceptible to users who voted their own projects up. A possible solution for this could be to ask all users at certain points of the design process to take a moment to

During the focus-group participants were asked what they thought of the commenting system. Some users told us they would have liked the visualisation of comments to be more personal, highlighting new comments they had not read yet or comments they received on their proposals:

“I felt that (..) comments you have not yet read in the tree should be [highlighted], so you know you haven’t seen it yet.” - user 9 “Indeed, or you should be able to see when comments were made on your own proposal.” - user 1 Participants also remarked that the order of the comments was counter-intuitive, since new comments appeared on top of the comments-list, but reply comments appeared beneath the original comment:

“I did not find the order of the comments very intuitive: on fora the newest post always appears on the top, but here the newest could just as well be at the bottom [in the case of a reply]” - user 3 They also felt that comments should allow users to include pictures:

“It would be nice if you could add a picture to a comment. This would especially come in handy if you don’t really have a new idea, but still want [to

- 98 -

fig 5.20: Graph 1: These graphs display the average rating (y-axis) and the number of comments (x-axis) proposals received during the three design sessions. Proposals are marked with their serial number.

- 99 -

fig 5.21: Graph 2: These graphs display the average rating (y-axis) and the number of votes (x-axis) proposals received during the three design sessions.

- 100 -

post something visual]” - user 1 “[adding to user 1’s comment:] For reference projects for example.” - user 7 Users also felt that comments should have a thumbs up/down rating system to allow them to agree with someone’s opinion:

“[With a thumbs up button] you would be able to say if you agree with someones statement. It’s better than having to reply to show your support, and it allows you to see how many people agree with the statement.” - user 6 Participants were generally not very happy with the 5 star rating system, as they felt that little difference in rating between proposals could be perceived:

“(...) the sizes of [the proposals in the project-tree] were all very similar.” - user 7 “Indeed, all ratings seemed to be between 2.5 and 4 stars.” - user 2

5.2.3.2 “Users feel comfortable editing other designs and having their designs edited” Collaborating on the platform meant users had to open their ideas up for others to adapt, giving up part of their control and intellectual property rights to serve this open form of collaboration. We claimed that the participants would not feel bothered by this and embrace the fact that their ideas were used by others as this helps progress the overall design. In response to a likert question in the final survey, all but one user responded positively to the statement that they did not mind when others edited their proposals (av=4.1 sd=0.54). When asked if they felt comfortable editing other peoples ideas, 7 out of 10 responded positively (av=3.8 sd=0.87). A statistical T-test showed that both results differ significantly from the neutral point of 3 (with 90% certainty). We can assume from this that participants had no problem with this open form of collaboration. This can also be seen in the project-tree, as the three popular branches are also the ones developed by a variety of people (fig. 5.11).

5.2.3.3 “The project-tree gives users a good impression of the role they and other users played in the design process” By visualising users contributions to the design in the project-tree, be it comments or proposals, we claimed that participants would have a good overview over which users played prominent roles in the design process. During the focus group session, we asked participants who they thought were the most prominent contributors. After the first session, participants particularly praised user 3 and user 6 for commenting on almost all proposals and posting two proposals each. They also commended user 7 for the quality of her comments. After the second session, user 3 and user 5 were commended for the quantity of their comments, and user 10 and user 3 were praised for the quality of their proposals, which often introduced new ideas that challenged the design assignment in interesting ways. We verified if this praise was readable in the project-tree. In the case of user 3 and user 6’s comments and proposals during session 1, turning on the authorship parameter and only highlighting their colors clearly shows a good portion of the posted proposals are theirs (fig.5.22, top). Their overall presence in the comments becomes clearly visible after zooming in a little (fig.5.22, uppermiddle), as many of the orbs around the proposals show their colors. The platform did however lack a way of awarding quality to comments, so user 7’s importance to the design is hard to discern from the project-tree without reading all the comments. After the second session the quantity of comments posted by users 3 and 5 became hard to notice, since the tree became so large that the comments were hardly visible without zooming in a lot (fig.5.22, lower-middle). In the case of visualising quality of proposals, we have already seen earlier that users saw the number of offspring proposals, references and comments a proposal received as a measure of popularity. This was visible in the case of user 10 and user 3’s most popular proposals (fig.5.22, bottom).

- 101 -

fig 5.22: The two images at the top show the project-tree after the first design session. The two at the bottom show the tree after the second. It appears that the project-tree visualisation succeeds at highlighting contributors with many proposals (top) or important proposals (lower-middle), but fails to highlight qualitative commenters and struggles to highlight commenters with many comments when the tree grows large and complex (bottom).

- 102 -

It appears to us that the project-tree visualisation succeeds at highlighting contributors with many proposals or important proposals, but fails to highlight qualitative commenters and struggles to highlight commenters with many comments when the tree grows large and complex.

5.3 Efficacy of the evaluation The evaluation results show us two important flaws of our evaluation methodology which should be considered in the future when trying to evaluate an open flat design environment: â&#x20AC;˘ The design community should choose their own design problem Although we tried to emulate a flat condition by giving the students an assignment that was strongly related to them and could technically have emerged from themselves, two of the most popular branches in the project-tree where those who challenged the design brief and looked at the design problem from a different perspective. This can be seen as a flat decision by the design-community, who worked on the problems they actually found relevant. â&#x20AC;˘ Designers should work on a project only if they are motivated to do so Forcing the designers to work in the projecttree for three consecutive sessions resulted in a general sense of fatigue.

- 103 -

- 104 -

Conclusion f. Reward The visualisation of community feedback, especially the visualisation of comments and child proposals, was visually rewarding for most participants

The goal of this research was to define and evaluate a conceptual platform where people can collaborate on design problems they define themselves. Our concept mainly revolved around the project-tree, a central interface that gives an organised overview of design proposals and helps users keep an overview over the complex collaborative design process. To evaluate the efficacy of our concept, we made a set of hypotheses about the desired impact of the platform on user motivation, the design process and the social dynamics of collaborative design. What follows is a summary of our findings:

g. Recognition In general, users feel more recognised when they receive comments and ratings, rather than adaptations of their proposals. We assume this difference stems from the fact that proposals would get quicker feedback in the form of a comment or a vote, and that a majority of proposals did not receive adaptations. h. Duty In general, participants were engaged during the experiment. Their motivation to contribute diminished at the end of our evaluation due to a general sense of fatigue.

1. Impact on user motivation a. Ideology Our results indicate that participants felt their contributions contributed to the larger cause of finding a solution to the design assignment, whether these contributions were actual designs or just the feedback they gave on other peoples proposals.

2. Impact on design process

b. Career Visualising user performance in the social performance visualisation so it could be referenced to as their amount of â&#x20AC;&#x2DC;contributionâ&#x20AC;&#x2122; was met with mixed results. It encouraged users to contribute, but did not necessarily take the quality of their contributions into account. c. Challenge As the design progressed and the tree grew large, taking other peopleâ&#x20AC;&#x2122;s proposals into account made designing more challenging. Whether users experienced this increase in challenge as positive or not is not apparent to us, but they did appreciate working with this variety of designs and opinions. d. Social Even though users appreciate feedback from others, a real social experience of working together seems to be dependent on whether designers share a similar vision for the design, as was the case in the various branches. e. Fun In general, participants found working on the platform an enjoyable experience.

- 105 -

a. The project-tree forms a correct representation of the collective design process, and helps users keep an overview of the design process Using the project-tree, users generally maintained a clear overview over the design process since it provides an easy way to browse through proposals and their comments, and the different branches in the tree gave a clear representation the different design solutions that were being explored. However, as the tree grew bigger, the references between projects became very complex and unreadable. b. Rather than dealing with one design proposal at a time, the community takes the liberty to work on different possibilities at the same time. The different branches of the project-tree each centered around a different design approach, showing that the project-tree allowed users to explore a variety of concepts at the same time.

3. Impact on social dynamics of collaborative design a. Our platform provides effective means for a design-community to give each other feedback, and visualises this feedback effectively in the project-tree to highlight important design proposals. Comments and the number of offspring proposals were generally seen as a good way to evaluate a proposal’s importance. The five star rating system however proved to be little effective in this evaluation, as the average proposal received only a few votes during the normal design sessions, and the voting and commenting session resulted in a majority of proposals with a rating around the neutral point of three. b. Users feel comfortable with editing other designs and having their designs edited. Overall, participants had no problem with an open form of collaboration that required them to let others adapt their designs. c. The project-tree gives users a good impression of the role they and other users play in the design process. The project-tree visualisation succeeds at highlighting contributors with many proposals or a few qualitative proposals, but fails to highlight qualitative commenters, or commenters with many comments when the tree grew large and complex.

to evaluate if our feedback system was a sufficient way for the design community to self-select their favourite proposal to then develop it further. In other words, the evaluation only indicated to us that this platform works good as an open platform for brainstorming ideas, resulting in a divergent project-tree. We were not able to prove that this divergent tree could converge back into a single design. Also lacking in our evaluation is a study on the impact of the ‘role’ system, since all participants had full permission to create proposals, comment and rate during the entire course of the evaluation. Finally, we noticed that our predefined assignment effected participants design behaviour, as the assignment did not resonate with some participants, whom decided to redefine the assignment to address issues they felt were more important. This shows the importance of letting a design team choose their own assignment in an open flat design collaboration. Future research should focus on improving the feedback system, and should research ways in which a divergent open design collaboration can re-converge into a single focused design direction. We believe that a dynamic way for the projecttree’s admin to change user roles during certain phases of the design may be a way to achieve this. We hope the concepts introduced in this research will form a basis to help future platform developers create more intuitive and accessible ways for open architectural and urban design collaboration.

We are pleased to see that the project-tree meets one of our main design goals: it is generally perceived as an easy to use interface to browse through proposals and comments, and helps users keep a clear overview over the design process. The main area that needs improving is the feedback system and the way the importance of proposals in the tree are visualised through it. In our collaborative open design, the best measure of importance did not come from a rating-system, but from the amount of feedback a design-community gives through comments and adaptations. It appears to us that the branches of the projecttree inadvertently serve as an effective means to visualise this type of importance. Due to the fact that we cut our evaluation short during the third design session, we were not able

- 106 -

- 107 -

- 108 -

Ontwerpstudio Arenberg: Appendix I Tutorial-tree & Design Brief Tutorial Tree “This tutorial-proposal will guide you trough the basic principles of this website. Just follow the instructions as indicated in the images above. Please note that you need sketchup 8 to complete this tutorial. If you are using a Mac, we recommend you use Firefox 4.0 and up instead of Safari. If you are using a PC, use Google Chrome or Firefox instead of Internet Explorer.” (quote from http://archibrain.org/content/tutorial_ proposal/tutorial) The following images were part of the ‘Tutorial tree’, and help explain the basic concepts behind the website

- 109 -

- 110 -

- 111 -

- 112 -

- 113 -

Designstudio Arenberg: Design Brief “The design ateliers provided for the architecture students in the castle of Arenberg are often crowded and full of maquettes. That’s why we will be designing an additional pavilion where students can work, design, share ideas or learn from their teachers. The pavilion will be located somewhere on the castle grounds. The pavilion needs to be an independent volume with a surface area of about 100-120m² (+/-50 students). Even though this pavilion will not be built, try and make the design as economically and structurally feasible as possible. Remember, affordability doesn’t necessarily imply boring design or bad quality! Bring in your own experience as an architecture student. In what kind of space would you like to work? Remember, this is a design by us for us! Though a pavilion for only 50 students sounds small, this design should provide an insight into the possibilities of an affordable, qualitative and gradual expansion of the design ateliers of our castle, one pavilion at a time.” (quote from http://archibrain.org/content/proposal/ assignment)

These images were part of the ‘assignment proposal’, and help explain the design brief.

- 114 -

- 115 -

- 116 -

Appendix II

Survey questions & focus-group topics

I.a Full overview BACKGROUND SURVEY 00- ”name” personal info 01- “gender 02- “age” 03- “occupation. If student, specify your current studies” experience with social media (multiple choice: never/rarely/monthly/weekly/daily) 04- “how often do you visit the following websites/webservices? -facebook/twitter -youtube -flickr/picasa -wikipedia -Google -Blogger (or other blogs) -gmail/hotmail (checkboxes) 05- “On which of the following websites/services do you have an account? -facebook/twitter -youtube -flickr/picasa -Blogger (or other blogging services) -gmail/hotmail -none of the above -other (multiple choice: never/rarely/monthly/weekly/daily) 06- “how often do you contribute content to the following websites? -facebook/twitter -youtube -flickr/picasa -wikipedia -Blogger (or other blogging services) design experience & designer profile (multiple choice) 07- “When I design architecture, my primary concern is... - form - function - construction - context - other 08- “When I design architecture, my secondary concern is... - form - function - construction - context - other 09- “When I design architecture, my last concern is… - form

- 117 -

- function - construction - context - other 10- “When I design in a group, I usually... -try to push my ideas -try to find the middle ground between my proposals and others -try to elaborate other participants ideas INTERMEDIATE SURVEYS (text-field) 00- “name” (likert scale) 01- ”Coming up with good proposals during this session was challenging” 02- ”I feel like I am part of a team” 03- ”I took the feedback that other testers give on my proposals (comments and rating) into account” 04- ”I enjoyed participating in this last session” 05- ”The proposals I made during this session have been important to the design process” 06- ”I appreciate the adaptations that have been made out of my proposals during this session” 07- ”I mainly contributed because I have to, not because I want to” 08- ”The comments I posted during this session have been important to the design process” 09- ”I appreciate the ratings and comments I got on my proposals during this session” 10- ”I appreciate the responses I got on my comments during this session” 11- ”My comments were taken into account by the other testers” (open questions) 12- ”What proposal do you think was the most popular during this session? Explain your answer.” 13- ”What proposal do you think was the least popular during this session? Explain your answer.” 14- ”What proposal do you think had the most important/interesting discussion (comments) during this session? Explain your answer.” 15- ”Who do you feel was the most important participant during this last session? Explain your answer.” 16- ”Who do you feel was the least important participant during this last session? Explain your answer.” (multiple choice) 17- ”what would you say is the main role you played during this session?” -I mainly introduced new ideas and concepts for the design -I mainly tried to find the middle ground between proposals -I mainly elaborated on previous proposals -I mainly gave feedback on proposals 18- “In this last session I mainly focused on the designs... - form - function - construction - context - other

- 118 -

FINAL SURVEY (text-field) 00- “name” (likert scale) 01- ”I felt like my proposals helped improve the design” 02- ”Seeing my proposals in the project-tree felt rewarding” 03- ”During the design process, I felt like I had a good overview of the different proposals that had been made ” 04- ”I felt that by commenting and rating, I could help improve the design” 05- ”I enjoyed getting feedback (good or bad) on my proposals” 06- ”I enjoyed working with others using this platform” 07- ”I enjoyed seeing what other people came up with” 08- ”Seeing my comments visualised in the project-tree felt rewarding” 09- ”I felt like rating and commenting on other proposals was important” 10- ”I didn’t mind when people edited my designs” 11- ”I felt comfortable editing and elaborating on other participants’ ideas” (open questions) 12- ”Did you feel like the project-tree was an accurate representation of the design process?” Why (not)? 13- ”Did other participants introduce you to new ideas, concepts or ways of working? Elaborate.” 14- ”What do you feel was your most important contribution to the design? Ex- plain your answer.” 15- ”Did the project-tree represent your contributions well? Why (not)?” 16- ”What visualisations (rating/chronological/recent comments/authorship) did you use the most on the project-tree and why?” FOCUS GROUP: The focusgroup was an open-ended discussion. These questions only served as guidelines for the discussion. A. Archibrain.org as a concept -”Who could benefit from a platform like this?” -”Are tools like these important for the architect profession?” -“What kind of skills do you think participants (designers and especially non-designers) can learn by collaborating on a platform like this?” -“how / in what situations would you use a platform like this?” -”In the long run, what do you think would be the impact of open strategies on architectural and urban design?” B. Usability issues -“What did you think of the interface? was it... ” -easy/difficult to understand? -beautiful/ugly to look at? -fun/boring to work with? -useful/useless to the design process? -”What features did you like?” -”What features would you remove/add? -“What were the main issues you encountered during the test”? C. The design -“Explain how you got to the final design from beginning to end. Elaborate on decisions that were made along the way, and point out the most important designs”

- 119 -

-”Focus on Individual designs: Why is this design popular/unpopular? Why was there so much/little feedback?” D. Personal design experience - “Did you feel comfortable with people adapting your ideas?” -”Have you learned something in terms of collaborative design?” -”Did seeing your contributions visualised feel rewarding? Why did/didn’t it?” -”Perceived performance vs. actual performance review: How important do you think you were to the design? What role do you think you played? How well do you feel your proposals were received? How important were the other participants?”

I.b Reliability of surveys The surveys were taken online via Google Docs ‘Form’ documents. The questions were composed by me (author) and approved by prof. Andrew Vande Moere (promotor). The questions were tested during a pilottest evaluation. In the opinion of the author they needed no change and were used unchanged during the actual evaluation. The number of participants was kept small (n=10). We can assume that this influences the reliability of the conclusions we can draw from quantitative questions in the survey like likert questions.

- 120 -

Appendix III Survey results

background survey

- 121 -

- 122 -

- 123 -

Intermediate survey 1

- 124 -

- 125 -

Intermediate survey 2

- 126 -

- 127 -

Intermediate survey 3

- 128 -

- 129 -

Final survey

- 130 -

- 131 -

- 132 -

Appendix IV Focus group transcription

A. Archibrain as a concept THOMAS VB: “Eerst had ik een paar vragen over de website in het algemeen en het concept er achter. In welke situaties denken jullie dat zo’n website gebruikt zou kunnen worden? Wie zou hier gebruik van maken? USER 9: “Ik denk dat dit wel goed is voor groepswerk, maar als je in een bureau werkt is er altijd iemand de baas, en dit kun je hier niet echt implementeren. Iemand die kan zeggen: ‘we gaan nu ‘die’ richting uit.’” USER 3: “Ik denk dat dat wel goed kan zijn. [Bij ons ontwerp] vandaag zijn we in zoveel richtingen gegaan dat het wat chaotisch werd. Voor beslissingen is het moeilijk” THOMAS VB: “Misschien moet ik even verduidelijken dat wat we vandaag gedaan hebben één van de mogelijke methoden is om de website te gebruiken. Ik was vandaag de ‘administrator’ van de boom, en ik had controle over wie wat kon doen tijdens het ontwerp. Het is dus wel een mogelijkheid om op bepaalde momenten tijdens het ontwerp participanten andere permissies te geven.” USER 7: “Ik denk dat het wel handig kan zijn als in een bureau twee mensen moeten samenwerken, en een iemand thuis halftijds thuis werkt. Zo kunnen ze toch ideeën verder uitwerken. Of bijvoorbeeld als twee bureaus [in verschillende landen] willen samen werken, kunnen ze [via de website] brainstormen.” THOMAS VB: “Zouden jullie het vooral in de conceptuele fase [van een ontwerp] gebruiken, of zien jullie het ook gebruikt worden in voorontwerp of in uitwerking, zij het dan [met striktere permissies voor de participerende ontwerpers]? USER 3: “Zoiets doet autoCAD nu al, niet? (cfr. autoCAD WS) USER 7: “Ik denk vooral voor mensen die meer en meer thuis willen werken en werk met hun gezin willen combineren, dit zeker handig kan zijn.” THOMAS VB: “Een ander concept achter de website is het idee dat het bepaalde principes van ‘open source’ software toepast op architectuur. Je kunt een project opstarten waarbij iedereen die wil meewerken kan meewerken, en zo zijn/haar kennis kan bijdragen aan het project. Vinden jullie dit een valabel principe voor architectuurontwerp?” USER 10: “Ik vind het vooral interessant dat iedereen ‘gelijk’ is op dit forum. Over het algemeen heb ja anders altijd meer ‘authoritaire’ ontwerpers die medewerkes hun ideeën direct afbreken. Het is dus interessant dat [deze website] meer ‘een open veld’ is waar elk idee gewoon wordt gewaardeerd op zijn manier, en dat er niet een dominant figuur is” USER 9: “Maar als je nu kijkt [naar ons ontwerp], zijn we gewoon op heel veel verschillende [ontwerpen] uitgekomen, allemaal ‘beginstukjes’ die op twee minuten in elkaar geflanst zijn en eigenlijk is er op niks echt concreet verder gegaan.” USER 2: “We hadden tenslotte maar een dag... “ USER 6: “Maar we hebben wel verschillende start-ideeën waarop verder gewerkt kon worden” USER 9: “Maar hoe wordt dan bepaald hoe hierop verder gewerkt moet worden? En op wat er verder gewerkt moet worden?” USER 10: “Het is misschien wat idealistisch om te verwachten dat alle ideeën naar een perfect ontwerp zullen samen komen” USER 1: “Je kunt wel tot 2 à 3 [gedachtegangen] komen” USER 2: “Dit was bij ons al wel duidelijk; [je had drie gedachtegangen]: de spanten, de circulatie-as, de boot. Je had eigenlijk drie richtingen waarin het ontwerp uitging” THOMAS VB: “Als er iemand de baas blijft van het project kan hij op een bepaald moment wel zeggen: ‘nu stoppen we [met het verkennen van ideeën] en proberen de concepten te combineren.” USER 2: “We zouden inderdaad nu na een halve dag ontwerpen kunnen zeggen van: “ok we gaan verder op de as, we gaan verder op iets anders... “ USER 9: “Ik vind dat je dan echt moet overgaan tot een echte discussie, face to face praten.”

- 133 -

USER 6: “[Commentaar geven op archibrain] werkte denk ik wel, het is gewoon iets anders als face to face, het zijn gewoon twee verschillende dingen denk ik.” USER 1: “Maar als je altijd moet wachten tot iemand antwoord [op een commentaar die je hebt geschreven] …” USER 10: “Maar eigenlijk is dit minder ‘ontwerpen’, maar meer een soort ‘ideeën-bad’ waar je inspiratie kunt opdoen [aan de hand van] andere mensen [hun projecten of commentaren], en waar je je eigen idee kunt toetsen.” USER 7: “Wat ik ook wel vrees is dat, als het gewoon online wordt gezwiert voor heel de wereld, dan zit je ook aan een paar tijdsrestricites. Als je even twee dagen je computer niet checkt, is [het ontwerp] opeens iets helemaal anders, en heb je een hoop gemist. Misschien moeten voor sommige projecten bepaalde [vaste werkmomenten voorzien worden], zodat iedereen op hetzelfde moment er op aan het werken is. Anders zit je met het probleem dat mensen er aan werken wanneer het hun uitkomt, en krijg je clusters van mensen [die het actiefste ontwerpen], waardoor anderen [met minder tijd] uit de boot vallen en niet evenveel kunnen meewerken. Voor hun wordt het heel moeilijk om te volgen en wordt het snel onoverzichtelijk. Je zag [bij ons ontwerp] bijvoorbeeld wel hoe ingewikkeld die boom kan worden.” USER 2: “Anderzijds is [op verschillende momenten werken] wel een pluspunt denk ik. Als je [een medewerken] in België hebt en een in Australië, kan een verder werken terwijl de andere slaapt.” USER 7: “Natuurlijk, in dat opzicht is dit wel goed, maar het moet nog wel georganiseerd kunnen worden qua timing.” USER 1: “Wat ik me afvroeg is of er geen beperking zou moeten zijn van het aantal mensen [die kunnen mee ontwerpen]?.” USER 6: “Het is misschien inderdaad vooral een goed systeem voor bureaus.” THOMAS VB: “Zoals ik eerder zij, kun je, wanneer je een boom maakt, zelf bepalen wie mag meedoen en welke permissies de deelnemers hebben.” THOMAS VB: ”Een andere mogenlijkheid van de site is om je ontwerpproces te communiceren met externe groepen. We hebben dit tijdens ons ontwerp niet gedaan, maar je zou bijvoorbeeld je ontwerpproces kunnen delen met omwonenden. Denken jullie dat dit via deze site zou kunnen werken?” USER 3: “[Omwonenden] kunnen dan ook voorstellen doen? In Antwerpen hebben ze zoiets gedaan bij [het ontwerp] van de kaaien. USER 2: “Maar dit was eerder toen een ontwerp al bijna af was.” USER 1: “Ja het ging toen meer om aanpassingen aan het ontwerp [aan de hand van voorstellen van omwonenden]” USER 3: “Het voorstel kwam van de architecten, buurtbewoners [konden er dan kritiek op geven], en architecten keken dan wat er aan het ontwerp aangepast moest worden.” THOMAS VB: “Zo kan je ook via [archibrain.org] werken, door omwoners toelating te geven om ‘comments’ te posten...” USER 2: “Wat we nu hebben gedaan zou ik bijvoorbeeld wel niet openstellen voor omwoners.” USER 7: “Ja, ik denk dat bepaalde beelden [niet-architecten] kunnen afschrikken. Bijvoorbeeld de afbeelding van de zeppelin: wat wij kunnen interpreteren als een conceptueel beeld, zien zij direct als realiteit.” USER 10: “Ik weet het niet, ik vind dat dit nog wel zou kunnen.” USER 1: “Het is een beetje zoals die wedstrijd voor de extra toren voor de kathedraal in Antwerpen. Daar werden ook de meest zotte conceptuele voorstellen voor gedaan.“ THOMAS VB: “Denk je dat de website een goed platform is om andere experts zoals ingeniers, sociologen, etc. bij het ontwerpproces te betrekken?” USER 6: “Dit zijn denk ik vooral doelgroepen die beroepsmatig al meer te maken hebben met architectuur, en daarom eenvoudiger zullen kunnen volgen wat er op de site gebeurt.”

- 134 -

B. Usability THOMAS VB: ”Wat vonden jullie van de interface? Was het moeilijk/eenvoudig te begrijpen? Duurde het lang voor jullie er mee weg waren? USER 9: “Ik vond het heel duidelijk. De boomstructuur geeft goed weer hoe [het ontwerp] evolueert. USER 1: “De boom laad wel een beetje traag.” USER 8: “Ik vond het visueel wel in orde.” USER 6: “Ik vond het best intuitief.” USER 9: “Ik vond wel dat bij het plaatsen van een comment, de comments die je nog niet gezien hebt in de boom rood zouden moeten worden. Zodat je weet wat je nog niet gezien hebt.” USER 1: “Inderdaad, of dat je bijvoorbeeld weet wanneer er op jouw ontwerp [comments geplaatst worden].” THOMAS VB: “Een soort van ‘wall’ (cfr. facebook) voor je projecten?” USER 1: “Ja inderdaad.” USER 7: “Je hebt ook soms comments bij moeder-en dochter-projecten, en vaak lopen die eigenlijk vrij parallel. Misschien moeten die op een of andere manier gelinkt worden. Vaak gaat het echt over hetzelfde, want die projecten zijn aan elkaar gerelateerd.” USER 3: “De volgorde van de comments vond ik niet heel intuitief; op fora komt het nieuwste altijd vanboven, en hier kan het nieuwste ook even goed vanonder staan [in het geval van een reply].” USER 9: (leest voor van een lijstje problemen) “Ik had ook een probleem bij het veranderen van mijn profiel foto. Deze veranderde niet bij de members pagina.” THOMAS VB: “Dit ligt waarschijnlijk aan de cache.” USER 9: “Ook duurde het laden van de boom vaak lang. Misschien kunnen de projecten [aan het begin van de boom] eenvoudiger voorgesteld worden.” USER 3: “Ja, of gewoon een file-path.” USER 2: “Ik vond het ook jammer dat de boom de instellingen van de parameters (rating/authorship/ recent comments) niet onthoud. Nu zet hij [wanneer je de pagina refreshed] enkel ‘rating’ automatisch op. Ik moest telkens weer gaan kijken naar ‘recent comments’ om me eraan te herinneren welke deze nu weer waren. [De boom] zou dus wel mogen onthouden [welke parameters ik op heb staan].” USER 3: “‘Recent comments’ zou ook enkel die mogen zijn die je nog niet hebt gelezen.” THOMAS VB: “Meer op de individuele gebruiker gericht, dus?” USER 3: “ja.” USER 9: “Ik had nog een andere opmerking. Je moet [bij het aanmaken van een proposal] twee afbeeldingen invoegen: de afbeelding voor je klein icoontje, en dan de afbeelding die je eigenlijk wilt zien. Wanneer je op een project klikt is het jammer dat je eerst de ‘icoon-afbeelding’ ziet, die niet zoveel zegt, en als tweede pas je goede afbeelding. Misschien moet dit omgewisseld worden?” USER 1: “Het zou ook handig zijn mocht je nadien nog beeldjes kunnen toevoegen.” USER 7: “Ja inderdaad, dat je het achteraf nog kunt aanpassen.” USER 3: “Want anders blijf je maar dingen bijmaken [omdat je je originele proposal niet kunt editen], wat de boom alleen maar onoverzichtelijker maakt.” USER 1: “Het zou ook leuk zijn als je afbeeldingen in een comment kunt toevoegen. Dit is vooral handig als je niet echt een nieuw idee hebt, [maar toch iets visueel wilt posten]. USER 7: “Voor referentie projecten bijvoorbeeld.” THOMAS VB: “Welke features van de website vonden jullie goed/handig?” USER 9: “Vooral de boom. Ik vond bijvoorbeeld de manier waarop iedereen een eigen kleur kreeg [door authoship aan te vinken] zeer handig.” USER 3: “Op zich stimuleerden die [auteurs-kleuren] wel.” USER 10: “Dat stimuleerde inderdaad enorm om bijvoorbeeld te commenten, [je ziet je bijdrage verschijnen]. USER 7: “Het was wel een beetje stresserend.” USER 10: “Inderdaad, soms voel je wel dat je jezelf een beetje moet bij-benen.”

- 135 -

THOMAS VB: ”Waren er features op de website die volgens jullie ontbraken? USER 9: “Een ‘like’ knop (cfr. facebook) voor proposals en comments.” USER 6: “...zodat je kunt zeggen dat je akkoord gaat met iemands uitspraak. Dat is beter dan dat je moet replyen [om je steun te betuigen]. Dan kun je meteen ook zien hoeveel mensen er akkoord gaan met een bepaalde uitspraak.” USER 3: “Dit is misschien ook een goede vervanging van de sterren ‘rating’ [op de proposals]. Nu is [de rating] altijd 3 sterren, of 1 of 5 sterren.” USER 2: “Soms vond ik 2 of 4 sterren best handig, bijvoorbeeld als je denkt dat er enkele goede elementen in een voorstel zitten, maar ook een paar fouten.” THOMAS VB: ”Waren er features op de website die volgens jullie overbodig waren? USER 9: “Nee, niet echt.” USER 10: “Nee.”

C. The Design THOMAS VB: “Wat waren volgens jullie de belangrijkste ontwerpstappen en ontwerpvoorstellen die genomen zijn tijdens het ontwerpproces?” USER 1: “Het is een beetje zoals we daarjuist al hadden gezegd: er waren eigenlijk drie stromen die ontstonden. USER 9: “Inderdaad, die boot op de Dijle, die circulatie-as door het park, en de paviljoenen opgebouwt uit spanten.” THOMAS VB: “Weten jullie nog hoe deze ideeën tot stand zijn gekomen?” USER 1: “User 6 was begonnen met die spanten.” USER 9: “User 3 was begonnen met [het concept van de circulatie-as door het park], niet? Daarop was dan wel veel commentaar gekomen dat er teveel paviljoenen waren.” USER 6: “Ik denk dat [het idee van de circulatie-as] eerst goed werd onthaald. Dan werd er verder over nagedacht en [begonnen mensen er meer hun twijfels over te hebben].” THOMAS VB: “Kunnen jullie de drie concepten even in het kort uitleggen?” USER 2: “Je had het concept van de spanten: een herhaling van spanten die kort bij elkaar staan, en dan worden er zo structuren geconstrueerd. In bepaalde concepten wisselden spanten constant van grootte, maar het basisconcept van herhalende spanten bleef.” USER 1: “Dit idee van spanten kont wel in bijna elk ander idee geïmplementeerd worden.” THOMAS VB: “En het idee van de circulatie-as?” USER 3: “Het concept was om een korte weg te maken naar de campus voor studenten, met paviljoenen ernaast [waardoor passanten de werkende studenten kunnen aanschouwen]. Op een bepaald moment was er een opmerking dat iedereen kan binnenkijken, maar waarom zou dit een ramp zijn? Architectuur vind ik iets eenvoudig voor buitenstaanders om naar te kijken; mensen zien graag maquettes. Ik vind het dan jammer dat er nooit iemad binnenkomt. Daarom vind ik dat [het paviljoen] wel op een publieke plek mag zijn, zodat mensen [je werk] kunnen zien.” USER 9: “Ja maar een maquette kunnen zien is een ding, maar als ja ‘s nachts met twee meisjes aan het ontwerpen bent, en iedereen naar jou kan kijken? Wie komt er niet allemaal in het park... ?” USER 3: “Maar je zit bijvoorbeeld ook een hele nacht in het fablab, allez... “ USER 9: “Ja dan doe ik de deur op slot, en weet ik dat niemand binnen geraakt.” USER 10: “Ik vond het idee met de circulatie-as nog altijd het meest interessante omdat daar het beste werd uitgewerkt hoe we ons willen positioneren in het park.” USER 1: “Ja het leuke is dat ideeën van andere paviljoenen hier op geënt kunnen worden.” USER 9: “Het nadeel was vooral dat er veel paviljoenen zouden komen en dat het park versnippert zou worden.” USER 2: “Ja, eerst was de as zelf het paviljoen, en toen kwamen de paviljoenen er langs.” USER 10: “Eigenlijk volgde het idee van de as niet echt de opdracht van de paviljoen.” USER 9: “De derde tak was het idee om een ‘boot’ te maken op Dijle.”

- 136 -

USER 2: “Ik vond vooral de vormgeving van de boot zeer aantrekkelijk.” USER 1: “Ik vond een belangrijk concept van de boot de tijdelijkheid ervan.” USER 7: “Het was het minst invasieve ontwerp ten opzichte van andere.” USER 2: “Voor mij was dit altijd de tak die wat alternatiever en conceptueler was. Er kwamen wel leuke ideeën maar ergens wist je dat je na een tijd toch naar een ander concept moest overstappen.” THOMAS VB: “Waren jullie verdeeld tijdens het ontwerpen? Er waren volgens jullie 3 stromingen in het ontwerp; waren er bepaalde mensen die vooral op bepaalde ideeën voortwerkten?” USER 9: “Ik denk dat wel, dat iedereen zich bij een van de drie ideeën heeft aangesloten.” USER 10: “Er was zo een grote spanten-groep. Wij (verwijst naar user 3) hadden een anti-spant groep (lacht).” USER 9: “Kunnen we anders niet eens vragen wie voor welk idee was?” THOMAS VB: “Misschien is dit geen slecht idee. We zullen de groep eens rondgaan.” USER 5: “Ik vond de boot wel goed. Wel, niet per se de boot, maar het idee om aan het water te werken. Ook het idee dat je het paviljoen achteraf kunt verplaatsen” USER 8: “Ik ben vooral met die spanten bezig geweest, maar ik vond het idee van de circulatie-as toch beter. Op zich combineerbare ideeën.” USER 2: “Ik vond de boot als concept het interessantste om op verder te werken, maar ik had er zelf niet veel aanvullingen voor. Daarom heb ik verder gewerkt op de spanten, als bouwtechnisch ontwerper [was dit ook meer op mij gericht]. Het viel precies wel op dat er een richting was met vooral spanten (bouwtechnisch ontwerp), een richting die meer conceptueel bezig was met de boot, en een concept op een meer stedenbouwkundige schaal, met de as.” USER 1: “Ik vond de boot wel interessant, omdat het een alternatief bood op het ‘volbouwen’ van het park.” USER 4: “Die as was stedenbouwkundig gezien wel, maar ik had wel de vrees dat het park daarmee snel vol zou staan vol paviljoenen. Ik vond de locatie van de boot wel goed; het moest voor mij niet noodzakelijk drijvend op het water zijn, maar op die locatie konden wel interessante dingen gedaan worden.” USER 3: “De spanten vond ik zelf iets te veel in detail gaan. De circulatie-as en de boot waren twee ideeën die niet echt met elkaar te combineren waren.” THOMAS VB: “Maar de circulatie-as was orgineel jouw idee?” USER 3: “ja.” USER 10: “In het begin waren we begonnen met een goede discussie waarin we in twijfel trokken of de archies wel permanent [in het kasteel] zouden zitten en of het misschien niet beter is als het paviljoen tijdelijk is. Tegelijkertijd was er ook de discussie over hoe we ons willen ‘exposeren’ naar [de andere studenten] en hoe we de relatie willen aangaan met andere departementen.” USER 3: “Ik denk dat zo’n discussie vooraf misschien meer doorgevoerd had moeten worden, opdat we meer in één richting zouden gaan. THOMAS VB: “Wanneer in het ontwerpproces denken jullie dat de drie hoofd-concepten zijn gevormd?” USER 2: “Tegen het eind van de eerste sessie, begin van de tweede, waren de meeste ideeën reeds gevormd. Er bleven wel onderling referenties en aftakkingen gemaakt worden nadat de drie concepten tot stand gekomen zijn. Het is niet dat we duidelijk op drie takken zijn blijven doorgaan op een rechte lijn.” USER 6: “Ik denk ook dat bijna iedereen een eerste [ontwerp] heeft gemaakt.” USER 7: “Intuïtief gezien vond ik de boot het beste, omdat het tijdelijk was en eventueel naar de stad kan verhuizen. Ik dacht wel dat het ergens nuttiger zou zijn voor dit experiment om iets concreters te hebben, daarom heb ik vooral voortgewerkt op de spanten. Ik dacht dit dan eventueel te combineren met het idee van de as, wat ik heel interessant vond omdat het eigenlijk ook de ontwerpvraag open trekt en de schaal uitvergroot. Nu was de sketchup enkel gegeven op de nabije kasteel omgeving, en het concept van de as trekt dit eigenlijk open tot het grotere park, en stelt zo de ontwerpopdracht wat in vraag. Ook het idee van de servers van user 3 vond ik zeer interessant, ook omdat het de ontwerpopdracht meer in vraag stelt in plaats van er direct op te vliegen.” USER 9: “Ik vond de locatie aan het water zeer interessant, omdat je daar de dynamiek hebt van het water en er op je gemak een beetje afgezonderd kunt zitten. Ik was niet zo’n voorstander van de as, omdat je nu al een goed pad hebt. Verder staat er ook al een villa, en ik denk niet dat [de inwoners]

- 137 -

graag zouden hebben dat er een weg langs hun tuin komt. Het beetje bos dat daar is zou door de as en de paviljoentjes ook nog meer uitgedunt worden. Het zorgt voor teveel bebouwing. Van de boot was ik niet zo’n voorstander omdat je drijft [en dit het ontwerpen kan belemmeren]”

D. Personal Design Experience THOMAS VB: “Waren julie comfortabel met het idee dat andere mensen jullie projecten aanpasten? Stoorde het dat je ‘intelectueel eigendom’ werd herwerkt door anderen?” USER 3: “Zolang het voor je eigen groep is stoort dit niet echt, zolang je dus zelf kunt bepalen wie er kan meewerken” USER 9: “Ik vind het juist irritant als er niet op je project wordt voortgewerkt. Dan weet je niet of er naar gekeken is of niet, en je weet niet of mensen het goed of slecht vinden.” THOMAS VB: “Denken jullie dat je iets bijgeleerd hebt op vlak van coöperatief ontwerp?” USER 3: “Best lastig dat we niet mochten praten.” USER 7: “Misschien moet je skype toelaten, of video-chat.” USER 2: “Op sommige momenten heb je echt wel meteen feedback nodig. Zodat je gewoon van idee kunt wisselen, in plaats van he telkens in sketchup te moeten zetten.” USER 5: “Anders moet je als eerste aanzet, als een ontwerp begint, er voor zorgen dat iedereen direct samenkomt om ideeën te wisselen.” USER 6: “Ik denk dat het voordeel hier wel is dat je in zekere zin verplicht bent om ideeën te vormen. Als je direct met iedereen samen zit zijn er misschien maar een paar mensen die met ideeën afkomen. Hier is het interessant dat iedereen wel een idee bijdraagt.” USER 7: “De [social performance visualisation] zorgde er op zich wel voor dat je ‘verplicht’ was om dingen te editen of commentaar te geven [opdat je niet onderpresteerde]” THOMAS VB: “Wat vonden jullie eigenlijk van de prestatie-grafiek? Heeft hij jullie ontwerpgedrag beinvloed? Zagen jullie het ontwerp door deze grafiek als meer ‘competitief’? USER 1: “Als u baas u daarop gaat betalen, op hetgeen dat je hebt geëdit... “ USER 7: “Ja het zorgt toch wel voor wat competitie” USER 1: “De motivatie die de grafiek gaf was wat dubbelzinnig, wat goed en slecht. Maar ja, we hebben ook maar een halve dag gehad natuurlijk.” USER 9: “Als je bijvoorbeeld weet dat je betaald wordt op basis van [zo’n prestatie-grafiek] dan ga je snel veel dingen editen zonder goede reden.” USER 1: “... zonder rekening te houden met de kwaliteit van wat je maakt.” USER 7: “Het houd ook niet veel rekening met hoe snel sommige mensen werken of hoe traag, en op welke manier iemand ontwerpt.” THOMAS VB: “Vonden jullie het belonend om jullie bijdrage (comments of proposals) te zien verschijnen in de boom? Werkte dit motiverend?” USER 7: “Ja, toch wel.” USER 1: “Ik vond het wel interessant om te zien dat er bepaalde projecten veel bolletjes (comments) hadden. Het was vooral interessant om te zien waar veel rond gediscussieerd is.” USER 7: “Ik vond het vooral belonend als er veel uit je voorstel ‘groeit’. De grootte van de bolletjes (afhankelijk van de rating) was allemaal wat hetzelfde.” USER 2: “Inderdaad, alle ratings lagen tussen de 2.5 en de 4 sterren percies.” USER 9: “Maar eigenlijk merkte je precies bij de commentaren dat alles zeer individualistisch werkt: Wat is mijn project? Wat voor een rating krijgt mijn project? Hoeveel commentaar krijg ik op mijn project?” USER 7: “Ik vond eerder dat dit platform aanzette tot samenwerken.” USER 6: “Het is ergens wel logisch dat je aan je eigen dingen redelijk wat vast houd, maar we keken ook naar andere dingen, en als er een tak was die we interessant vonden dan werkten we daar op voort. Dat is eigenlijk niet individualistisch, het is gewoon logisch. Als een project je niet interesseerd

- 138 -

dan is dat zo en werk je er niet op voort. Je merkt ook wel dat er vaak referenties bijkomen van een tak naar de andere, dus [er wordt zeker wel samengewerkt].” USER 7: “Uiteindelijk ben je gewoon trots op wat de groep heeft gemaakt in plaats van dat je eerder kijkt naar je eigen projecten.” USER 6: “Misschien is de ‘authorship’ knop hier wat een rem op omdat het focust op individuele bijdragen, en belemmerd het het samenwerken op het platform wat... “ USER 9: “Zou elk ontwerp dat wordt toegevoeg misschien niet beter anoniem zijn?” USER 7: “Daar dacht ik ook aan inderdaad...” USER 9: “Dat je wel ratings hebt, maar zonder dat je weet van wie het project is.” THOMAS VB: ”Zou je gemotiveerd zijn om mee te doen met een groepsontwerp als je er geen credit voor krijgt?” USER 3: “uiteindelijk gaat het vooral om de groep [die aan het ontwerp werkt], dus anoniem is misschien geen slecht idee.” USER 7: “Maar als je het anoniem doet wordt het wel heel onpersoonlijk. Anders kun je nog mensen met een naam aanspreken. Anoniem heeft wel zijn voordelen, want moest het ontwerp nu openstaan voor iedereen, dan krijg je zo dezelfde toestanden die je krijgt op elke internetfora; dat sommige mensen heel veel bijdragen en andere niet, waarbij bepaalde mensen hun mening veel meer gewaardeerd wordt dan andere. En je kunt van die ‘klikjes’ (vriendsjespolitiek) krijgen” USER 1: ”Langs de andere kant kan iemand anoniem wel heel veel hetzelfde idee promoten, zonder dat je kunt weten of de groep er achter staat, of slecht een persoon.” USER 10: “Van anonimiteit kan misbruikt gemaakt worden...” USER 6: “Misschien moet het gewoon zeker een keuzemogenlijkheid zijn voor de gene die de boom aanmaakt om te beslissen met wie hij wil samenwerken. Als je je samenwerkers kiest en kent is het soms net wel handig om te weten wie wat zegt.” THOMAS VB: “Maar stel dus dat je een ontwerp volledig open maakt zodat wie mee wil doen, mee mag doen. Gaan mensen gemotiveerd zijn als hun naam of pseudoniem niet wordt vermeld? USER 1: “Wat hier een belangrijke factor is is of medewerkers zelf baat hebben bij het project: als een project bijvoorbeeld hier in het kasteel is dan hebben we er allemaal baat bij. USER 2: “Een gevaar met anonimiteit is dat mensen alles kunnen zeggen wat ze willen, en bijvoorbeeld een project volledig kunnen beginnen afkraken... “ THOMAS VB: “Hoe belangrijk vond je jezelf tijdens het ontwerp? Wie denk je dat het belangrijkste is geweest in het ontwerp, of in een bepaalde ontwerptak?” USER 3: “Ikke!” (sarcastisch) USER 9: “Je had er die veel commentaar gaven of weinig commentaar gaven.” USER 2: “Ik merkte wel dat user 3 op bijna op alles commentaar gaf en user 5 ook. User 5 gaf ook heel veel ideeën Het is wel wat moeilijk te oordelen wie het belangrijkste is geweest omdat we niet echt tot een ‘eindpunt’ zijn gekomen waarmee we dit kunnen afwegen” USER 10: “Uiteindelijk denk ik dat er heel veel actieve deelnemers waren.” USER 1: “Het was wel duidelijk waar of met wie iets starte, maar vanaf dan [maakt dat allemaal niet zoveel meer uit]. USER 7: “Ik heb ook niet echt bij de comments gekeken wie wat zij, en ik heb ook niet echt kwantitatief zitten bijhouden hoeveel iemand zegt.” USER 10: “In het eerste stuk hoorden we user 4 niet zoveel, maar hij was dan ook iets aan het uitwerken.” USER 3: “Naar het einde toe zag je ook dat er meer en meer werd gerefereerd naar andere projecten, wat betekend dat ideeën vooral gecombineerd werden.”

- 139 -

- 140 -

acknowledgments This research would not have been possible without the help and support of many people. First of all I would like to thank my promotor and copromotor for giving me the freedom to explore my own topic and guiding me throughout the course of this research. I would like to thank everybody who helped in the development of archibrain.org: the alpha-testers, Tom B. and Pascaline L., the pilot testers Dries M., Berno B. and Bart L., and especially Stijn C., who helped me set up and maintain the website. Furthermore, I would like to thank all participants of our evaluation study: Thomas W., Nick V., Heidi S., Erik V., Inge B., Evelyne W., Simon J., Didier F. and BenoĂŽt D. Without them the evaluation of this platform would not have been possible. I would also like to thank my friends, family and housemates who gave valuable feedback, support and encouragement during the development of this research. Finally, I would like to thank Linda V.B. for patiently proof-reading this research.

- 141 -

- 142 -

References Images fig. 2.1: ENGELI, Maia (editor), Bits and Spaces: architecture and computing for physical, virtual, hybrid realms, 33 projects by architecture and CAAD, ETH Zurich, Bïrkhäuser Publishers for Architecture, Basel, Switzerland, 2001, p.48 fig. 2.2, fig. 2.3: CHASE, Scott et al., Gather ’round the Wiki-Tree, Architecture in Computro [26th eCAADe Conference Proceedings / ISBN 978-0-9541183-7-2], Antwerpen (Belgium), 17-20 September 2008, fig.5, fig.7 fig. 2.4: screencapture of <http://openarchitecturenetwork.org/>, Viewed 20 August 2011 fig. 2.5: screencapture of <http://www.openideo.com/>, Viewed 20 August 2011 fig. 2.6: screencapture of <http://www.openideo.com/open/impact/concepting/>, Viewed 20 August 2011 fig. 2.7: screencapture of <http://www.openideo.com/profiles/vincent>, Viewed 21 August 2011 fig. 2.8: screencapture of <http://www.openideo.com/open/impact/concepting/#collaboration-map>, Viewed 20 August 2011 fig. 2.9: GILBERT, Eric; KARAHALIOS Karrie; Using Social Visualisation to Motivate Social Production, Siebel Center for Comput. Sciences, University of Illinois, 2008, fig.1 fig. 2.10: SMITH, Marc A.; FIORE, Andrew, Visualization Components for Persistent Conversations, Microsoft Research, Redmond, USA, September 2000, fig.3 fig. 2.11: DIMICCO, Joan M.; PANDOLFO, Anna; BENDER, Influencing group participation with a Shared Display, MIT Media Lab, Massachusetts, USA, 2004

Tables table 3.2: PISANO, Gary P.; VERGANTI, Roberto; Which Kind of Collaboration Is Right for You?, Harvard Business Review, 2008 <http://hbr.org/2008/12/which-kind-of-collaboration-is-right-for-you/ar/1>

Books BYRON, Angela; BERRY, Addison; HAUG, Nathan; EATON, Jeff; WALKER, James; ROBBINS, Jeff, Using Drupal: Choosing and configuring Modules to Build Dynamic Websites, O’Reilly, Sebastopol USA, 2009 (12008) FRY, Ben, Visualising Data - Exploring and Explaining Data with the Processing Environment, O’Reilly, Sebastopol USA, 2008 (12007) BENKLER, Yochai, The Wealth of Networks: How Social Production Transfroms Markets and Freedom, Yale University Press, 2006 - 143 -

DAWKINS, Richard, Memes: the new replicators, In: The Selfish Gene, Oxford University Press, New York, 2006 (11976), pp.189-201 HIRSCHBERG, urs; ENGELI, Maia (editor), Phase (X), In: Bits and Spaces: architecture and computing for physical, virtual, hybrid realms, 33 projects by architecture and CAAD, ETH Zurich, Bïrkhäuser Publishers for Architecture, Basel, Switzerland, 2001

Articles CHASE, Scott; SCHULTZ, Ryan; BROUCHOUD, Jon , Gather ’round the Wiki-Tree, Architecture in Computro [26th eCAADe Conference Proceedings / ISBN 978-0-9541183-7-2], Antwerpen (Belgium), 17-20 September 2008, pp. 809-816 DIMICCO, Joan M.; PANDOLFO, Anna; BENDER, Walter, Influencing group participation with a Shared Display, MIT Media Lab, Massachusetts, USA, 2004 GILBERT, Eric; KARAHALIOS Karrie; Using Social Visualisation to Motivate Social Production, Siebel Center for Comput. Sciences, University of Illinois, 2008 GILBERT, Eric; KARAHALIOS Karrie; CodeSaw: A Social Visualization of Distributed Software Development, Siebel Center for Comput. Sciences, University of Illinois, 2007 GILES, Jim; Internet Encyclopaedias go head to head, Nature, s.l, Published online December 2005, <http://www.nature.com/nature/journal/v438/n7070/full/438900a.html> LUSCOMBE, Belinda; Ten questions for Renzo Piano, Time magazine, s.l., July 2011 <http://www.time.com/time/magazine/article/0,9171,2079576,00.html> MAHER, Mary Lou; PAULINI, Mercedes; MURTY, Paul, Scaling Up: From Individual Design to Collaborative Design to Collective Design, Design Computing and Cognition DCC’10, University of Sydney (Australia), 2010, pp. 581-600 PISANO, Gary P.; VERGANTI, Roberto; Which Kind of Collaboration Is Right for You?, Harvard Business Review, 2008 SMITH, Marc A.; FIORE, Andrew, Visualization Components for Persistent Conversations, Microsoft Research, Redmond, USA, September 2000

Other BERGHMANS, Tom; LANNOO, Pascaline, Fragments of a regenerated Nazareth: An urban reconnection, Master Thesis submitted to obtain the degree Master in Engeneering: Architecture, K.U.Leuven, 2011 WILKINS, John A., et al., Zen v.6.x-2.1, Theme for Drupal 6, Posted April 2011, first version posted October 11, <http://drupal.org/project/zen> BERG, Brendan, et al., Interfascia v.3.0 Alpha, Library for Processing, 2006, <http://www.superstable. net/interfascia/download.htm> X., et al., giCentre Utils v.3.2.0, Library for Processing, s.d., <http://gicentre.org/utils/#download>

- 144 -

Bibliography Books ALBRIGHT, S. Christian; WINSTON, Wayne L.; ZAPPE, Christopher J., The t-distribution, In: Data Analysis & Decision Making, South-Western, Mason, USA, 2009 (12006), pp.434-438 DAWKINS, Richard, Nice guys finish first, In: The Selfish Gene, Oxford University Press, New York, 2006 (11976), pp.202-233 INGELS, Bjarke, Yes is More - An Archicomic on Architectural Evolution, BIG ApS, Copenhagen, 22009 (12009) L. FRIEDMAN, Thomas, The World is Flat: A Brief History of the Twenty-First Century, Picador, New York,32007 (12005) NEUCKERMANS, Herman, Ontwerpmethodiek, Uitgeverij Acco, Leuven, 2008 (12001)

Articles CARRARA, Gianfranco; FIORAVANTI, Antonio , How to Construct an Audience in Collaborative Design - The Relationship Among which Actors in the Design Process, Architecture in the Network Society [22nd eCAADe Conference Proceedings / ISBN 0-9541183-2-4] Copenhagen (Denmark) 15-18 September 2004, pp. 426-434 FRANCIS, Sabu, Web Based Collaborative Architectural Practice Using a Fractal System, Predicting the Future [25th eCAADe Conference Proceedings / ISBN 978-0-9541183-6-5], Frankfurt am Main (Germany), 26-29 September 2007, pp. 727-734 HEYLIGHEN, Francis, The Evolution of Memes on the Network, Ars Electronica Catalogue, Ingrid Fischer (ed.), Vienna/New York, 1996 X., “Print me a Stradivarius”, in: The Economist, February 12th 2011, pp.11 X., “3D printing: The printed world”, in: The Economist, February 12th 2011, pp.69-71 PETZOLD, frank, RICHTER, Nancy, SCHNEIDER, Sven, A Matter of negotiation: Managing uncertainties in an open design process, Technical University Munich, Bauhaus-University Weimar (Germany), s.d. RIDLEY, Matt; LOW, S. Bobbi, Can Selfishness Save the Environment?, The Atlantic Monthly, September 1993, Volume 272, No. 3; pp. 76-86

- 145 -

Lectures RIDLEY, Matt, When ideas have sex, TEDGlobal 2010, Filmed July 2010, Posted July 2010, Viewed 20 March 2011 <http://www.ted.com/talks/lang/eng/matt_ridley_when_ideas_have_sex.html> BENKLER, Yochai, Yochai Benkler on the new open-source economics, TEDGlobal 2005, Filmed July 2005, Posted April 2009, Viewed 23 May 2011 <http://www.ted.com/talks/yochai_benkler_on_the_new_open_source_economics.html> BROWN, Tim, Tim Brown urges designers to think big, TEDGlobal 2009, Filmed July 2009, Posted September 2009, Viewed 27 March 2011 <http://www.ted.com/talks/tim_brown_urges_designers_to_think_big.html>

Tutorials X., Increase upload size in your php.ini, Posted December 2010, <http://drupal.org/node/97193> X., Bob, Drupal video podcast: Basic CSS layout, Posted July 2008, Drupal video podcast: Node references, Posted February 2010, Drupal video podcast: Display suite, Posted March2010, Drupal video podcast: Display suite advanced, Posted March 2010, <http://mustardseedmedia.com/podcast>

Web Processing, open source programming language and environment, available at: <http://processing.org/>, Drupal, open source content management platform, available at: <http://drupal.org/>

- 146 -

- 147 -