Architectural User Interfaces by Luis Hernan

I AU Architectural User Interfaces Luis Hernan

MSc Digital Architecture Newcastle University Student Number 110385121

The book will kill the building ... When you compare [architecture} to the idea, which ... needs only a sheet of paper, some ink and a pen, is it surprising that the human intellect should have deserted architecture for the printing press? (Victor hugo cited in Eco and CarriĂ¨re, 2012, p.vii)

The paper book will be replaced by a category of software that we can call virtual books (...) Changing such a central component of an old technology opens up so many new possibilities that we can truly say that a new technology has been born. (Kurzweil, 1992, p.141)

AUI Acknowledgements I would like to give a very special thank you to all the people that participated, somehow, in the creation of this work:

My mother, for all her support and encouragement in the peregrination of getting to this point. It hasnâ&#x20AC;&#x2122;t been easy. My siblings, in reverse order of appearance: Julio, Jaime and Claudia. For all their support throughout these years. And specially to Julio for keeping me well stocked with nothing but the best music. Dr. Martyn Dade-Robertson, my dissertation supervisor. For being an outstanding tutor, for giving me direction in the early genesis of this work, for showing me the area of the architecture of information and for all illuminating Kofi-Bar sessions. And most notably, for doing what only the best tutors can doâ&#x20AC;&#x201D;push the boundaries of their students. All the guys at Culture Lab, for letting me use all the cool stuff and gears, and for letting me conduct my studies in their premises. All my friends, for being there when needed. And exceptionally to Carolina, whose shared interest in architecture and digital information helped in keeping me going and in finding direction. For all the love and for always being there. For everything

Content Introduction

Literature Review

Hypothesis and objectives

Information systems and semantic space Embodiment Information artifacts and Affordances Epistemic actions Architectural User Interface Macroscope

Methodology Wicked problems Research through design (RtD) Rationale

Individual interviews General description Structure Findings Incremental implementation

18 18 21 22 22 25 27 27 28

32 33 33 34

36 39 40 47

Video-Provotype

Participatory Design

Initial Considerations General structure

Initial Considerations Structure Findings Discussion and further implementation

48 50

54 56 59

Video Prototype

Conclusions Bibliography Appendix

76 80 83

Lo-Fi Prototyping Aui Interactions Narrative

66 69 70

Introduction The information revolution is normally associated with the demise of everything printed. Historically the imagery of the future has been assembled with computer screens and glossy materials, with an immediate denial of anything paper for good measure.

For instance The Office of the Future, originally published in 1975 by BusinessWeek, embodied the augury of a future where the anachronistic paper would become a museum piece. The paperless office, a catchphrase encapsulating a vision of a more efficient future, where piles of papers would disappear altogether from any office to be substituted with networked electronic terminals. This notion has been recently challenged by Sellen and Harper (2003b), characterising this trend as The myth of the paperless office.

shadowed the development of technologies. The notion has been extrapolated to a concept of paperless home (Sellen and Harper, 2003b), and in the most shiny armour, of the paperless life. There is something irresistible about this pull, and has consequently informed our relationship with paper. Anything that is printed suddenly acquires a taste of the old, the anachronistic media hardly clinging to its existence. The principle at work here is as follows: regarding the physical as a burden and cherishing the abstract.

More than thirty years later, this work is still embodied in paper. However, the false augury of a future without paper has over-

Even when ubiquitous computing has enabled processing tasks to fade in the background of our attention, our informa-

There is something irresistible about this pull, and has consequently informed our relationship with paper.

Anything that

is printed suddenly acquires a taste of the old, the anachronistic media hardly clinging to its existence.

Chapter

tion ecologies still depend largely on physical material. Books are still being printed, reports are still reviewed in paper and magazines are struggling to find a business model to conflate both the digital and physical editions. Our current research environments can be as well characterised as mixed ecologies of physical and digital information artifacts. Therefore, our research endeavours are developed through a continuos negotiation between the affordances of digital and physical information systems. When analysing the affordances of both paper and digital information mediums, it becomes clear that both yield different opportunities that can be complemented. Digital copies allow a cheap and fast distribution of texts. Also, they afford the articulation of massive information datasets that can be mined and queried to explore how different texts relate to each other, ultimately delivering very sophisticated analysis on the information. On the other hand, printed mediums continue to afford people with a flexible organisational schemeâ&#x20AC;&#x201D;it is possible to annotate, highlight and analyse texts in a very personal way. Moreover, physical copies allow performing classification of information based on spatial interactions of different materials. For instance, book stacks suggest conceptual affinity across its members. Further, if two texts are to be analysed further in the future, they can be left open in a specific section, creating holding patterns that embodied cognitive processes. When working on a desk, paper-based artifacts can be loosely classified by plac-

ing them at different distances. The same is true when storing and classifying paper-based material in a personal shelf. Several voices in the field of Human-Computer Interaction have contested that is only a matter of technological development before digital information systems are able to emulate and evolve every affordance offered by paper-based artifacts (Kurzweil, 1992). However, physical information systems are based on a complex network of sensory experiences. This reality has shaped and influenced the development of human processes of sense-making, to the point where studies in cognitive psychology suggest human beings are naturally inclined to process information based on embodied interactions rather than in their metaphorical counterparts. (McCullough, 2005, DadeRobertson, 2011) This dissertation subscribes to the notion of creating richer research experiences by taking advantage of the spaces created in our current negotiation of mixed ecologies, as opposed to developing artifacts that encompasses all the affordances of digital and physical information systems. It will be argued that information needs are expressed not only through a set of keywords, but that they are inherently accompanied by a series of embodied and spatial gestures. The development of the first information systems required the support of graphic design to develop two dimensional interfaces that facilitated the interaction between users and a vast and complex abstract space of information. Such interconnection

lead to the emergence of the WIMP paradigm, whereby interaction was structured around Windows, Icons, Menus and Pointers. Such system prevails in modern-day information systems. Even when the advent of Ubiquituos computing has seen the explosion of Tangible User Interfaces (TUI), purpose built objects used to codify specific set of commands through embodied manipulation, our information systems are still heavily confined to screen space.

desks, shelvesâ&#x20AC;&#x201D;every component is analysed in its affordances to be filled or modified through information. Furthermore, information artifacts offer a complex set of patterns and relationships. Each term and fragment of information relates to the overall corpus by means of multidimensional semantic linkage, which is made manifest by users in the arrangement of information embodiment in space.

The notion this dissertation puts forward is that interaction with information systems has an architectural quality to it. We start of by creating the boundaries of our research spaceâ&#x20AC;&#x201D;piles of books are lined up to outline the spatial framework of our research. Some of us may even find comfort in the fact of feeling surrounded by the materials of information. Then we start to expand, reclaim every surface for the benefit of our cognition. Walls, floors,

Considering this conceptual framework, this research proposes to create a set of interactional behaviours that can be codified as command inputs in a digital information system. As a matter of scope it is proposed to use Macroscope, a visual knowledge network system, to implement the proposed set of interaction gestures.

Our current research environments can be as well characterised as mixed ecologies 11

The term artifact also denotes an anthropological connotation. It refers to objects made through the purposefully manipulation of matter.

Therefore, they are testament to the socio-technological system in which they were created tour,

1992).

(La-

Hypothesis and objectives

In light of the above, this research has established the following question: How can the interactional affordances of information artifacts be used to query information?

This research elaborates upon the notion of the Architectural User Interface (AUI), advanced by Dade-Robertson (2011, p.147-151) and understood on the greater movement of remediating digital information by restoring the sense of materiality in information artifacts. However, what are the elements that make the AUI different from previous paradigms of interaction? When extrapolating an evolution line from Graphic User Interfaces (GUI) through to Tangible User Interfaces (TUI), it can be concluded that AUI ought to be dependant upon the spatial interaction of the objects articulating the personal architectonic system.

Therefore rather than focusing on the manipulation of a single object, as TUI framework proposes, the core of this research will be to enact the codification of spatial

interaction by equipping existing artifacts with embodied computing devices.

Consequently the following hypothesis has been established to explore and partially answer this problem: Architectural User Interfaces (AUI), enabled by ubiquitous computing components, can enhance the interactional affordances of physical text documents with filtering and querying digital capabilities.

In an effort to articulate this research, the following objectives has been established:

question and outlines the methodology used.

• Understand and define the interac-

Chapter

three details the initial qualitative study performed,

tional practices in ecologies of mixed

and discuss the findings and incremental implementation

digital-analogical information arti-

to the next stage. In

facts.

ture and techniques involved in the development of the

• Understand how interactional gestures of information artifacts encode information patterning. • Map interactional gestures of information artifacts to semantic query actions in a digital information system. • Understand how spatial configuration of grouped artifacts encode information patterning

video provotype.

Chapter 4 we analyse the struc-

Chapter 5 concerns with describing

and reporting results from the bodystorming session.

In chapter 6 we develop the main insights from previous sections, and develop a framework for spatial interaction gestures. We then describe the development and structure of the Video Prototype created to illustrate the Architectural User Interface. Finally, chapter seven offers conclusions on the future of the AUI, and offers different research avenues to further this work.

• Map spatial iteration of grouped artifacts to semantic query actions in a digital information system.

• Develop a set of prototypes to enable participatory design and to embody findings in this research.

Information artifacts and digital resources

This dissertation has been divided into seven chapters.

Throughout this work, paper-based information materials will be referred to as information artifacts. This

In the first chapter, we review some of the relevant lit-

particular terminology was chosen to reflect the

erature and introduce the main concepts used later on

conceptual operating basis of this work.

this work.

mation in generally regarded as an abstract set of data,

The second chapter describes the research

Whilst infor-

the concept of artifact reveals a more user-centred perspective.

Books and paper-based materials are described, in the context of information science, as an information storage and retrieval technology that has been developed over centuries (Marchionini, 2011). However, it will be contested in this research that paper-based materials constitute representational objects that act as symbolic instances for human cognition.

The term artifact also denotes an anthropological connotation. It refers to objects made through the purposefully manipulation of matter. Therefore, they are testament to the socio-technological system in which they were created (Latour, 1992). Information artifacts not only embody a cultural and sociological intent, but also hold the potential to embed patinaâ&#x20AC;&#x201D;the unintended deposition of material as a result of interactions with the environment. Conversely, digital information material will be referred to as

Paper-based materials constitute representational objects that act as symbolic instances for human cognition.

Literature Review Information systems and semantic space Information systems are deeply related to the tools used in the process of knowledge storage and dissemination. For instance, the introduction of mechanically encoded objects signified a major paradigm shift in the prevalent models of information. Where abstract concepts were communicated orally, the creation of rolls, and later books, enacted the translation of immaterial information into a mechanical process of representation (Johnson, 1999, pp.1415). Likewise, the computer marked another transitional point in models of information systems. Technologically, computers represented a major breakthrough. Whilst books are representational object produced mechanically—by means of the deposition of ink into a paper medium— 16

computers represent a symbolic-based system that dissociates information from any embodiment. Information is mapped into strings of binary code, which cannot be accessed directly by human users. Media theorist Steven Johnson points out: ‘for the magic of digital revolution to take place, a computer must also represent itself to the user; in a language that the user understand’ (Ibid). Moreover, as the storage of information takes a fraction of the space required to record information on paper, digital information systems are largely perceived as being independent of any material mani-

Whilst books are representational object produced mechanicallyâ&#x20AC;&#x201D;by means of the deposition of ink into a paper mediumâ&#x20AC;&#x201D;computers represent a symbolic-based system that dissociates information from any embodiment.

Chapter

festation. Technically speaking, digital information resides in a storage device codified as binary code. In order to be accessible to human users, it needs to be interfaced—located, retrieved and represented—into a human-friendly language. In order to perform this set of operations, digital information needs to be addressed and structured. This feature facilitated the imagination of an abstract place inside the computers systems, a place where all the information transactions would take place. Generally this place was defined as cyberspace, and involved the understanding that it could be inhabited by human minds alike (McCullough, 2005, p.8-10). Dubbed a ‘consensual hallucination’ (Ibid), cyberspace potentially entailed the notion that humans could be liberated from the shackles of materiality in order to inhabit a pure space of ideas and the mind (Dade-Robertson, 2012, pp.67-70). The concept of cyberspace—as a parallel reality that could be immersively inhabited—would finally be dropped. In its place, the abstract space where information is stored is now considered as an abstract space articulated by the topological geometries resulting of the interconnection of information objects. An example of these topological geometries is the notion of hypertext. Originally advanced by Ted Nelson as ‘a body of written or pictorial material interconnected in such a complex way that it could not be conveniently be presented or represented on paper’ (Nelson, T. cited

in Dade-Robertson, 2011, p.82), hypertext represented a radical new approach to information classification. In classical systems of information storage and retrieval, such as libraries cataloguing systems, each information artifact is classified according to its conceptual similarity with other information objects. Hypertext, on the other hand, classifies information in smaller subunits. Each word and concept is considered as an independent information object, and connected directly to similar concepts. This system creates a complex web of connections across documents. Kaplan and Moulthorp analyse the structure of digital information and coin a new concept—semantic space. They define semantic space as being ‘deeply connected to the production of meaning, interpretation and other activities involving symbols’ (Kaplan and Moulthopr cited in Dade-Robertson, 2011, p.89). Semantic space is multidimensional, therefore holding a high level of complexity in its organisation (Dade-Robertson, 2011, pp.102-106). Consequently, users cannot access it directly. To address this problem, digital information systems are created to interface and transverse the semantic space. A user creates a query on the system, and a computational agent finds the related connections on the semantic space, returning only the fraction of interest to the user. In representing the semantic space to the user, computational systems resort to metaphors of embodiment. This is, they create representations of the information space

using meaningful constructions found in the tangible reality. The next section will give an account of the concept of embodiment, and will establish its importance in the context of this research.

Embodiment One of the primary cognitive resources of registering knowledge from the environment is the categorisation based on specific places. Specific information is associated to a particular place, and to its specific tangible qualities. The primitive would make extensive use of this system, embedding meaning to its environment by modifying it. A stone laid in a particular position in space would codify particular data that could be later grasped in order to recall it. This strategy of memory and place defines our mental structure and embodied disposition towards knowledge. This set of notions is at the core of cognitive psychology, concerned with understanding mental structures used in abstract tasks (Lakoff, 2008, pp.216-217). The notion that our cognitive processes are a result of a slow evolution shaped by physicality was advances by Lakoff and Johnson (Lakoff, 2008). According to this assertion, it is impossible to understand and analyse

the mind as an abstract concept, independent of the relationship between our bodies and the external world (Lakoff and Johnson, 1999, p.3). Johnson sustain this claim by looking at the notion of mental schemas, developed by Ulric Neisser: A schema is that portion of the entire perceptual cycle which is internal to the perceiver, modifiable by experience, and somehow specific to what is being perceived (...) a schema is part of the nervous system It is some active array of physiological structures and processes (Neisser cited in Johnson, 1990, pp.20-21). Schemas are explained as operating as pre-programmed grooves in a neuronal level. Thereby the same the same neurones used to control motor senses are also used in higher thinking and abstraction (Dade-Robertson, 2011) Mental schemas are notably related to architecture. Aesthetic theorist Jan MukaĹ&#x2122;ovskĂ˝ defined architecture as the fundamental human action of altering the surrounding environment. Such description bear a close resemblance to that of mental schemas: Architecture organizes space surrounding man. It organizes this space as a whole and

with respect to man in his entirety, that is with respect to all the physical or physic actions of which man is capable... (Mukařovský cited by Dade-Robertson, 2011, p.35) Considering the notion of architecture as a basic organisation of space, Dade-Robertson puts forward the concept of ‘cognitive architecture‘ (Dade-Robertson, 2011, pp.3637) and defines it as a functional tool that is similar to language. Under this premises, architecture is considered as a tool to organise and articulate meaning by means of spatial disposition. This function of architecture is consistent with evidence recollected by Durkheim and Mauss on primitive cultures in America, China and Australia. It is concluded that societies organise their villages as a way of constructing social identity (Ibid). The system used to enact meaning through spatial means is known as the architectonic system, defined by architectural theorist Donald Preziosi (1979, p.4) as a code relying on: a system of relationships manifested in material formations, and the medium (...) is normally a mosaic of shapes, relative sizes, colors, textures, and materials—in other words, anything drawn from the set of material resources (...) including our own and other bodies

However, it has not been defined the minimal unit of organisation under the architectonic system. According to Preziosi (1979) and Dade-Robertson (2011), the architectonic system is employed across scales to codify different layers of meaning. Thereby, the architectonic system is used in creating meaning out of the the particular position of an object in space—as this is relevant to the individual and the action of position encodes a specific meaning to it—or even to the complex layout of a city.

Information artifacts and Affordances Abigail Sellen uses the theory of affordances in The myth of the paperless office (2003a) to compare the possibilities yielded by information artifact and digital resources of information. Initially introduced by James J. Gibson, affordances are understood as opportunities for interaction exhibited by objects (Lakoff, 2008, pp.215-216). Affordances are made evident as physical traits, which are in turn perceived by users through the visual channel. Such traits are suggested to be the basis for the intuitive interaction schemes with objects (Sellen and Harper, 2003a, pp.17-18).

Current research environments can be characterised as mixed ecologies, where both information artifacts and digital resources interplay simultaneously. When understood through the theory of affordances, it can be concluded that each resource have unique affordances that can be complemented. Early in the introduction, this research stated that it subscribed to the belief that the most efficient research environments are those that conflate both information systems, physical and digital, instead of looking to create digital resources that comprise all the affordances. Moreover, an analysis of affordances for information artifacts is instrumental in understanding the complexities of human experience in cognitive processes of sense-making. The following sections analyse information artifacts and digital resources under the theory of affordances, and suggest common territory for a hybrid system of interaction.

Affordances of digital resources of information Since digital resources of information are not tied to a physical form of codification, several copies of the same

information material can be distributed. This allows the possibility for a cheap and fast distribution across users. However, this nature involves further challenges in copyright management. As an extension, the possibilities for appropriating information material by allowing full access and manipulation are rather limited. (MacFadyen, 2011, p.6) Also, the editorial process behind publishing is exponentially streamlined under digital models. Readers have the possibility to interact with writers nearly in real time, yielding also the possibility for writers to reach their audience without intermediates. (Birkerts, 2006, p.43). Interestingly, the time frame between production and distribution of texts is shortened, making it a fairly efficient process (Chartier, 1994). Semantic space also offers the possibility to statistically analyse large corpus of texts, therefore exploring connections between concepts and texts at an unprecedented rate (Dade-Robertson, 2011, pp.102-109). This process is also enhanced by the possibility of continuously performing analysis on an ever-growing corpus of information. As stated in the previous section, a computational agent traverses the semantic space to create connections meaningful to a specific information need. In this regard, two different agents can be identified. The first one returns a list of related results, which are then explored following a hypertext structure. Exam-

ples of this model are the modern online search engines. The second type of agents is the one that create visualisations based on the data. Knowledge networks falls under this category, which use spatial and topological relationships to encode data structure and meaning (Ibid). Finally, digital resources offer scalability. This means that new interaction features can be implemented in the future to expand or refined the existing affordances. However, scalability also refers to the possibility to incorporate other media of information, such as video, audio or interconnected resources.

Affordances of analogical instances of information First, interactional affordances offered by books are wide reaching. Based on the simple fact of their persistent physical presence, books afford the possibility to create holding patterns and categorise information based on the spatial location of the artifact. This feature is also true when considering the affordances of information artifacts ecologies. Meaning can be invested in the relative position of artifacts within each other and to the architectural space containing it (Sellen and Harper, 2003a, pp.188-

189). Furthermore the engagement with physical books is local and direct, which means the possibility to create bespoken physical filters of information. The way books are annotated and highlighted reflect different cognitive processes, giving new meaning to the same text. This property is also linked to the appropriation of information. Fundamentally, reading is a â&#x20AC;&#x2DC;practice embodied in acts, spaces and habits (...) all who can read texts do not read them in the same fashionâ&#x20AC;&#x2122; (Chartier, 1994, pp.3-4). Physicality of information artifacts allow user to engage into personal strategies of information management and sense making. This can be illustrated briefly by observing strategies based on the manipulation of information artifacts by use of stickers, notes and other devices throughout the text. It has been suggested (Sellen and Harper, 2003a, pp.188-189) that these practices aid in the appropriation of knowledge, and a greater exploration of connections across concepts. Physicality of information artifacts is also associated to the navigational affordances. The configuration of books allows fluid schemes of navigations, such as fast skipping, riffling and parallel manipulation of sections (Pattuelli and Debbie, 2010, Fowler and Baca, 2010, Sellen and Harper, 2003a). Similarly, embodied information allows physical navigation through collections of information artifacts, enhancing processes of memory.

Finally, information artifacts also hold the potential to store patina, or unintended encoding of information. Artifacts are transformed as a result of their interaction with the environment, thereby carrying residual meaning. A good example of this is when a well-thumbed copy provides information on the sections of interest for a specific community of readers. Moreover, a recent study (MacFadyen, 2011, p.2) reported that 43% of french students deem smell as an important aspect of their reading experience. Even when all of these affordances can find a digital metaphorical counterpart, it has been reported (Pattuelli and Debbie, 2010) that efforts to digitally codify the complexities of the reading experience generally fail. In part, this is explained by the fact that such counterparts add more layers of deception, which hardly can compete with the actual experience. In the following section, we introduce the concept of epistemic actions, which is closely related to the affordances of embodied information. It will be explored how the manipulation of information artifacts contribute in specific processes of sense-making.

Epistemic actions The theory of epistemic actions refers to the notion that people resort to physical actions in aiding their own cognitive processes. Such activities do not aim to manipulate the physical environment, but are symbolic resources to make mental computation easier (Kirsh and Maglio, 1994). Fast and Sedig (2010, pp.172-173) propose an example: when playing chess, a player might advance a piece temporarily to simulate the implications of a potential move. In pragmatic terms, the movement did not modify the systemâ&#x20AC;&#x2122;s state. However, the temporarily rearranged board affords the user to fragment the problem space between the physical space and the mind, thus reducing cognitive complexity. Interacting with external physical representations allows people to extend the processing potential of cognitive powers (2010, p.444). Cognitive processes are intuitively oriented towards cost-effective techniques (Kirsh, 2010, p.442). Thus when dealing with sensemaking in complex information systems, users will naturally recourse to any tool at hand to encoding meaning and find new patterns and relationships. For instance, creating piles of information artifacts have been found to be a effective and recurring strategy in identifying patterns and structuring work processes.

(Malone, 1983, Sellen and Harper, 2003a, Fast and Sedig, 2010). As pointed out by Fast and Sedig (2010), embodied interactions are based on root metaphors of how we use our body to interact and manipulate the environment. The method of root metaphors was originally introduced Stephen C. Pepper (1957) to explain cognitive processes of sense-making: A man desiring to understand the world looks about for a clue to its comprehension. He pitches upon some area of common-sense fact and tries if he cannot understand other areas in terms of this one. This original area becomes then his basic analogy or root metaphor. He describes as best he can the characteristics of this area, or, if you will, discriminates its structure. A list of its structural characteristics becomes his basic concepts of explanation and description (...) As a result of the impact of these other facts upon his categories, he may qualify and readjust the categories, so that a set of categories commonly changes and develops (Pepper, 1957, p.91). Our continuos process of shaping and understanding the world depends upon a set of basics categories and structures. Therefore using a root metaphor of similarity, elements are clustered together dependant upon their perceived degree of connection. Lakoff and Johnson (1999) take the method of root metaphors and develop the specific type of containment. Our primary perception of the world is structured as being outside the boundaries of our body. Thus, we perceive our bodies as containments. Con-

sequently, we experience the world as an entity outside our bodies, thus primarily defined in terms of boundaries. The root metaphor of containment shapes our experience and perception of the world. Lakoff and Johnson (1999) contend there is an active sense-making process involving the dynamic definition of abstract boundaries: ‘But even where there is no natural physical boundary that can be viewed as defining a container, we impose boundaries— making off territory so that it has an inside and a bounding surface—whether a wall a fence or an abstract line or plane’ (Lakoff and Johnson, 1999, p.29). As a consequence of the metaphor of containment, we physically place objects together in order to codify meaning of conceptual closeness or to suggest a category. The process comprises the perception of common properties, and the construction of an imaginary boundary defined by the physical presence of the classified object. This research contends that the metaphor of containment and boundary is constantly embodied in the research practice through the spatio-temporal manipulation of information artifacts. The interface paradigm advanced in this work takes advantage of the interactional affordances of information artifacts to create a system of physical/digital exploration of semantic connections. In the following sections we discuss some specific qualities of such systems, and discuss MACROSCOPE, a knowledge network software that has been taken as a basis to implement the Architectural User Interface (AUI).

Architectural User Interface This research started as an attempt to understand how digital information can be accessed and manipulated through the existing physical environment. It is of particular interest to enable users a richer exploratory experience of research materials through the possibility of embodied journeys and epistemic actions. There is an architectural quality immanent to the practice of research. We collect material, sort it and pattern space with information artifacts in a nearly unconscious level. We start off by creating the boundaries of our research space—piles of books are lined up to outline the spatial framework of our research. Some of us may even find comfort in the fact of feeling surrounded by the materials of information. Then we start to expand, reclaim every surface for the benefit of our cognition. Walls, floors, desks, shelves—every component is analysed in its affordances to be filled or modified through information artifacts. Furthermore, information artifacts offer a complex set of patterns and relationships. Each term and fragment of information relates to the overall corpus by means of multidimensional semantic linkage, which is made manifest by users in the arrangement of information embodiment in space.

structed whilst researching. One occurs at the material level—our interactions with the information material. The other is a semantic space constructed through the topological connections between ideas and concepts that will ultimately be embodied in a research paper. However, the embodied research outcome is merely the map—for the semantic space connecting all the information is the actual territory. Our interaction with digital information is normally mediated through various layers of interfaces. In order to address an information need, we normally use a keyboard and a mouse to type in a symbolic representation of our question. Then, the system returns a list of hits that would hopefully relate to our information need. However, in so doing a number of cognitive events take place. We use our bodies to retrieve and navigate information, as suggested by Fast and Sedig (Fast and Sedig, 2010), embodied interaction is a powerful tool in the process of sense-making. We then make use of the interactional affordances of information artifacts to sort and give meaning. It is after this process that we are in a position of abstracting our queries, translate them into a symbolic language and type it in the keyboard.

Additionally, there are two dimensions being con-

Macroscope Jointly developed by Dr. Dade-Robertson and Tom Schofield under the PATINA project, MACROSCOPE is a force directed visualisation tool that allows exploration of conceptual relations between different research papers. To this end, users can examine papers which are grouped after being analysed as a corpus by performing a Term Frequency Inverse Document Frequency (tf*idf) analysis (Dade-Robertson, 2012). The code is publicly available under Creative Commons Attribution-ShareAlike 3.0 Unported license (Schofield, 2012). The tf*idf technique is an algorithm used in the field of information science to assesses the similarity of a document by performing a weighted statistical analysis (Manning et al., 2008, pp.117-119). Each document, or vector, is assigned a discrete number of dimensions representing terms (words or phrases) and a corresponding weight calculated as a function of term-frequency and inversedocument frequency methods. (Idem) In terms of visualisation, MACROSCOPE group documents together and analysed to find relevant words within the same corpus. Then, each document is assigned a ranking value that represents its distinctness within the greater

corpus, which is codified into a two-dimensional representation based on a physics-driven particle system. The system refers to the concept of semantics of relative space (Dade-Robertson, 2011, pp.109-110) to convey graphical meaning to a system. Relative semantics involves that it is possible to invest meaning in a relative system by resorting to a set of three basic metaphors of embodimentâ&#x20AC;&#x201D;(1) Containment, whereby elements grouped together are intuitively thought of as semantically connected. Therefore, the overall meaning and definition is articulated directly by the relative location of members, (2) Closeness, the topographical relation of one element to the other describes a level of affinity between two members, i.e. the closest the elements, the more semantically connected they remain, (3) Connectivity, an explicit graphic linkage between members creates an evident relationship across them. The rationale for choosing this particular environment to develop the AUI lies in the exploratory capabilities it yields given the nature of its core algorithm. This research proposes an Architectural User Interface to Macroscope. The prototype, presented in the video prototype, is thought of as having the capability of performing real-time tf*idf analysis of physical information artifacts. Physical artifacts, ranging from printed copies of research papers through to books are regarded as proxies

equipped with a tagging technology. Several tagging technologies hold promising to enable physical interaction sensing in future developments of this research, such as Near Field Communication (NFC) (Sarmenta, 2012) and RFID. Therefore each artifact is connected to a digital copy that allows the system to perform statistical analysis on the text.

theories of sense-making. The concept of epistemic actions is instrumental in the dynamics of the AUI. The following chapter described the methodology that will be used to asses these concepts in the research practice of postgraduate studies. Further, a series of experiments will be performed, culminating in the illustration of the AUI in a video prototype.

In this chapter we have discussed the conceptual framework for the Architectural User Interface. The primacy of embodiment in our cognitive processes has been explained and furthered with

There is an architectural quality immanent to the practice of research. We collect material, sort it and pattern space with information artifacts 27

Methodology This research started as an attempt to understand how information can be accessed through the built environment. Thereby t seeks to couple interaction design with architectural experience and spaces. Given the complexity of such endeavour, there are a number of questions to be addressed: How do people combine digital and physical information artifacts? Far from the heralded future of a paperless information system, users implement mixed digital-physical ecologies in their information-related activities. However, it is unclear to what extend this situation 28

affects user experience and what interaction opportunities emerge from it. How the patterning of information is encoded through the interactional events of physical information artifacts? A number of previous research efforts (Marshall, 2008, Malone, 1983, Rimmer et al., 2008) provide valuable information on the matter. The study conducted by

Given

this arguments, this research will take a designerly approach to answer the research question:

How can the interactional affordances of information artifacts be used to query information?

Chapter

Malone (1983) is of special relevance given its specific focus on the spatial qualities of desk organisation. However his focus was limited to office settings and was aimed at producing graphic metaphors of the physical space. Additionally, research age hinders the applicability of their findings. It is therefore necessary to create a bespoken study on the matter. How the grouping of information artifacts, and their interaction in space reveal information patterning? Research conducted by Dr. Dade-Robertson (2011) offers a conceptual framework to approach this subject. However, it is necessary to further study their application on the system described in this research.

Wicked problems The analysis of the subset of studies reveals a high complexity underlying this research. As analysed by Fallman (Fallman, 2007), the use quantitative methods in HumanComputer interaction stems from a tradition of empiricism, objectivism and cognitivism. However, in order to perform quantitative experiments, it is necessary to identify a set of variables susceptible of being measured numerically. An initial analysis of the previous section reveals the im-

possibility to synthesise the set of enquiries into numerical terms. It is not possible to describe our interaction with information artifacts, the impact of a hybrid ecological information system through a standardised set of variables (Zimmerman et al., 2007, p.495). However, this fact should not suggest that they are impossible to solve, or plainly irrelevant. The problem poised by this research can be described as being representative of design research. The ‘unsolvable problem’ paradox steered conceptual revolution in the midst of design practice throughout the 1960s and 1970s. Stemming from this stance, Herbert Simon defined design problems as ‘wicked’ (Bayazit, 2004, p.21). His argument revolved around the notion that this sort of problems could not be defined in terms of goals, constrains or rules. Moreover, a partial response to the main problem would lead to a new set of unsolved problems (Zimmerman et al., 2007). The definition of design challenges as ‘wicked’ problems unravelled a greater paradigm shift in design. Previously it was considered plausible to develop a set of “rational methods incorporating scientific techniques and knowledge into design process to make rational decisions to adapt to the prevailing values”(Bayazit, 2004, p.19). Wicked problems allowed design to be redefined—a number of practitioners, commonly identified as the second design methods movement (Coyne, 2005), advocated for a new definition of design research that looked more into the ‘hummanness’ of the practice.

Human Computer Interaction has followed a similar path. Fallman (2007) identifies a school of thought characterised as post-cognitivistic, concerned more with capturing personal and qualitative experience in interaction, rather than objective and standardised measurements of usability: These researchers tend to be interested in the relationship between user and artifact in terms of for instance that relationship’s affective qualities rather than efficiency; meaning rather than various performance metrics; fun and playability rather than error rate; and sociability rather than learnability, and so on. (Fallman, 2007, p.295)

Research through design (RtD) Dalsgaard (2010) likens Research through Design (RtD) methodology to design-oriented research. In so doing, he defines it as “research in which a designerly approach and perspective is employed by the researcher” (Dalsgaard, 2010, p.201), In a similar token, a qualitative study conducted by Zimmerman (2010) found that

RtD is regarded as the single most valuable contribution form the design practice to the broader research community. Given the unspecific nature of the adjective designerly, it is unsurprising that the definition of RtD still remains blur. Designerly approaches reject the notion that activities related to design are, indeed, reducible to logical proposition and quantitative metrics. There is not ‘objective system’ (Cross, 2001) to tackle design problems. Cross (Idem) ascribes this notion to the work of Donald Schön, who advocated for the recognition of the messy and problematic nature of problems studied by design. He later proposed a reflective practice, which would later be appropriated as designerly thinking. This kind of thinking contemplates ‘an epistemology of practice implicit in the artistic, intuitive processes which some practitioners do bring to situations of uncertainty, instability, uniqueness, and value conflict’ (Cross, 2001, p.4) Zimmerman (2010, p.310) elaborates on this character of reflective practice when defining research through design as a collection of methods and that employ design processes and products as a ‘legitimate method of enquiry’(Idem). In general terms, RtD compiles a range of theories and practices from different backgrounds in an iterative scheme, which allows designers to continuously reframe solutions towards a nearly optimal condition. The nearly optimal character is stressed by Gaver (2012), who considers that RtD creates provisional, contingent and aspirational theories. Therefore, 31

it operates as â&#x20AC;&#x2DC;antidote to assumptions that technology should provide clear, efficient solutions to practical problemsâ&#x20AC;&#x2122; (Gaver, 2012, p.938) Additionally, Chistopher Frayling (1993) finds yet another convergence feature for all the disciplines undertaking RtDâ&#x20AC;&#x201D;the design artifact. RtD employs design artifacts as a technique to investigate different aspects of the research. They embody a set of philosophical, functional, social and aesthetic decision that converges into knowledge (Frayling, 1993, p.5). In arguing the epistemological distance between science and design, Stolterman (2008) advances his notion of design artifacts as ultimate particulars. Science concerns itself with the study of the natural world, and constructs logical models to explain the laws governing its interactions. In this sense, it solves the puzzle of reality, and through that finds the truth (Kuhn cited by Gaver, 2012, p. 939). Ultimate particulars are, on the other hand, the equivalent concept of truth in the field of design (Stolterman, 2008, p.59). Given this arguments, this research will take a designerly approach to answer the research question: How can the interactional affordances of information artifacts be used to query information?

Additionally, it will follow a Design through Research framework. The specific design of techniques to be implemented is detailed in the following section.

Rationale Dalsgaard (2010) suggests that Research through Design should be considered an iterative set of process leading towards a research outcome. This proposition involves that the researcher should approach the study with a defined structure of activities, methods and expected outcomes. However, there should also be room to adaptation and refining. Following the diagram suggested in (Dalsgaard, 2010), the design of activities for this research has been structured around two groups: artifacts and activities (Figure 3.1). In principle, information gathered through the activities phase will serve to modify the initial design of design artifacts proposed.

Video Provotype

Artifacts

Activities

Initial Interviews

Video prototype

BodyStorming

Figure 3.1

Initial

Development Participatory Development

individual interview with seven participants.

Development of a video provotype to elicit discussion in bodystorming event

design event, commonly known as bodystorming.

of a video-prototype. A video prototype was be developed to showcase the interaction design yielded from the previous stages of this research.

Individual interviews General description A series of open-ended in-depth exploratory individual interviews were performed following a Grounded Theory Approach. Evidence of previous work in the area following the grounded theory approach can be found in (Thom, 2010)(Nichols and Cunningham, 2009, Marshall, 2008, Malone, 1983, Rimmer et al., 2008). In following the methodological notes proposed by Holstein (Holstein and Gubrium, 2003, pp.311-315) , a general approach of topics and base questions have been delineated. However, the interviewing process follows an emergent technique, relying on the ability of the researcher to identify and pursue interesting leads. Questions have been designed to initiate conversation on the topic and invite interviewee to expand and reflect on their answers. Data was captured using a video-camera supplemented with an 34

audio recorder and a note taking device (notebook). Participants are archetypical of postgraduate studies in the UK. Eight students were asked to take part in the study, both male and female and from all demographic backgrounds. This study does not claim to study a representative sample of participants, which is unfeasible given the time allocated. The selection of participants has intended to create a diverse group to best capture a picture of interaction with

I would have a stack of books, immediate stack of books, where

I would just

go and grab and open and get a quote.

But I always,

always,have the books with me because it makes me feel secure. [Participant C]

Chapter

and Gubrium, 2003, pp.311-315) , a general approach of topics and base questions have been delineated. However, the interviewing process follows an emergent technique, relying on the ability of the researcher to identify and pursue interesting leads. Questions have been designed to initiate conversation on the topic and invite interviewee to expand and reflect on their answers. Data was captured using a video-camera supplemented with an audio recorder and a note taking device (notebook). Eighta students were asked form. to take part information artifacts.Participants are archetypical of postgraduate studies in the UK.sign confidentiality

in the study, both male and female and from all demographic backgrounds. This study does not claim to study a representative sample of participants, which is unfeasible given the time allocated. The Each participant has been assigned a letter codifytoand shows background information selection of participants hasto intended create a diverse groupTable to best 1.1 capture a picture of interaction with information artifacts. relate answers. Additionally, each interviewee was asked to pants.

on the partici-

Each participant has been assigned a letter to codify and relate answers. Additionally, each interviewee was asked to sign a confidentiality form. Table 1.1 shows background information on the participants.

PARTICIPANT

FACULTY

GENDER

Participant A

School of Architecture, Planning and Landscape

Female

Participant B

School of Arts and Culture

Female

Participant C

School of Architecture, Planning and Landscape

Female

Participant D

School of Architecture, Planning and Landscape

Male

Participant E

School of Education, Communication and Language Sciences

Female

Participant F

Business school

Male

Participant G

Science, Agriculture and Engineering

Male

Table 4.1

Structure In order to address the subject, three main events in research practice has been identified. The first one concerns with the initial approach to the information artifact. A second event concerns with the creation of mental and conceptual nodes of the information. A third moment will be the process of writing the final piece of research. These milestones were roughly signalled during the interview. At the beginning of the interview, the researcher offered a brief explanation to the interviewer. In order to avoid any unintentional bias, interviewers were explained that the project concerned with research practices and the use of books and digital sources of information. A first set of question enquires on the practices whilst approaching the information artifacts for the first time. Special attention was paid to the set of spatial interactions of information artifacts. • First, could you describe the physical space where you do research? • Do you like to consult more than one book at the same time? • Once you have located an interesting piece of information, how do you highlight it. What materials do you use?

• When you have both physical and electronic, do you consult them at the same time? A second section in the interview introduced the activities related to the networking moment of research • How do you make notes on the information you are reading? Do you use a notebook? or do you use a digital tool? • How do you organise the space around you? Do you have a particular code to sort books given its relevance to the topic? • In the context of having bits and pieces of information scattered across different books. Do you prefer to have them in the same desk that you are working? Or do you prefer to store them in other location and retrieve them as you need them? A third moment was concern with the practices associated with writing up and preparation of the final research outcome • Tell me about your experience in transferring text from a book to the computer.

â&#x20AC;˘ Do you prefer to have the relevant books in the same desk? Or do you prefer to have them stored in another location?

Findings In this part of the chapter we analyse some data obtained through the series of interviews. The actual approach to data gathering was a mixture of direct observation, and detailed analysis of audio and video captured. Given the scope of this research and the time frame allocated, it was not possible to provide a full transcription of all the interviews. Instead, the whole series were initially analysed and individually categorised. Once main categories for each interview were allocated, interviews were cross-referenced to build up general themes that emerged through the whole body of interviews. Using this framework, interviews were further analysed in detail. Specific fragments of qualitative data were then transcribed to support each category. Captured qualitative data fall crudely into four categories: (1) Memory and embodiment, (2) Architectonic system and surfaces, (3) Information artifacts embody information filters, and (4) Information need related to media. In the following sections we analyse each category and provide direct quotes from the interviewees.

Memory and embodiment The first topic that emerged through the interviews was the use of personal schemes to create memory tokens based on physical places. As suggested by Dade-Robertson (2011, pp. 102-111), memory is often associated with physical places. Participants suggested that their research efforts are often supported by different memory tokens, which normally take the form of actual physical places. On other cases, physical traits are used in aiding memory processes, such as post-its or page-markers. In this regard, Participant F mentioned: If I take multiple information from a book, then if I have to refer back to those book again. I usually put this markers. The next day when Iâ&#x20AC;&#x2122;m looking for information I always pick up these books, open those relevant pages, put them all around me. [Participant F]

Information is embodied either through notes placed on the actual book, or by extracting the information into a paper-based medium. In some cases, these practices have been reported to help participants in creating semantic networks during early stages of research. Relevant information, which is found scattered across all the research materials, is conceptually connected in an abstract network. For instance, Participant D described creating a

physical diagram to embody the semantic structure of the available information. I like to draw the kind of structure, the literature structure or the literature map on the notebooks. Sometimes I make some diagrams around it. When I start reading, I just draw lines on the book. If I find that the sentence is quite meaningful for me I just copy it to my notebook. At the end, after this process I just take all these pages out of my notebook and stick it on the wall. Make it like a physical diagram. By now, I have a kind of knowledge web [Participant D]

The importance of physical place as memory aid is also made evident in the generalised preference of physical artifacts over digital sources of information. Out of seven participants, six preferred physical material to digital. When enquired on the reasons, participants pointed at memory and mental clarity. Digital resources, it turns out, tend to be forgotten by the participants: In pdf, you cannot remember in which paper you found that information. If you don’t put the number directly and don’t note it down you wont find it again—it’s too difficult to find it. Because you read so many things and you’re so confused and if you don’t code them correctly, you lose them. [Participant A]

Embodied information is also correlated to mental clar-

ity. It is reported that physical information contribute to a more efficient process of sense-making: I don’t like to see the screen to search information. Because I cannot see each page. You can only see one page, I like to see several pages simultaneously. So I need to see several physical materials at the same time.[Participant D]

In some cases, information is embodied through digital means. Participants report to display digital information consistently through their research activities, therefore creating a persistent physical presence: If I cannot print or save a file, what I usually do I just keep the page open all the time, and unless my computer crashes and I loose it, I always have it there. Then I know that there’s that information and just pick it up from there. [Participant F]

Embodiment and physicality are reported as important features to remember where a particular piece of information can be retrieved. As suggested by Sellen et.al (2003a), physical artifacts are spatially arranged to create memory holding patterns that reflect a mental process: There’s always a point where I don’t know how can I write this sentence, how can I go on (...) So very often I have like three books in front of me, opened. So I have the first book open in front

of me, and beneath two other books which I might need at the same time. All of them opened somehow.â&#x20AC;&#x2122; [Participant E]

It is reported that even when digital resources represent a very efficient means for getting information, they fall short when enabling readers to remember the information. The importance of embodiment is not only observed when information artifacts are used as memory tokens, but also when their spatio-temporal properties encode meaning and semantic connections. These concepts are discussed in the following section.

Architectonic system and surfaces Another feature that is closely related to place and memory is the architectonic system. In the introduction to this work, we discussed that the fundamental principle of architecture is the patterning of the environment in meaningful schemes. This concept can be extended to imply that people inherently codify information when placing object in space. In this sense, the series of interviews gave an interesting insight into how architecture is related to research practices. To start with, participants described a patterning of their immediate environment as a consequence of their

research consequences. It is reported that activities such as tidying up coincide with crucial moments in research, such as starting the exploration of a new topic or commencing to write up a report. The impact of the physical space on the cognitive processes goes on as to suggest that one is a consequence of the other. As means of illustration, Participant A narrated a series of activities that can be better described as a ritual: I like to start totally empty. In the beginning is just me and my laptop, maybe a notebook to keep notes. And then by researching this desk start having more things (...)I cannot work if the house is not clean. I want to everything to be tidy, otherwise that make me confused. [Participant A]

Further, there is evidence to suggest that the spatial quality of an information artifact does indeed codify meaning. For instance, materials that are placed in the near proximity of the researcher are those specially relevant to the topic that is being written. This spatial strategy changes over time as different topics are addressed throughout the research: Itâ&#x20AC;&#x2122;s not according to the subject, but to the importance. I mean which one I currently use, those books are in front of me and the rest are a bit far over there and then I replace them. (...) I mean itâ&#x20AC;&#x2122;s happening, even subconsciously I do that. If I am writing about this certain subject, of course, those books are closer to me here and there. With the time, when I switch to

another topic, I take the books that were there and replace them with the new ones (...) [Participant B]

Closeness to information artifacts is also linked to a sense of security. In some cases, position of information artifacts not only codifies conceptual closeness, but also plays an important role in the emotional construction of the subject. Participant C reported constructing two stacks of books, one of which is for the benefit of feeling secure. Books allocated in the second stack do not really play a role in the active information retrieval, but embody a background conceptual framework. This pile is physically located far from the subject, thus suggesting the loose conceptual link to the active research exercise, yet in a strategic position as to pattern a personal atmosphere: I would have a stack of books, immediate stack of books, where I would just go and grab and open and get a quote. And then I would have another stack of books at my right side, behind me. I would probably pop them there just for the sake of having them there so that I can refer back to them when I need to. But I never do. Itâ&#x20AC;&#x2122;s like a stack of safety books that I would go and look at if I ever feel the need to. And then the ones that I need most are always on my left hand side, near me. But I always, always, always have the books with me because it makes me feel secure. [Participant C]

It can be gathered from the testimonials that stockpiles constitute a patterning resource to classify information material. In the cases analysed so far, each pile represents a varying degree of attention ranking. In other cases, books that are pilled together share a common subject classification. As discussed in the introductory chapter of this work, piles are connected to the root metaphor of containment. When places in proximity, books define an abstract boundary for a new category. Furthermore piles are also employed in defining physical boundaries for the research space, as illustrated in the experiences narrated by Participant C. Another important aspect is the appropriation of existing surfaces. In all the cases, participants described their research spaces as being a mixture of bedroom and workspace, thus involving a limitation on the actual available space to handle information artifacts. Instead of containing the research practice to the surface of desks and shelves, other architectural elements are appropriated and transformed to pattern information. Beds, floors and even walls are used to create a suitable space for knowledge networks. Participant E mentioned: I have one stockpile with marketing books, other with culture books. So I organise it a bit like this. For the articles basically I donâ&#x20AC;&#x2122;t have enough space. If I have thirty printouts of articles and then my room is full of articles I just put them like this [claps

both hands as to signal a pile]. My room is quite small and when I write an essay it’s so crowded. I try to order everything but in the end, sometimes, if I’m working on a specific essay and I need two journal articles at the same time, or three, I just put them sometimes in the floor because I don’t have space on my desk. It’s such a small desk, or I put it on my bed, which is close to my bed. So I just put it like these and I have a look and I say ‘Oh, I can write this and this” and then just type it. Sometimes is really a problem with the space [Participant E]

The appropriation of space is linked to the embodiment of research. The patterning of space through information artifacts can only be understood as related to the human body. In some cases, research specific activities are carried out depending on the relative position of information artifacts. Usually at the beginning, when I am trying to find the information I do not organise them at all. Because some of them I will discard (...) when you have your idea, I start to organise and put the numbers. I have the information in the floor, spread out, so you can see the post it. So I am usually sitting on the floor to have a very close contact with the papers. [Participant A]

It is also expressed how information artifacts afford flexible information retrieval and exploration. Participant D is specially keen on information artifacts as he can consult them whilst carrying out other activities.

Recently, I wanted to start reading a book, which is a ebook of 300 pages. I didn’t want to print it, so I tried to look it on the screen. There are two problems when I’m reading ebooks. One problem is that my reading speed is very slow. When I’m reading a physical book, I can read it in the toilet, in the bath, wherever I want. The other problem is that I cannot jump form the page 100 to the page 20. I have to slide.[Participant D]

Again, a pattern of appropriation is observed. A highly programmatic space as the bathroom is also shaped in terms of information artifacts.

Information artifacts embody information filters Information artifacts also embody unintended information filters. Sellen and Harper advanced that: ‘the fact that paper affords rich variegated marks that are persistent and static, also has a variety of different implications for perception and action in work situations’. A similar phenomenon is observed in research related activities. Participants report that marked books affect the way they perceive information. When asked on whether existing marks were taken into account in their research process, Participant C mentioned: Sometimes, it depends on how knowledgable the

person who read the book before you were. Because sometimes you read a book and then the person who was probably looking at the same things that you would highlight the points that you’re interested. When you read a book like that, you skip all the things that are not highlighted. If you read a book which is highlighted and you think that that point should be highlighted then you think ‘Oh this person thinks the same as me’. (...) Sometimes I would go for a rubber and rub the whole thing off because it can be very distracting having the whole thing highlighted. [Participant C]

What is interesting is how these marks leave patterns that are followed by the next reader, particularly in library books. In some cases, participants reported to skim the text based on the highlighted versions, or even being directed to other sources based on the marks left on the sides of pages. It is not only the case that artifacts pattern space, but also artifacts exhibit a semantic architecture of its own.

Information need related to media Of particular interest is the emergence of patterns on media choice based on information needs. All the participants reported a well-structured research scheme, where digital and information artifacts were used in

specific moments of the research. For instance, some people preferred digital resources to physical artifacts in the first moments of research. This pattern is associated to the pressing need of obtaining very precise information to further the research exploration. First I start with Google. If the ideas are new to me, I start with wikipedia to understand exactly what they mean. Then I go to JSTOR to search papers that are more accurate, more scientific. In the beginning i just skim basic things to understand, you know? Then I write down the important things and then start looking for them. In the beginning everything is very fluid [Participant A].

These behaviors reveal another pattern. Generally digital information is regarded as transient. An information flow was suggested, whereby information is initially ranked through digital resources. For example, if a particular article was considered extremely relevant to the researched topic, it was printed out. Otherwise, it was stored in digital version. I first have a look through the articles, I check the PDF and scroll down and have a look at the article. If the article sounds really helpful to me, I print it always out because I really need to have it in hard copy in front of me. I need to mark it like(...) For me it’s really difficult. I don’t like it too much to have all the time only the computer screen. I think it’s also for my eyes. I feel it not very comfortable.

If you spend like hours in front of the screen. I am a person who really need to have the paper in my hand. If I know the article is important, I print it out for sure, then I take the text marker to highlight the most important things [Participant E].

Perception on the primacy of information artifacts over digital resources is linked to the navigational affordances of information artifacts. In some activities, such as locating specific terms within a text, digital information is preferred: ‘ You can use the search field within the PDF so you search for certain keywords (...) when you just search for a word within a text it is mentioned. It’s much quicker that printing it out and having a look through everything on my own’ [Participant E]. In other cases, paper is reported to be a better choice, specially when extended reading is involved. Moreover, interactions such as place-holding with one hand one section whilst rifling through another is specially valued in exploring information in detail: If the paper is very relative to the thing that I am searching, I print it. If its not very important, I keep it as a PDF. If its a small paper, for example four pages, I print it also. If its too big, for example 200 pages, I keep it in my desktop—I don’t print them. (...) I cannot read the PDF, sometimes I need the thing, I need the paper. I feel more concentrated, It’s more close to me. I have my pen, you can feel it, its flexible. So if you want to rest, you go to bed and take the paper with you. [Participant C]

When asked on the possibility of reading a long book entirely on the computer’s screen, some participants were specially critical of the inconveniences of reading digital formats. Also, it is reported that information artifacts conveyed more subtextual information than their digital counterparts. It is suggested that such information is crucial in categorising information: I like to have a proper book. You can just flip quickly, and then go to the index and then look at the page (...) You just know by looking at it, by looking at the formatting if its the right book. You realised it too technical, or not technical enough. You just know by opening a book if its too technical or too complex or at the level you’re writing up. Whereas with the PDF you have to scroll for ages... As geeky as it sound, I like the feeling of pages, the smell. PDF they all look the same, they have different formatting, but you can’t get the feeling of the book by scrolling down. [Participant C]

What is specially interesting is the hybrid nature of research experiences. Even when specific formats are preferred for different actions, both digital and information artifacts are used concurrently: I would into the look into the index the first thing and I would look and I’ll fin out whatever topics which is what I want to research on. I would then look into all the books, and see which of all the topics are there and in which pages and then I might go through all the books . Sometimes I might get my information through the internet as well, I’d go through google and see what

it is. I would then see if it is similar to what I’ve read and understood. Sometimes Google might come up with something easier to understand. Books sometimes tend to complicate things, authors might try to get their intellectual ideas in there in a more complicated way. Google might come up with very easy ways to get information. It’s a mixture of computers and books. (...) Even one book it’s fine. If it’s one book, then it would usually be that book and the computer. At first it would be different sources on the internet and then only one source. But If I have different books with me, I might just go through all the books and one source in the computer online. [Participant F]

Incremental implementation to next stage The series of interviews yielded an important insight into the research practices of current postgraduate students. What is remarkable is that even when every interviewed participant use technology regularly, information artifacts still feature as an important source of information. All the participants expressed to own at least one computer, and some of them acknowledged using tablets and other mobile computers. Still, books are the preferred medium given the fluidity in their manipulation, associated levels of comforts whilst reading, and the sensory complexity they afford. Only participant F declared to base his research practice on digital sources. He mentioned that his degree required him to

acquire information predominately from journals. In this case, research practice is based on digital resources due to printing costs. He would later acknowledge information artifacts as more comfortable to consult. Another important trend to discuss is the perceived efficiency of the research practices. In three of the cases— participants A, C and F— prefaced the description of their paper-based research practices with comments suggesting low efficiency and anachronism. This phenomenon presented an additional challenge to this research. As people perceive paper-based activities as being less efficient, it was deemed likely that the participatory event would be hampered by this perception. In order to address this matter, a provocation prototype was prepared to raise awareness of the mixed nature of present day research environments. It was expected that this tool would prevent designers in the body-storming session to restrain their efforts to input paradigms such as keyboard and mouse. The next section gives a general description of the narrative for the provocation prototype. Generally speaking, the four main areas observed in this qualitative exercise were integrated: (1) Memory and embodiment, (2) Architectonic system and surfaces, (3) Information artifacts embody information filters, (4) Information need related to media.

Video-Provotype

Initial Considerations The introductory paragraphs to this research stated that our current research practices can be described as a mixed ecology of information resourcesâ&#x20AC;&#x201D;a continuos transaction between the digital and the analogical. As discussed in the literature review, the future of information artifacts is not without controversy. Whilst some researchers in the area of human-computer interaction posits that it is only a matter of technological development until tablets and digital screens supersede paper (Kurzweil, 1992) , others voices affirm that books embody a more complex sensory experience that is impossible to supplant with digital systems (Eco and CarriĂ¨re, 2012). Data collected in the qualitative studies 46

suggests that this controversy may as well be observed in the individual opinions on the convenience of one source of information over the other. For instance, some participants were emphatic in declaring their preference on information artifacts over digital information. However, they also considered that these practices were outdated or even inefficient. Therefore, designing systems that conflate both information sources is anticipated to have inherent tensions. In order to ad-

Chapter

5 provotypes as â&#x20AC;&#x2DC;provocateur provoking discrepancies in the concrete, everyday practice to call forth what is usually taken for grantedâ&#x20AC;&#x2122; Mogensen (1992, pp. 15) 47

dress the potential tensions to be found in the participatory design of the Architectural User Interface (AUI), the framework to this research included a provotype between the qualitative studies and the bodystorming. Such provotype takes the form of a short-film depicting some of the practices and insights collected in the preceding qualitative studies in this research. Provotypes are a variation of prototype techniques used in Human-Computer Interaction to channel tensions in a problematic situation towards participatory innovation (Boer and Donovan, 2012, pp.388-389). Mogensen (1992, pp. 15-17) describes provotypes as a â&#x20AC;&#x2DC;provocateur provoking discrepancies in the concrete, everyday practice to call forth what is usually taken for grantedâ&#x20AC;&#x2122;. In so doing, these mediums of provocations are intended not to state a possible solution to a specific problem, but to address specific issues to address. For instance, Carlshamre (2002) implemented a series of provotypes to support the tasks of software development planners. Provotypes are specially described to assist in environments with a multitude of conflicting variables at stake, commonly characterised as wicked problems. In the context of this research, the intention for the provotype is twofold. Firstly, it is intended to embody insights collected on the qualitative studies, thereby assisting in the transition between qualitative data and design. Mogensen considers provotypes as intended:

to be used between the activities of initial investigation/analysis of the current and design of the new. The object of provotyping is current practice, as in investigation/analysis, but the focus is on furthering change (...) Provotyping can serve as a bridge between analysis and design (Mogensen, 1992, p.17) On the second hand, the proposed provotype raises the complex nature of the negotiation between analogical and digital information. Instead of following an event-based narrative, the video comprises a collection of video sequences showing different activities that participants in the interviews series described as their research practices.

General structure Provotype is structured as a short-film with a total duration of three minutes and twenty seconds. It resorts to cinematic techniques in conveying practices related to the manipulation of information artifacts and digital resources. In order to achieve a clean 35mm-like aesthetics, the video was filmed using a DSLR camera with a 28-80mm lens. Closed diaphragm exposition was favoured to facilitate shallow focus shots, thereby creating a visual

Figure 5.1. Photogram from scene 1

Figure 5.2. Photogram from scene 2

Figure 5.3. Photogram from scene 2

Figure 5.4. Photogram from scene 3

Figure 5.5. Photogram from scene 3

Figure 5.6. Photogram from scene 3 49

Figure 5.7. Photogram from scene 3

Figure 5.8. Photogram from scene 4

Figure 5.9. Photogram from scene 4

Figure 5.10. Photogram from scene 5

Figure 5.11. Photogram from scene 5

Figure 5.12. Photogram from scene 6

discourse centred on the details of research practices. In order to avoid bias, the video focused on showing the research practices form a close-up perspective, avoiding any attempts to give a perspective of the interaction between information artifacts and the enclosing space. Following, we present a description of every scene alongside the motivation for each one.

Scene 1

Scene 2

Scene 3

Book stacking in a shelf. Participants alluded to the use of stack to categorise information artifacts. The scene depicts the action of clustering books in different physical arrangements. Stacks are intended to suggest a categorisation strategy enacted in the physical interaction. Book clustering and physical interaction schemes were two organisational schemes that emerged during the initial interviews performed for this research. The Memory and embodiment section of the previous chapter discuss in detail how this actions are incorporated in everyday research practices.

Initial exploration on information artifacts. The scene shows a researcher placing books on a desk and exploring semantic connections. Associated techniques are depicted, such as skim reading, highlighting and handling multiple materials concurrently. Exploration and interaction with information artifacts was a found to be an important practice in research environments, specially on the early stages of research. Findings from the interviews section of this research suggest that these practices are deeply related to human body and the space, thereby inspiring the sequence presented in this scene. Conclusions on this matter are offered in the Memory and Embodiment and Architectonic system sections of the previous chapter.

Digital information exploration. The sequence depicts the actions associated to information gathering using digital resources, such as online search engines. The topic Information architecture and interaction design is used to outline the events of searching for specialised information. The researcher is portrayed as consulting general resources, specialised journals, downloading digital copies, riffling through the information and printing out articles. The dynamic nature of this transactionsâ&#x20AC;&#x201D;suggesting a fluid negotiation between physical and digital informationâ&#x20AC;&#x201D;is based on the insights gained through interviews. A detailed discussion on findings in this topic is offered at Information need related to media on previous section.

Scene 4

Scene 5

Scene 6

Physical classification. A stop motion animation is used to highlight the spatial-temporal interactions of information artifacts when clustered in piles.

Dynamic and parallel manipulation of information artifacts. The scene shows the manipulation of information artifacts suggesting parallel information retrieval. Opened books are placed on top of one another. Similar to the practice of stacking books, this research found through qualitative studies that spatiality is an important trait in personal organisation schemes. Moreover, spatial location of information artifacts is found to be instrumental in the assimilation of information.

Information highlight. The final scene depicts the hands of a researcher reading printed versions of articles, whilst creating notes and highlighting sections using a pen marker. As suggested in the literature review, epistemic actions intend to aid mental computation by representing cognitive processes through material symbols (Kirsh and Maglio, 1994). Findings in the series of interviews suggest that participants take advantage of the physicality of information to aid in their own cognitive process. 51

Participatory Design Session Initial Considerations Bodystorming refers to the process of recollecting highresolution interactional user data by â&#x20AC;&#x2DC;enacting experiential awarenessâ&#x20AC;&#x2122; (Schleicher et al., 2010, p.47). There are a number of alternative methods available to HCI design to gather interactional experience. The first is adopted from anthropological studies and revolves around the detailed observation and documentation of actions. The second one is to perform actions in a controlled environment given an initial scenario (Oulasvirta, 2004, pp. 250-251). Both of them present considerable shortcoming nonetheless. The first option relies on the ability of the researcher to register and interpret every 52

action dutifully, which commonly leads to corrupted or omitted data (Idem). The second option involves a fixation bias. People asked to perform actions may adopt a premature idealisation of the analysed system (Schleicher et al., 2010). Schelicher (2010) proposes three methodological variants of embodied storming. The first is to simply perform all design tasks in the environmentâ&#x20AC;&#x201D;only by designing in the place, the designer will be confronted with context-specific traits

Observed patterns seem to suggest that tangibility plays a crucial role in cognitive processes of information retrieval.

This

feature seems to be a memory token additional to the use of colours or the position of the information

Chapter

that will invariably affect decisions. The secondâ&#x20AC;&#x201D;strong prototypingâ&#x20AC;&#x201D;involves testing prototypes in a reproduction of the intended context. In the context of this research, we will concern with the third variantâ&#x20AC;&#x201D;user case. A number of participants are asked to perform specific actions on their original context. The rationale for the effectiveness of this technique is grounded in the theory of embodied cognition, which argues that all of our mental schemas are determined by our physical experience of the world (Johnson, 1990, pp.20-21).

Structure A participatory design session was organised and followed a user-case variant of body-storming methodology with a two-step layout. In the first half, the bodystorming will serve as an observational study to gather detailed information on the spatial interaction of users with information artifacts and digitals resources. Then, a video provotype will be shown to participants in order to raise awareness on the sensory complexity of contemporary research environments, specially on the mixed nature of the information sources. Finally, a group discussion will be proposed to discuss impressions on the activities performed in the first half of the event, and suggest possible features for the development of the Architectural User Interface (AUI). Design of mixed media digital systems require a back54

ground in digital information and a formation in space design. After evaluating the profile for participants in the body-storming, five postgraduate students from the School of Architecture, Planning and Landscape at Newcastle University were invited to take part in the study. All participants share a background in design degrees, and have undertaken modules related to Digital Information, Interaction design and programming principles. The proposed sample is consistent with methodology suggested by Oulasvirta et al. (2003), as it is considered that participatory design sessions are more productive when run with a small group of between four and eight participants. In order to collect rich insights, an effort was made not to have the same participants of the interviews series in the bodystorming. Only one participant from the interview series also participated in the bodystorming. The session had a total duration of sixty minutes and was held on Space 9 of Culture Lab at Newcastle University. Data was be captured using two cameras mounted on tripods and will be later codified by the researcher. Support from an external reporter was requested to handle videorecording and photograph the session. Interpretation of results will be performed following the same Grounded Theory approach detailed in the individual interviews. All data has been anonymised and referred through the name code presented in Table 6.1. The following sections give a brief description of the activities performed during each stage in the bodystorming.

Table 1 outlines participants’ background.

PARTICIPANT

FACULTY

GENDER

GROUP

Participant A

School of Architecture, Planning and Landscape

Male

Participant B

School of Architecture, Planning and Landscape

Female

Participant C

School of Architecture, Planning and Landscape

Female

Participant D

School of Architecture, Planning and Landscape

Male

Participant E

School of Architecture, Planning and Landscape

Female

1 Table 6.1

The session had a total duration of sixty minutes and was held on Space 9 of Culture Lab at Newcastle University. Data was be captured using two• cameras mounted onintripods andecosystem will be later Nowadays, we live a mixed codified by the researcher. Support from an external reporter was requested to handle video-recording of reading—we recourse to both, digital and photograph the session. Interpretation of results willand be performed following the same Grounded physical information in our research Topics Theory approach detailed in the individual interviews. All data hasConsidering been anonymised referred practices. this, isand it plausible to foresee that the architectural typology of through the name code presented in table 1. In order to replicate the embodied practices related to physical libraries will change radically? research, a series of activities were planned. Activities The following sections give a brief description of the activities performed during each stage in the revolved around the main theme of writing a research Additionally, two sets of keywords were prepared for bodystorming. paper. Two hypothetical topics were prepared: each topic. Keywords were extracted from key topics found in the information materials provided to each • Considering the reading behaviours associgroup. For topic number one, the following set of keyated with digital and printed technologies, words was considered: will e-books supersede physical books? Topics In order to replicate the embodied practices related to research, a series of activities were planned. Activities revolved around the main theme of writing a research paper. Two hypothetical topics were prepared:

• Reading experience • Affordances • Cognitive strategies • Hyperlink and hypertext • Provenance • Technological shift • Embodied mind • Mental schemas Topic number two included the following set of keywords.

The physical space was arranged with a large desktop to accommodate two groups on each end. Each group was provided with a total of 4 books, 5 printed articles and two internet-enabled computers. All the printed articles were also provided in a PDF version located in the desktop folder of each computer. Also, other materials were arranged in the same desktop: scissors, masking tape, thread, colour markers, post-it, page markers, wood skewers and blu-tack. A researcher was responsible for guiding the activities and assisting the participants in any way possible. The following activities were observed:

• Physical navigation • Information systems, retrieval and classification • Mundaneum and Le Corbusier • Paperless information system • Paradigm shift • Affordances • Paul Otlet • Ubiquitous computing

Observation study A thirty minutes observational study was structured around research practices. Participants were asked to follow a series of activities planned to replicate some activities related to research.

• Participants were organised in two groups and presented with all the material available to them in the room. It was explained that all information sources related to a topic that was to be presented in a later stage of the process. Then, they were asked to arrange the materials in any way that was meaningful or useful for them to start their research process. Participants were invited to use any surface in the room to this end. Ten minutes were allocated to this activity. • Each group was handed a piece of paper with the one of the previously defined topics. Participants were asked to read the topic and further find new research resources (such as more books, articles, internet-based data, etc). Seven minutes were allocated to the activity.

â&#x20AC;˘ Each group was presented with the keywords associated to their research topic. Participants were asked to locate such topics on the research material they had at their disposal. This activity ran for a total of seven minutes. â&#x20AC;˘ Groups were asked to create a representation of the semantic connections they had found in the information. It was defined that such representation could be of any kind and should help them in the process of writing their research paper. The researcher reminded the participants of all the materials available to them. A total of ten minutes were allocated to perform this activity.

Group discussion Upon completion of the first half, participants were given a brief description of the nature of this research. In order to create a clear transition towards a group discussion, the video provotype was shown. It was expected that performing the action related to manipulation of digital and analogical information, coupled with a explicit graphical depiction of such activities, would help elicit some of the main issues of designing for mixed-media systems. Group discussion took the form of a semi-structured conversation on the general impressions of the activi-

ties, and possible features that a hypothetical AUI system could have. A researcher was responsible for directing the conversation through specific questions and suggest topic to be discussed.

Findings Tangibility of information Information tangibility emerged as an important feature during the observation studies, and was further discussed during the focus group. Observed patterns seem to suggest that tangibility plays a crucial role in cognitive processes of information retrieval. This feature seems to be a memory token additional to the use of colours or the position of the information artifact. When the lead researcher raised the topic in the focus group, participants accorded in considering shape as an important feature in creating and sorting categories. It was suggested that the possibility to manipulate and give shape to custom memory tokens or proxies may enact a more specialised and effective organisation scheme. As means of illustration, one of the groups created a series of plasticine artifacts to bookmark information found in books. When prompted to expand on the motivations for this strategy, Participant A men-

Figure 7.1. Plasticine-based memory

Figure 7.4. Photogram from scene 5 58

Figure 7.2. Exploring information

Figure 7.5. Photogram from scene 5

Figure 7.3. Creating a knowledge

Figure 7.6. Embodied semantic network

Figure 7.7. Exploring semantic connections

Figure 7.10. Plasticine memory holder

Figure 7.8. Tangible system of semantic connections

Figure 7.11. Watching the provotype

Figure 7.9. Giving instructions

Figure 7.12. Preparing to the next stage 59

tioned: You can see all the information, recall all your memories, from this side [points at the side of the book with his left hand] which can save a lot of space. When you’re doing research you always put your books like this [places three books on the table, side by side] with some tags on it. But with this you can see all your information and recall you memory from the side [grabs the three books again and stack them into a pile]. It’s more tangible. The colours help you to link information to each other. And maybe the colour can also be linked to the webpage colour. We used three shapes, rings, blocks and tails. That may also give you another kind of link [Participant A].

Information on brackets are editorial marks describing body language associated to the subject discourse. The strategy to evidence semantic connections can also be observed in the keyword sheet handed to the team. In order to have a consistent colour and shape code for each topic, Participants A and E decided on placing a small piece of plasticine by the related word. Semantic connections were embodied also by connecting the pieces of plasticine with thread, therefore creating a tangible manifestation of the semantic connections constructed during research. The importance of tangibility seems to be linked to the possibility of having all the information in sight, therefore creating a clear mental map of the semantic connections. In this regard, participant C described the organisation scheme used in group 2: 60

Personally, how I do things is I would separate everything. Hopefully in two categories, and give them different colours for each category. And then subcategorise them. As I go along I put strips along to show “this is the initial thought, and this is where it is going in the end. And then you can see right by looking at all of the sources which colours coordinate with each other and which ones don’r [Participant C].

When further asked if this trait was used as a memory aid, Participant C elaborated: Yeah, being able to see it from distance. Being able to tell “that’s for that, and that’s for that” without having to read anymore because you have already read it [Participant C].

The importance of the tangibility of information is also made manifest in a preference to use information artifacts over digital resources. During the observation studies, participants spend the majority of their time searching for information on books and printed articles. Computers were used only during very short periods of time. In all the cases, participants would use computers to place a keyword query on Google and examine the first couple of results. This interaction lasted for no longer than two minutes per event. When asked to explain the motivation for this interaction pattern, participant C considered ‘I think it’s because is available. It’s in front of you and it’s there. So why bother searching when you’ve got the actual thing at hand just in front of you’ [Participant C]. A further characteristic of tangibility was discussed in the

focus group: compatibility. Computers provided to each group were running MacOS, which turned out to be unfamiliar to the participants. Some problems arose with this matter, and the lead researcher helped to perform actions such as highlighting texts in a PDF. The matter was discussed on the focus group. Participant C suggested that the problem with tablets, for instance, was that it requires users to learn a new interactivity protocol. Books, on the other side, allow a natural set of interactions due to its tangibility. Interaction events such as marking, making notes, shuffling or riffling are natural sets that make the research practice more fluid when it is based on information artifacts rather than in digital resources.

Fluid boundaries of attention Another observed pattern that is worth mentioning is the fluid boundaries of attention. In defining how space and architectural principles may be applied to computational systems, Malcolm McCullough reflects on the concept of periphery and the impact on attention: periphery is background that is outside focal attention but which can quickly be given that attention when necessary. This is one way to deal with information overload (...) We find it more natural to use

our considerable power of sensing the surroundings, and then to experience more capacity and resolution where our attention is focused (McCullough, 2005, p.49) Collected evidence seems to suggest that the boundary between context and focus is not statically defined. Depending on the phase of research, people tend to adjust the threshold of what is in focus and in the periphery of their attention. For instance, in early stages participants tended to hold information artifacts close to their bodies. Once they had a better understanding of the topic and the information material at hand, they gradually expanded spatially, leading to a shifting choreography performed with information artifacts. Some information artifacts would transit from being in the focus of attention to the periphery several times over the exercise. This pattern can be described as the fluid boundaries of attention as enacted in interactional intents. What is worth mentioning is that computers were always on the periphery of participantsâ&#x20AC;&#x2122; attention. Even when participant profile included strong links with technology and digital information systems, none of them preferred to work their way through the information using the computer primarily. This pattern seems to be explained with the lack of tangibility of the digital resources. Notably, during the observation study

Participant A asked permission to place a spot of plasticine on the screen in order to physically bookmark some information he had found on Internet.

enabled the codification of semantic connections between information materials.

Information patterning as a dynamic process

Discussion and

In the first task of the observation study, participants were asked to arrange information materials at hand using any scheme. The task proved overly complicated. In the allocated ten minutes, participants spend their time going through the information materials. Sorting activities extended throughout the rest of the session. Consequently, there was not point in time where information materials could be considered as definitively arranged. As outlined early on this chapter, information artifacts were scattered through the room using any available surface. Upon arrival, participants found books and printed articles lying on the chairs, desk and even on the floor. As the activities unraveled, these materials were slowly changing position into meaningful schemes for the participants. Materials that were being reviewed stayed closed to them, and afterwards positioned in the proximity of conceptually related artifacts. This pattern did not remained static nevertheless. As participants learned more about the topics and the information at hand, information artifacts were rearranged. In the case of group one, the position of information artifact took a crucial role in their organisation scheme. The spatial interactions of information artifacts, coupled with the strategy of plasticine and thread, 62

further implementation Findings in the bodystorming seem to be consistent with those in the qualitative studies. In the previous chapter of this dissertation, insights from the interviews were categorised as: Architectonic systems and surfaces Information artifacts embody information filters Information need related to media It is observed that participants in the bodystorming created representation systems that codified meaning in their physical interactions. In some cases, these principles were embodied in a tangible system based on plasticine and thread. This pattern seems consistent with findings in the Architectonic systems and surfaces section of Chapter 4. Concepts such as semantic connections and spatial intent are put into practice. Also, surfaces were observed to be appropriated by participants in the bodystorming, as hinted at in the interviews. In some cases, even a computer screen was considered as a possibility to implement a system based on tangible aids to the research practice. Systems of tangible memory proxies also demonstrate how information artifacts embody information filters. Once

a text has been read and processes, the artifact is modified to evidence the information it contains and suggest connections with other materials. Also, the framework of information need related to media, discussed in Chapter 4, was consistently observed in the bodystorming. Interaction pattern with computers suggests that digital resources are used only to locate very specific informationâ&#x20AC;&#x201D;for instance, when defining an unfamiliar term. When combined, both studies seems to suggest that tangible systems are better suited to research related activities

such as exploring new information, finding connections between different sources and building semantic networks. Insights from both studies, interviews and bodystorming, will be used to establish an interaction framework for the Architectural User Interface. In the following chapter, we discuss a specific set of actions to be implemented, and describe a video-prototype, which was produced to deploy them. As outlined in the introduction of this work, such actions are based on the spatial interaction affordances of information artifacts, and conflate them with the processing capabilities of computational systems in an effort to

Once they had a better understanding of the topic and the information material at hand, they gradually expanded spatially, a shifting choreography 63

Video Prototype

A video prototype was produced in the last phase of this research. The video showcases the first AUI prototype with all the implemented features, which results from the cumulative knowledge gathered through the previous phases.

The purpose of this prototype is to implement a User Interface based on two main aspects: (1) the interactional events between information artifacts and (2) the information patterning encoded in their relative position in the architectural space.

interactions included in the AUI. Finally, the video will be described in detail alongside some snapshots. A copy of the video is included in the disc accompanying this work.

In this chapter, we will first introduce the lo-fidelity prototyping method, and give a brief overview of the theory behind videoprototypes with some examples of its application in the field of Human-Computer Interaction. Then, we define the set of

Lo-Fi Prototyping The place of artifact in design research has been briefly reviewed throughout this work. It is almost impossible to think of design interaction without testing its func-

The

video follows events associated with the inclusion of a new book in an existing information ecology Chapter

tionalities. However, given the time frame and resources of this research, it would be unfeasible to plan a fully functional prototype. A number of alternatives arise when analysing literature from HCI research. Prototypes are normally classified as low-fidelity and high-fidelity. Low fidelity prototypes contain certain traits of interaction that need to be proven before a full technical implementation. Therefore their interactivity is either limited, mediated or inexistent. Whilst it could be argued that non-functional prototypes overlooks important features that inevitably affect the interaction with end-user, lo-fidelity provide a fast and accessible way to demonstrate and evaluate different interaction concepts early on the process (Rudd et al., 1996, p.78). Although low fidelity prototyping is a widely spread technique in interaction design, there is not agreement on the standards or materials employed in their fabrication. Lo-Fi prototypes can range from paper prototypingâ&#x20AC;&#x201D;whereby a moderator enacts the interactivity of a paper version of screen interfaces (Snyder, 2003, pp.1-3)â&#x20AC;&#x201D;to a wood table cut to the exact size of a hand held device for the user to carry and ergonomically test (Rudd et al., 1996, p.78). Common ground seems to be found in the definition relating the level of fidelity with the likeness of visual appearance and detail and where interaction is somehow constrained or limited (Petrie and Schneider, 2007). Most of the lo-fidelity prototypes presented in HCI literature refer to methods appropriate to screen-based interaction or mobile devices. There is however another method 66

that seems to be emerging in the field: video-prototyping (Mackay et al., 2000). The first incursion of the method can be traced back to 1980s, when Laurie Vertelney (1989) wrote a paper detailing different video techniques available to interaction designers. She included animated drawings, cutout animations and computer animation (Vertelney, 1989, pp. 58-60). The paper highlighted the special case of mix-media prototypes, whereby different video and post-production cinematographic techniques were employed to convey the look and feel of a conceptual designâ&#x20AC;&#x201D;unfeasible to be constructed due to the high level of sophistication. To illustrate, she points to the paradigmatic case of the Knowledge Navigator series developed by Apple (Idem). As suggested by Mackay (2000), there is always a continuos trade-off between resolution and quality of prototypes, and the possible effect these have on the evaluation of prototypes. Video-prototypes are suggested to remediate this problem: We have been exploring how to use video artifacts throughout the design process to help us manage the tension between qualitative details and design abstractions. Our video artifacts act as both the output of one design activity and the input to the next (Mackay et al., 2000). Additionally, video-prototypes afford the possibility of deploying an interaction system alongside a visual discourse of the motivations and conceptual framework. As described in the following sections, we used the visual nar-

rative of the video-prototype to convey the conceptual framework of this research whilst demonstrating the possibilities of the system.

Aui Interactions Using the findings of the previous set of interviews and bodystorming, the framework proposed by Fast and Sedig (2010) was appropriated to define a set of epistemic gestures to articulate a User Interface based on Spatial Interaction. The following actions are considered:

1. Manual chunkingâ&#x20AC;&#x201D;By placing documents in piles or in proximity to one another, the system will group them into a query corpus. In chapter 3, it was outlines how participants in a series of interviews described a series of spatial strategies to organise and explore information. Piles were of special importance in classifying information and creating holding patterns that reflected memory and research processes. Furthermore, the observation study performed in the bodystorming revealed a similar trend. Materials were shifted and spatially organise to denote meaning and semantic connections.

2. Filtering â&#x20AC;&#x201D; Macroscope yields the possibility of querying one active corpus, leaving other groups greyed in the background. This will be codified by the relative position of the user to the chunks of information. This information will serve to discriminate what is in the centre and periphery of attention, therefore affecting the level of information shown by Macroscope. During the series of interviews, participants reflected on the use of space as a tool to discriminate information and prioritise attention. For instance, in some cases the research space was structured around the placement of information artifacts in two piles. The first one containing background information, whilst the second contained information actively used. Alternatively, the bodystorming session showed how the threshold of attention is an evolving property relative to the position of the body. It was observed, for instance, that early in the research participants held the information artifacts close to their bodies. 3. Probing â&#x20AC;&#x201D; The proposed interface includes the possibility of hiding and invoking different granularities of details shown for each information artifact. The system will codify active manipulation of artifacts by the user and return more information on those materials. For instance, if two opened books are piled together, this will be interpreted by the interface as a request for more detail information on

those specific sections. Previous studies, bodystorming and interviews, revealed a well defined strategy of information retrieval on different media depending on the nature of the information need. In most cases, computers and internet-based search engines are used to learn very specific facts about a topic. Conversely, books are used in more detailed information retrieval. It is expected that this spatial manipulation will enact the possibility of getting more information about a semantic connection without the need of using a keyboard or mouse. 4. Automatic chunkingâ&#x20AC;&#x201D;In the background, the system will perform automatic queries to match new materials similar to the ones chunked physically. Results will be shown as dotted nodes. As stated in the previous action, digital systems enable the possibility of giving detailed information about a topic. However, the possibility of statistically analysing a document is not reported to be used often in research. Therefore, this

interaction is planned to give users a more detailed view on the possibilities of a corpus, suggesting new semantic connections. 5. Rearrangingâ&#x20AC;&#x201D;Once a chunk has been configured, the gesture of changing one book to another group will be interpreted as a request for reconfiguration. The book will be automatically shifted to the new corpus, which in turn will trigger a new TFID analysis. Insights collected in the interviews revealed that research is a complex activity, and is often difficult to characterise. Similarly, it was observed during the bodystorming session that the process of arranging and classifying information never stops. The semantic dimension being constructed through research is always evolving and reconfiguring. Therefore, this trait in the system would allow users to visualise how the different connections affects such semantic networks, yielding the possibility of fluidly explore new possibilities.

Narrative The video follows events associated with the inclusion of a new book in an existing information ecology. A researcher is portrayed to introduce the book This is not the end of the book by Umberto Eco and Jean-Claude CarriĂ¨re (2012) in their research environment, and uses the Architectural User Interface to find how the new text fits into the existing semantic network. In the following section we analyse the scenes in detail:

Scene 1

Scene 2

Arrival A messenger bag is placed on a table. When opened, it reveals the cover of the book.

Tea sequence A series of short clips are assembled to narrate tea brewing. This scene was included in order to convey the conceptual framework of bringing back the materiality to information. As mentioned by Dade-Robertson(Dade-Robertson, 2011) and McCullough (McCullough, 2005), digital systems has undergone a process of liberating the information from materiality. However, it is also materiality and embodiment the basis of our cognition processes. Findings in this research seem to suggest that tangibility is a highly valued trait in information artifacts to aid memory and cognition in research activities. The inclusion of tea is meant to suggest the underlying materiality of information and research, by linking them with a human activity such as drinking tea.

Scene 3

Scene 4

Establishing shot Dolly movement showing some architectural features of the space, such as mouldings and a fireplace, and introducing the researcher sat on a desk reviewing some articles.

Desktop view. Scene designed to showcase Manual Chunking feature in the Architectural User Interface. Top view with camera pointing downwards. Researcher is depicted reading some papers, and then moves them apart. Researcher reclines on his chair and grabs his cuppa, which pulls in a split screen effect. Drinks and leaves the cuppa on the edge of the split. He starts examining the main book and then starts to examine semantic connections with other books by staking them. In the second screen, we see Macroscope interface. As the books are put in the stack, a new node appears to show the information of the book on the system. The semantic network inside macroscope refreshed and creates new connections as the researcher manipulates the information artifacts. After chunking four books, the researcher reclines on the chair. Meanwhile, the system shows the feature of Automatic Chunking by creating suggested node that are pulled from internet.

Scene 5

Scene 6

Rearranging sequence Scene designed to showcase the rearranging feature in the AUI. The research leaves the desk carrying a pile of books. Camera is position at ground level and sees the researching approaching two piles of books in first plane. Researcher is seen squatting, and second split screen is introduced. Researcher rearranges books between piles, which causes Macroscopeâ&#x20AC;&#x201D;shown on the right section of the screenâ&#x20AC;&#x201D;to reconfigure semantic networks.

Grabbing a new book sequence After reincorporating, researcher is seen to leave the screen by walking to the right. Then we see him grabbing a book from a shelf and returning back to the left of the screen.

Scene 7

Scene 8

Embodied interaction sequence. Scene designed to showcase probing feature in the AUI.

Architecture of information. Scene designed to show Filtering feature, alongside manual chunking and rearrangement.

Camera points downwards looking at the main book. Then, the camera is lift revealing a complex arrangement of information artifacts laid on the floor. The researcher is seen manipulating information artifacts. Then, the scene splits in two. On the left side we see the researcher chunking opened books, and rearranging them constantly to query the system. On the right side, we see MACROSCOPE displaying different granularities of information on the books.

The camera starts on a extreme close-up shot of books laid on the floor. Then, camera follows the researcher walking in the scene. As the camera moves, it reveals a room with information artifacts lined up in the floor. The researcher then squats and rearranges a pile of information artifacts. In the background, we see MACROSCOPE superimposed on a wall. The system reconfigures and reacts to the researcher manipulating information artifacts.

Figure 8.1. Arrival, scene 1

Figure 8.2. Tea sequence, scene 2

Figure 8.3. Tea sequence, scene 2

Figure 8.4. Establishing shot, scene 3

Figure 8.5. Establishing shot, scene 3 71

Figure 8.6. Desktop scene

Figure 8.7. Desktop scene

Figure 8.8. Rearranging sequence

Figure 8.9. Rearranging sequence

Figure 8.10 Grabbing a book sequence

Figure 8.13. Embodiment sequence

Figure 8.11. Grabbing a book sequence

Figure 8.12. Embodiment sequence

Figure 8.14 Architecture of information 73

Video Prototype Conclusions This research effort started as a counter argument for the now well-established assertion that the book is, as it were, in its last legs. As convincing as this argument may sound when holding the last release of an Ipad, such a strong an assertion deserves to be reconsidered over. The question at hand is not a mere technological issue, but a complete cultural transformation. Therefore, is not this just a blind spot of technology? As McCullough puts it Â´Humanity has had thousands of years to build languages, conventions, and architectures of physical spaces. Wave upon wave of technology has transformed those cultural elements, but seldom done away with themâ&#x20AC;&#x2122; (McCullough, 2005, p.11) The argument heralding elegies for the book seems to be missing a rather crucial point: books are not just information storage systems. They are as well read, consulted and used by humans as a proxy of memory and cognition. This quality is of special attention to architecture. Books do not only carry information, but also shape the physical space of research and construct a semantic as well as a material space. Moreover, books are ultimately the agent 74

of the lengthy process of selection and filtering culture is build upon (Eco and CarriĂ¨re, 2012, p.ix). In the literature section of this work, we introduced the concept of cognitive and pragmatic actions. Cognitive actions refer to physical manipulation of the environment as a symbolic resource to aid cognition. Similarly, we may talk of the research environment as being constructed by two level of architecture: pragmatic and epistemic. Pragmatic architecture takes care of arranging a desk, shelves and other furniture inside a physical space to best suit research. Epistemic architecture on the other hand creates bot a physical space, delineated by the purposeful placement of information artifact in the space, and a semantic dimension, the conceptual network connecting all the sources of information. Under this terms, the Architectural User Interface enhances the construction of digital semantic networks by means of epistemic architecture. This research aimed to enable exploring information by using the vas processing capabilities of digital systems, whilst operating a material realm. Given the nature of our

embodied cognition, the materiality devolved to information in the Architectural User Interface is expected to create a more natural approach to assimilating and exploring new informationâ&#x20AC;&#x201D;a new interface to find the hidden connections that lie beneath information artifacts. Consequently, this work builds on the argument eloquently put forward by researchers such as Dade-Robertson (2011), McCullough (2005) and Suchman (2006)â&#x20AC;&#x201D;that embodied information is not the remnants of an anachronistic technology, but the basis of our

Chapter

cognitive process of sense-making. In facing the challenge of incorporating such concepts in a User Interface, this research reckoned two routes of action. First, build and implement a system based on the designerly interpretation of theories of cognition, embodiment and informationâ&#x20AC;&#x201D;inevitably a top-down approach. Alternatively, it could start a step back and learn directly from the users what are the specific practices of researching in a mixed ecology of information. This work opted for the latter. Human cognition, and as an extension research practice, are complex phenomena. Even when the arguments of this research relies on the groundwork laid by structuralist functionalism, such as the notion of the Architectonic System, it acknowledge that it is not possible to create a highly programmable set of spatial interactions that would codify human cognition. Instead, the concept of Architectural User Interface developed in this work aims at creating a flexible system that operates on flexible thresholds. As a consequence, it is a set of actions that allows users to transfer their motor abilities to manipulate complex sets of abstract data (Naumann et al., 2008, p.2377).

Video Prototype Further Work The contribution of this work to the field is humble. It is the start of a greater research effort to remediate digital information by restoring the sense of information artifacts. Thus, a number of technical challenges raises in the future implementation of this work. For instance, the implementation of gestural interaction and connection to a digital system poises a number of technical difficulties. Whilst this work was developed, this researcher established informal communication with Tom Schofield, original developer of Macroscope. After presenting some of the main concepts and motivations for providing a knowledge network visualisation with a embodied interaction, Tom started building an early implementation of interfacing Macroscope with physical books. His work, which is included as a video demo in the appendix of this research,

builds upon Reactivision, a computer vision framework for Processing based on fiducial markers. This new implementation of MACROSCOPE is capable of recognising when a book is in sight of a webcam, mounted on top of the research space. The physical presence of an information artifact in the research space includes a new node on the system, showing basic information such as title and keywords. Books are augmented with a unique fiducial marker, printed in piece of paper, pasted on the cover. By coupling a system of computer vision and NFC system, it would be plausible to implement the set of interactional gestures described in his research. For instance, the use of Near Field Communication (NFC) protocol has been reported (Sarmenta, 2012) to be applied in digital application based on manipulation of physical artifacts.

Bibliography BAYAZIT, N. 2004. Investigating Design: A Review of Forty Years of Design Research. Design Issues, 20, 16-29. BOER, L. & DONOVAN, J. 2012. Provotypes for participatory innovation. Proceedings of the Designing Interactive Systems Conference. Newcastle Upon Tyne, United Kingdom: ACM. BRIEFING, E. 1975. The Office of the Future. BusinessWeek. New York: Bloomberg L.P. CARLSHAMRE, P. 2002. Release Planning in Market-Driven Software Product Development: Provoking an Understanding. Requirements Engineering, 7, 139-151. CHARTIER, R. 1994. The order of books: readers, authors, and libraries in Europe between the fourteenth and eighteenth centuries, Stanford University Press. COYNE, R. 2005. Wicked problems revisited. Design Studies, 26, 5-17. DADE-ROBERTSON, M. 2011. The Architecture of Information: Architecture, Interaction Design and the Patterning of Digital Information, Routledge. DADE-ROBERTSON, M. 2012. Macroscope—ArchaID, Architecture and Interaction Design [Online]. Available: http://www. archaid.org/projects/macroscope/. ECO, U. & CARRIÈRE, J. C. 2012. This Is Not the End of the Book: A Conversation Curated by Jean-Philippe de Tonnac, Random House UK. ECO, U. & CARRIƑÇRE, J. C. 2012. This Is Not the End of the Book: A Conversation Curated by Jean-Philippe de Tonnac, Random House UK. 78

FALLMAN, D. 2007. Persuade into what? why human-computer interaction needs a philosophy of technology. Proceedings of the 2nd international conference on Persuasive technology. Palo Alto, CA, USA: Springer-Verlag. FAST, K. V. & SEDIG, K. 2010. Interaction and the epistemic potential of digital libraries. Int. J. Digit. Libr., 11, 169-207. HOLSTEIN, J. A. & GUBRIUM, J. F. 2003. Inside Interviewing: New Lenses, New Concerns, Sage Publications. JOHNSON, M. 1990. The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason, University of Chicago Press. KIRSH, D. 2010. Thinking with external representations. AI Soc., 25, 441-454. KIRSH, D. & MAGLIO, P. 1994. On Distinguishing Epistemic from Pragmatic Action. Cognitive Science, 18, 513-549. KURZWEIL, R. 1992. The Future of libraries, Part 2: The end of books. Library Journal, 140-41. MALONE, T. W. 1983. How do people organize their desks?: Implications for the design of office information systems. ACM Trans. Inf. Syst., 1, 99-112. MANNING, C. D., RAGHAVAN, P. & SCH√ºTZE, H. 2008. Introduction to Information Retrieval, Cambridge University Press. MARSHALL, C. C. 2008. From writing and analysis to the repository: taking the scholars’ perspective on scholarly archiving. Proceedings of the 8th ACM/IEEE-CS joint conference on Digital libraries. Pittsburgh PA, PA, USA: ACM. MCCULLOUGH, M. 2005. Digital ground: architecture, pervasive computing, and environmental knowing, MIT Press. MOGENSEN, P. 1992. Toward a Provotyping Approach in Systems Development. SJIS, 4, 31-53. NAUMANN, A. B., POHLMEYER, A. E., HUSSLEIN, S., KINDSM, M. C., #252, LLER, MOHS, C. & ISRAEL, J. H. 2008. Design for intuitive use: beyond usability. CHI ‘08 extended abstracts on Human factors in computing systems. Florence, Italy: ACM.

NICHOLS, D. M. & CUNNINGHAM, S. J. 2009. The use of paper in everyday student life. Proceedings of the 10th International Conference NZ Chapter of the ACM’s Special Interest Group on Human-Computer Interaction. Auckland, New Zealand: ACM. OULASVIRTA, A. 2004. Finding meaningful uses for context-aware technologies: the humanistic research strategy. Proceedings of the SIGCHI conference on Human factors in computing systems. Vienna, Austria: ACM. OULASVIRTA, A., KURVINEN, E. & KANKAINEN, T. 2003. Understanding contexts by being there: case studies in bodystorming. Personal Ubiquitous Comput., 7, 125-134. PEPPER, S. C. 1957. World Hypotheses, a Study in Evidence, University of California Press. RIMMER, J., WARWICK, C., BLANDFORD, A., GOW, J. & BUCHANAN, G. 2008. An examination of the physical and the digital qualities of humanities research. Inf. Process. Manage., 44, 1374-1392. SARMENTA, L. G. 2012. Tangible and Casual NFC-Enabled Mobile Games. In: KAY, J., LUKOWICZ, P., TOKUDA, H., OLIVIER, P. & KR√ºGER, A. (eds.) Pervasive Computing. Springer Berlin Heidelberg. SCHLEICHER, D., JONES, P. & KACHUR, O. 2010. Bodystorming as embodied designing. interactions, 17, 47-51. SCHOFIELD, T. 2012. Macroscope—Open Processing [Online]. Available: http://www.openprocessing.org/sketch/30349. SELLEN, A. J. & HARPER, R. H. R. 2003a. The myth of the paperless office, MIT Press. SELLEN, A. J. & HARPER, R. H. R. 2003b. The Myth of the Paperless Office, MIT Press. SUCHMAN, L. 2006. Human-Machine Reconfigurations: Plans and Situated Actions, Cambridge University Press. THOM, E. 2010. Virtual Bookshelf: Connecting Scholars’ Research Spaces. MRes, Swansea University. ZIMMERMAN, J., FORLIZZI, J. & EVENSON, S. 2007. Research through design as a method for interaction design research in HCI. Proceedings of the SIGCHI conference on Human factors in computing systems. San Jose, California, USA: ACM. 80

Appendix

AUI