Our Process Book by Camila Afanador

OUR PROCESS BOOK UNFINISHED

OUR PROCESS BOOK Olivia Verdugo, Micah Barret, Catherine Siller, Camila Afanador, Keitaro Sakamoto, Niu Miao GSpeak Wintersession Group 2011

This books is a collection of texts and images that emerged from the Independent Study Group working with the GSpeak System during Wintersession 2011.

GSPEAK WINTERSESSION GROUP RISD 2011 STUDENTS Olivia Verdugo, Micah Barret, Catherine Siller, Camila Afanador, Keitaro Sakamoto, Niu Miao.

INSTRUCTORS Lucinda Hitchcock and Jamie Jewett.

PROCESSING WORKSHOP AND HELP: Clement Valla Special thanks to Lane Myer for being part of our process.and to Oblong Industries for allowing us to use it and discover the potentials of the system.

33 DAYS FIRST MEETING

JAN 6 /2011 -

FINAL SHOW

FEB 7 /2011

WEEK 1 & 2 January 3 - 11 / 2011 Processing Workshop by Clement Valla

06/01/2011

During 4 days be had an intensive Processing Workshop with Clement Valla in the GSpeak room.

Selected sketches produced during the Workshop

Loop. By Camila

First sketch. By Keitaro.

int xpos = 0; int ypos = 0; void setup(){ size(400, 400); background(255); frameRate(10); } void draw(){ for (int j=0; j<400; j = j+10) { ypos = j; for (int i=0; i<400; i= i+50) { xpos = i; strokeWeight(2); stroke(0,0,0); fill(random(255),random(155),rand om(255), 85); rect(xpos,ypos,50,50);

void setup() { size(500,500); stroke(200); background(255); }

fill(random(155),random(155),ra ndom(255), 65); rect(xpos,ypos,20,20);

void draw() { //background(255);

fill(255,255,255,50); rect(xpos,ypos,10,10);

}

fill(random(255),random(255),ran dom(255)); noStroke(); ellipse(mouseX+random(15),mouseY +random(15),20,20); }

First sketch. By Micah.

void setup(){ size(900, 500); background(77, 29, 29); smooth(); } void draw(){ background(map(mouseY, 0, 500, 0, 100), 29, 29); stroke(255); strokeWeight(random(0,10)); line(0, 0, mouseX, mouseY); stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10)); line(100, 0, mouseX, mouseY); stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10)); line(200, 0, mouseX, mouseY); stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10)); line(300, 0, mouseX, mouseY); stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10)); line(400, 0, mouseX, mouseY); stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10)); line(500, 0, mouseX, mouseY); stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10));

line(600, 0, mouseX, mouseY); stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10)); line(700, 0, mouseX, mouseY); stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10)); line(800, 0, mouseX, mouseY); stroke(255); strokeWeight(map(mouseY, 0, 500, 1, 10)); line(900, 0, mouseX, mouseY); stroke(255); strokeWeight(random(0,5)); line(0, 500, mouseX, mouseY); stroke(255); strokeWeight(random(0,5)); line(100, 500, mouseX, mouseY); stroke(255); strokeWeight(random(0,5)); line(200, 500, mouseX, mouseY); stroke(255); strokeWeight(random(0,5)); line(300, 500, mouseX, mouseY); stroke(255); strokeWeight(random(0,5)); line(400, 500, mouseX, mouseY); stroke(255); strokeWeight(random(0,5)); line(500, 500, mouseX, mouseY);

Loop. By Catherine.

First sketch. By Olivia.

int xpos = 0; int ypos = 0; void setup() { size(400, 400); // background(0); } void draw() { background(mouseX,mouseY,0); for(int j=0; j<=400; j = j+50) { for (int i=0; i<=450; i = i+30) { //i starts at 0, goes to 40, increases by 1 every time //starting condition, exit condition, incriment xpos=i; ypos=j; noStroke(); fill(255,0,155); rect(xpos,ypos,10,20); stroke(255); // fill(mouseX,mouseX,mouseY); fill(random(255),random(255), random(255)); ellipse(xpos-40, ypos, 10, 30); fill(255); ellipse(xpos+40, ypos, 10, 10); stroke(xpos,xpos,xpos); line(xpos-10, ypos+15, xpos+20, ypos+35); } } }

void setup() { size(700,700); stroke(255); background(53,165,145); } void draw() { //background(53,165,145); strokeWeight(mouseX/25); stroke(mouseX/3,mouseY/3,mouseX/ 3,mouseY/3); fill(random(200,255),mouseY/3,mou seX/3,mouseX/3); triangle(mouseX, mouseY, width-mouseX, (mouseX+1)/ (mouseY+1),heightmouseY,mouseY/2); }

07/01/2011

First sketch. By Niu. First program for GSpeak. Basic drawing tool activated with the movement of the hand. By Camila

int int int int int

j; x=20; y=30; w=40; h=40;

void setup(){ size(400,400); // shapes = new Shape[i]; background(255);

}

void draw(){ stroke(random(0,255),random(0,25 5),random(0,255),random(0,255)); for(int i=0; i<100; i++){ line(i,i+j,mouseX,mouseY); j++; } }

First program for GSpeak. By Keitaro

WEEK 3

January 17 - 23 / 2011 First Group Discussions and Work Sessions

07/01/2011

January 18th, 2011: G-Speak meeting notes Notes by Olivia Verdugo (please excuse typos, spelling mistakes and incomplete sentences! also, auto-correct kept changing G-speak to “speak” sorry)

lucy:

keitaro:

ideas to think about: art, history, music, science, movies, games etc? what interests you?

so far, he has been able to connect his program with g-speak, if possible, he’d like to use type in an experiment. one interest is in data visualization using type/numbers, another possible interest is in using the whole field in a kinetic manner (tracking movement). a third interest might be using “real” imagery: building, stars, nature etc. to do something.

system is performative, is it possible to turn the “performers” around such that we be able to engage with our audience. even if our back is to our audience, how can we consider that audience? framing and composition, how we make these connections. visualizing these elements. jamie: choreography: move from a full body point of view, and then see how gesture emerges from that, or, see how whole-body choreography could emerge from specific gestural vocabulary necessarily working with local space: what happens if we use up and down or back and fourth in a more extreme way? we tend not to use the volume of the space in its entirety when we use the system what’s happening on stage that we may direct the audience’s attention to? something we think about in any performative medium. thinking about spacial relationships not only between the performer and the screen, but also the audience, the demarcated space of the system etc. where is the space for the person to talk? where is the space for the technology to talk? camila: primary interests: drawing tools as a starting point, discussion on the drawing experience, history, collaboration using gspeak. Documenting the whole experience.

primary interests: type design & typography, gaming catherine: very interested in tracking movement (dance) and potentially attaching sensors to things other than hands. attaching sensors to things other than hands may be difficult, however, as sensors are reading the spatial relationships between tags, ergo it we put the tags somewhere else, it would not necessarily mean anything to the computer. but, maybe we should just try it? if we place tags on parts of our body, would it be possible to contort into the x,y relationships necessary to trigger the system? approach from two angles, one, movement without worrying about what’s being made on the screen, two, the reverse: how to get a very specific result on the screen, what movements would be required? also interested in the incorporation of language as well (written). perhaps a collaboration with some writers to “perform” a text? we must consider the 3d component of the system when we are creating. how might something created in this environment stand up outside of this environment?

what is the work beyond this room? how could it possibly exist in the worked at large?

practical + like to frame things as they are

primary interests: movement, language

jamie:

evelyn:

defamiliarizing our physicality allows us to be more open/ expand what we do

haven’t thought about g-speak too much. traditionally before coming here very interested in thinking about tools for artists (digital tools for children and artists) very interested in the idea of collaborative tools. would like to develop as an artist as well as a tool maker. drawing tools would be something of interest, particularly if there could be a collaborative aspect involving more than one person at a time. taking how we work in the studio and applying that here. thinks she should be able to spend a bit more time here now that the intense part of drawing marathon is over.

desire to be pushed into a less familiar place?

o: (will add) lucy: when we talk about what g-speak was first designed for, would you say it was basically designed to be a bigger projection that we can get on our laptops jamie: the interface, not merely the projection opec: used in a huge room to parse huge amts. of data pertaining to the mineral content of the earth

lucy:

micah:

should there be both a pragmatic project, and a poetic one? again, the interest in spatially exploring typography.

g-speak is more of a one-to-one relationship, quicker navigation etc.

micah: interested in how the impute devices affect how we interact with the content of the computer and think about the screen in general. for example, the new trackpad + hopw it has affected the way undergrads draw using the trackpad how does the technology we use affect the very way we create/what we create how would using both parts / both of our limbs to control content creation affect US

non-linear, and spatial way to visualize/ traverse information information is not “chopped up” the same way (the power pt problem) immediacy of information availability lucy: what are the kinds of artwork that get generated out of processing language always look similar, time to move on? o: (will add) lucy:

as with ipods/ipads, the content is shaped differently + our reading habits are changed

what is drawing? old school thinker explains drawing, what is the limit that makes drawing not drawing anymore

get away from making specific tools towards exploring the design of relationships and how it affects us as users of that tool

what is a mark?

pitcher that can pitch both right and left handed have made so far: exploring the practical technical side of things primary interests: music, food, sports lucy: i know that you have a tendency for the

why is mark making so essential? jamie: from an art-making pt of view: one-to-one interactivity: drawing what is the interactive paradigm that we are working with? how may we define interactivity? citrus bohn (sp?) “a system is only interactive if the user changes the system, and

07/01/2011

the system changes the user” what if there are variables and diff functions that the computer responds to and then we resound to that? interface loop? critique about non-understanding of interacting in dm could the system be a part of the dialogue?

there is a shadow and the machine is reading some response to the hands hand: puppet shadow tools

camila:

transcribing information vs navigating information

the potentials of creating, accidental creation, chance, not-knowing what the computer is going to do next?

picking up the relationships of what your hand is doing as transcribing information

puppets on gloves? scrim/wyang kulit type shadow theater?

deafness and the way that a blind person exploders an object with their hands and how that-kind of tactile exploration might influence the way we create form

catherine:

children’s games?

julie taymor(sp?) lucy:

using mistakes: being able to uniformly obstruct information

shadows: camille utterback

mirrors

catherine:

the fact that the machine creates scenery

we want to explore something with speak that is more that you can just do with your computer with your mouse.

assistance from others while “performing”? (so, a kind of participation?)

lucy:

jamie: 3d gestural collaborative micah: but, we could create a drawing app on g-speak that does what you could never do on a computer

andy warhol and his placing the camera on the doorknob, or just observing very basic things taking the system in and out of the room,: how things begin in a salon or gallery before they are able to move somewhere else the extension of a projector - light film etc. without thinking about interface “the one-eyed projector god”

lucy:

art and design: history about what happens when something is used “wrong”

would we be comfortable with being very open/not having a totally “finished” program/product at the end? perhaps a proposal?

mapping the inner-self: what would the battle-ground for one’s body be? what would happen if we could organize the 3d space that was self?

jamie:

melding of household and self in an externalized way

but, the “this is what i made” part is very important. without making, being here (in the g-speak space) is irrelevant lane: i have a huge list! interesting things: thinking from purely a sculptural standpoint: space, light, simple investigations dealing with the fact that

film before film? how people attempted to breathe life into an inanimate object before film some connection btw watching slow transitions between things that cause cracks in the way we see/understand information drawing in the 3rd dimension: key things

to think about: what’s so cool about the word drawing is that it doesn’t exclude the notion of drawn (pulled out from) once you understand that notion of draw-ing you know that you are pulling it out and then shoving it forward (vs rendering) to draw and then project extrusion drawing as an activity the translation between what we see in our open minds and the tapered tools with which we render those visions (love that idea) what does it take to get someone emotionally invested? violation of security of understanding evelyn: PUSh on the system to see what it can do! it will push back try to understand the parameters within which we are working catherine: do not undervalue the tools that we do have lucy: Richter Floating rectangles to sound? work within our limits and do not be negative about our simple gestures lane: is there a bulletin board to post just the simples thing that someone could create? a way to just see what we’ve discovered lucy: ugop(sp?): flash pro guy look at what he is doing o: (will add) lucy: if we could take everything everyone has tossed out…

moving skyscrapers in manhattan Q: designing a typeface. speak could be a very interesting tool to explore the creation of a typeface through gesture perhaps a script face? but very intuitive creating a system to design a typeface? micah: body language - could the system respond to shifts in our body language? lucy: how could we move to the next level of creating our projects now? catherine: maybe we could start to mock things up? could what we’re doing be BOTH practical and experimental? lucy: type, drawing, language, body & computer interface what part of this can we really do in 4 weeks? go have lunch together and mull over this question maybe: half of us: purely practical design? half of us: purely exploratory exploring the connection btw hand gesture and image? catherine: first have a tool, then repurpose that tool for something experimental rather than just trying to make something impractical micah: experiential vs practical? lane: responsive system transcribes and then allows us to be responsive to what it transcribes

niu:

no matter what we do, it may be interpreted into something we can interact with in a physical way

plays the accordion!

from general to specific

building sketches?

going from the self

11/01/2011

kind of like a three-dimensional tape recorder?

our personal interests could be explored after wintersession

lucy:

(now we are looking at our sketches/previous work)

the magic of what your hand can do is a bit less magical unless you kind of pull whatever is truly unique and surprising out of the system.

lucy: specificity of hand gesture should be what we are working with

what can we do with this system/with our hands that is NOT just mouse like

the limited from the limitedness

how can we name a direction that will bring the amazingness of this experiential space forward?

nancy dwyer - three dimensional typography umâ&#x20AC;Ś extruded

lane: can we storyboard the circumstances that cause what the dream is? abstract thinking/feeling next step: get specific, get visual and demonstrative how can we get it out of a specific laptop into a communal space? lucy: independent study: urban curators?

or vice versa

how do we see type? when does type become something else? the portent of what is to be said? lane: orthographic drawing hidden potentiality? mutations catherine: perhaps videotaping our hands close-up and projecting on another wall our hand gestures

using found gold frames draw attention to various things around town this is different as we each/all want to be the architect of the programming and the recipient of the ah-ha start with something that is a GIVEN is helpful, a dot, a line, a word, a poem, a prop? if we could start with a thing we wanted to solve visually/expressively that might be able to start things moving/going niu: performace game two terminal directions in the middle space there is a lot we may explore lucy: could we say that our primary goal is to use g-speak in a group situation? two groups feels more manageable in terms of having some voice

group discussion - january 18th, 2011 Edited version (Original notes by Olvia Verdugo)

The system has performative qualities, is it possible to turn the "performers" around and engage with an audience. How can we consider that audience? ∞∞∞ We are necessarily working with local space: what happens if we use up and down or back and fourth in a more extreme way? ∞∞∞ Think of spatial relationships, not only between the performer and the screen, but also with the audience. Where is the space for the person to talk? Where is the space for the technology to talk? ∞∞∞ Programming drawing tools as a starting point and discussion on the drawing experience. Documenting the whole experience as part of the learning process. ∞∞∞ We could use type in experimental ways and work with data visualization. ∞∞∞

Approach from two angles: movement without worrying about what’s being made on the screen, and how to get a very specific result on the screen, what movements would be required? How might something created in this environment stand up outside of this environment? What is the work beyond this room? ∞∞∞ Exploring the idea of constructing collaborative tools. Development as artists as well as a tool-makers. ∞∞∞ Interests in how the input devices affect the way we interact with the content of the computer and think about the screen in general. ∞∞∞ How does the technology we use affect the very way we create, what we create? Get away from making specific tools towards exploring the design of relationships and how it affects us as users of that tool. ∞∞∞ Defamiliarizing from our physicality

18/01/2011

allows us to be more open and expand what we do. ∞∞∞ When we talk about what G-Speak was first designed for, would you say it was basically designed to be a bigger projection that we can get on our laptops? The interface is not merely the projection. G-Speak is more than a one-to-one relationship; it allows quicker navigation and a non-linear, and spatial way to visualize/traverse information. ∞∞∞ What is the interactive paradigm that we are working with? How may we define interactivity?

just do with your computer with your mouse. ∞∞∞ But, we could create a drawing application on GSpeak that does what you could never do on a computer. ∞∞∞ Explore the idea of transcribing information vs. navigating information. Picking up the relationships of what your hand is doing as transcribing information. Using mistakes: being able to uniformly obstruct information the fact that the machine creates scenery. ∞∞∞

“A system is only interactive if the user changes the system, and the system changes the user”. Could the system be a part of the dialogue? ∞∞∞ What are the potentials of creating with code and not knowing what the computer is going to do next? ∞∞∞ We want to explore something with GSpeak that is more that you can

“The one-eyed projector God” ∞∞∞ Drawing in the 3rd dimension. ∞∞∞ Translation between what we see in our open minds and the tapered tools with which we render those visions. ∞∞∞ Push on the system to see what it can do! It will push back. Try to understand the parameters within which we are working. ∞∞∞

Designing a typeface? GSpeak could be a very interesting tool to explore the creation of a typeface through gesture. Perhaps a script face? but very intuitive... ∞∞∞ Body language. Could the system respond to shifts in our body language? ∞∞∞ Could what we’re doing be both practical and experimental?

Start with something that is a GIVEN is helpful, a dot, a line, a word, a poem, a prop? ∞∞∞ Could we say that our primary goal is to use g-speak in a group situation? ∞∞∞ Specificity of hand gesture should be what we are working with perhaps videotaping our hands closeup and projecting on another wall our hand gestures

∞∞∞ Exploring the connection between hand gesture and image? Experiential vs practical? ∞∞∞ Responsive system transcribes and then allows us to be responsive to what it transcribes. ∞∞∞ What can we do with this system/with our hands that is NOT just mouse like ∞∞∞ Get specific, get visual and demonstrative ∞∞∞ How can we get it out of a specific laptop into a communal space?

drawing applications, hand gestures, starting point, participatory drawing tools, hand gestures, construction of an experience, group project, movement with visual responses, influence of movement, participation, languages, typography, stage, face an audience, body changes, performance, spatial relationship, framing and composing

the input device affects how we explore and generate the content

18/01/2011

25 / 01 / 2011 BRAINSTORMING Lane helped us writing down all the ideas in a brainstorming session we had. We proposed to develop a group project.

â&#x20AC;&#x153;...the proposal is to create a way to visually map (in 3d) gesture whether that gesture be behavioral or emotive with a focus on (spatial) relationship between hands and bodies and what info we may parse from those spatial relationships.â&#x20AC;? Catherine's description of our project

Participants of the discussion: Micah Barrett, Olivia Verdugo, Catherine Siller, Niu Miao, Camila Afanador, Keitaro Sakamoto, Jamie Jewett, Lane Myer and Lucy Hitchcock 21

WEEK 4

January 24 - 30 / 2011 Group Discussions and Work Sessions

19/01/2011

group discussion - january 24th, 2011 Notes Jan 24th, morning meeting:. (sorry guys, this one is REALLY chock-full-o typos. i will try to edit it when i have a chance)

first: catch-up catherine: working directly on system rather than only on our computers encountered problem: not able to save things once we’ve uploaded them to g-speak jamie: there has been a new user set up for you guys (not clements) we should find login info for this catherine: perhaps we should ask evelyn? jamie: if there is a problem with the system, let me know instead of going to clement directly b/c i can try to fix it first &/or alert Peter interesting to publish the notes that i am taking olivia: process document consisting of google docs notes, video, photos, etc. jamie: who is hosting the wordpress blog? (we think risd/dm?) catherine: worked on the technical aspect of developing our projects yesterday (on sunday) 1 group project (micah, Camila, catherine, olivia) niu: doing own thing/ working on cityscape processing keitaro: working on own typographic project (edit - we all plan on working in the space at the same time so that we may help trouble shoot one another’s sketches etc.) lucy: niu kept talking about fingers? catherine: working on manhattan sketch as primary project catherine: talking about the general group

project: what we ended up agreeing on as a group project was the idea of using the gspeak system to translate gestural expression into visuals i.e. two people having a conversation and using gspeak to record a gestural record of what the hands are doing during a conversation we each did a little visual brainstorming lucy: makes me think about standard typographic example: massin’s bald soprano (visualizing typography) the other thing that brings to mind is the greek theater chorus trope - narrating and coloring what is happening on the stage if something is happening here (in the gspeak space) in a small way (hands) where does that leave the rest of the space? jamie: working 1:1 way background of the environment that this is happening on top of could be impacted by secondary or tertiary levels of interactivity still GENERATIVE thus most of the gspeak stuff i’ve seen has been 1 layer- neutral is there different layers of content/interpolation of data that could create a layered record? mining the data from the hands in different ways to create layers? even just using time: slow down, speed up etc. (edit: this is a really good point to think about!) camila: how do the visuals affect the comprehension of what someone is saying? what kind of text or speech would we want to read? if the gestures will be enough

movement for us to produce the visual part of the project? lucy: my dinner with andre? recording a conversation with two people micah: making someone type/rip paper/ cut paper - what do different actions LOOK like? lucy: i like that this pulls out how we interact with tools it’s saying something/making a statement culturally or sociologically camila: VISUALS: how can the movement of the hands be visually represented, thinking about the movement of a ribbon (like olympic ribbon) element in 3d space etc. (shows us an image of two intertwined ribbons of color) this was the basic idea of thinking of the seismograph but not thinking of the lines as “lines” but objects in a space with surface (shows processing sketch) dimensional lines generated by the movement of the mouse across the screen catherine: color will also play a role of course camila: what can happen if we track the interaction btw. both hands? jamie: one of the points of gspeak is that it applies DEFINITION and MEANING to gesture most of the gestures we use in conversation would have more than one meaning most of the gripes we have defined are not conversational lucy: maybe we need a middle ground/ translating device? (talks about friend’s project involving wind) couldn’t you record the movement of people speaking with some other device and then feed that into gspeak to be manipulated gesturally? in order to get it to that place of pure gesture instead of just gripes... jamie: i think thats going to be a rub that we come up against with gspeak often

you’re saying can i have a realtime motion capture data, which is the first layer, but we have access to the gripe info, not that catherine: everything we do generates some kind of gripe information if we can use this as a tool to identify what gripes are not in the packet that could help us another thing that could help is figuring out how to assign, for example, 3 different gripes to the same effect, so that there is a little bit of leeway built into the gesture camila: when you see the system, it us a little too rigid, we started to try to build in a level of flexibility lucy: what if you had a mannequin, or someone that wasn’t human and did stopframe animation while moving the fingers? micah: i imagined programming little “traps” catherine: someone should maybe jot down questions and concerns we have as we move along two of the things you have brought up is the specificity of the system? lucy: for me it’s not (okay taking a mini break from typing to present my stuff - these notes are in the google doc olivia notes ideas ) lucy: i love how these things are hovering around real possibility for use in the world i am thinking about the stem/steam thing the idea that you could take an ephemeral gesture and turn it into a form so that the thing ends up looking like something no one has ever seen read christopher alexander talks about the relationship of the whole system to the parts what i love is the idea that the concrete aspect of the system (this moth) is in its nature to do what else works that way? do two hands talking make some sensible form as this does? or will it be a mess? what is the difference between us and this perfect being?

20/01/2011

24 / 01 / 2011 MICAH Micah trying out his sketch in the GSpeak system,

25 / 01 / 2011 KEITARO: I succeedED in controlling the camera angle, and now trying to figure out how to use ‘translate()’ at ‘hand.pos.z’.

20/01/2011

jamie: one of the differences btw this and a standard like quality is TEXTURE tracking the relationships btw two hands and the fingers could create interesting texture micah: trying to track relationship btw individual fingers etc. is it possible to track each tag in space individually? or do they only track in relation to one another? jamie: what you are trying to do is subverting the intent of gspeak - going behind the layer they have created. it might be as interesting to NOT do that you can draw the path of the hands from the path of the hands what happens if you draw the path of the gripes instead of the motion of the hands? what happens if you track the path of the hands in relation to THEMSELVES? from the standpoint of not doing what a lot of other people have done, how can we present it differently by taking advantage of what the gspeak system does best? what can we do with the gripe information to create a different kind of positional data? (edit: i agree) lucy: what is the conversation thats being had and what does that do to the hand gestures? micah: (showing sketch for dynamically drawing bezier curves) based on where the mouse goes it changes the curve what i was envisioning is a more human looking line i agree with what jamie said that this is very computer-looking jamie: maybe its because i’m a geek but even so i find this so pleasing micah: if the representation of it is some … what if all the dots/lines are coming out of the matrix of the tag, instead of just a single line? i’ve been thinking about what a seismograph does, it takes this really long record and

i’m really interested in the possibility of maybe (the central square) never moves from the center, but the evidence of motion is captured behind it so we would have this long trail of evidence (sketch done in illustrator) bands of color emerge from the individual balls other tags creating very interesting paths jamie: as opposed to just tracking a swishy line that follows my fingers there’s a “blob” of meaning that appears relating to specific gesture? catherine: so the weight or quality of the line would be changed/determined by gesture? micah: so the lines would all coalesce to make a shape. one thing i’ve been really interested in is the relationship of people to the machine example: fb: you have a transcript of the convo with this you get something else lucy: what if you traced the gestures of a convo, and sent that info to someone else? what would that look like? catherine: a dm student wrote a program that drew squares according to sound louder speaking-larger squares ended up with a catalogue of conversations; visual record of convo with mom, sister etc. it was more intuitive/easier to read than one would initially think lucy: video someone made of people describing the relationship btw two people hands playing with yarn micah: i’m also interested in taking it out of the conversational concept: a pair of scissors, how do we use our hands as tools? lucy: other situations where it would be interesting to look at what the hands are doing? political rally? what other situation would we be able to look at? catherine: something i thought was inter-

esting visually

language is being spoken?

(video with green type and threadlike structures)

another idea that camila had, that might be interesting to look into: there may be some kind of way that the content of the convo could somehow be built into that visual ribbon

dealing with space 3dimensionally ribbons all on the same page visually how could this exist outside of the system? i will be the voice for impracticality@! how neat it would be to have some model for some kind of three-dimensionally thing but then what if i built it myself out of dowels and yarn instead of having it rapid prototyped what if we did videotape the hands? so we could see what they are doing while we are tracking their movements? what are the circumstances? this is the really big thing we haven;t figured out are we going to put random people in here besides ourselves? is it two people having a convo? would we project text onto the other walls? what about the performative aspect of this? (viewpoints book) visual take on theater one of the ways scientists use seismographs is to map the inside of the earth that we can not see lucy: t looks like a geologic map catherine: exactly. what if we create this kind of tool that enables us to map someone’s internal landscape? jamie: interesting bc its one of the things that gspeak is being used to do: parse geological data? lucy: would we overlay data? what’s happening inside you (gesture) and whats happening in the ground in rhode island right now?

dos it make sense for the person being mapped to be able to watch the map evolve? jamie: that goes back to whether this is a stage or not. is this performative? or is this lucy: you have e an interest in the record and the documentation (note to self: olivia & lucy said something interesting at 20:03-21:00 listen to recording and add into notes) [then we talk a little bit about how we each might be interested in being involved during the spring semester] jamie: if we continue in the spring we might not have to continue working on this specific project, but we could build upon what you’ve learned and created lucy: catherine before we move on what , in a nutshell, is the group project? catherine: i think that the proposal is to create a visual / a way to visually map (in 3d) gesture whether that gesture be behavioral or emotive with a focus on (spatial) relationship between hands and bodies and what info we may parse from those spatial relationships. keitaro: want to create processing applet to manipulate typefaces and converse through gesture (shows examples of text manipulation by Peter Cho) Jamie: have you guys talked about Daniel Howe’s Rita library?

(edit: interesting)

keitaro(shows examples of how he hopes he may be able to manipulate single letterforms through gripes)

catherine: whats the situation, who are the people that we are putting in this situation? how can we begin to pin this down

lucy: why single letters rather than phrases?

cultural aspect:

keitaro: technical constraints

how do gesture shift depending on what

20/01/2011

lucy: it might be interesting if instead of squeezing for example the c, you went from something like a wide letter-form like a w to narrow letter-form like an l catherine: we also worked on a sketch yesterday where the line will follow your handâ&#x20AC;&#x2122;s x and y position and the z coordinate of your hand affects the width of the line jamie (shows some fire calligraphy dance sketches to us as an example) texture mapping flame on top of the ribbons created by dancers (also shows us troika ranch dance example) lane (43:00-44:00 - fill this in) jamie: ancillary movement why we have motion capture so that the movement we use in our animations are more real-

seeming lucy: designer who created the string project: leslie kwok http://lesliekwok.info <this is her main page http://lesliekwok.info/mfa-thesis/sociograms/ <this is the string example [took a break here to talk about scheduling and decided that Monday at 6pm Feb 7th will be our final showing of our wintersesh work] (camila shows her gspeak sketches - and they work! and look awesome) we discuss the possibility of getting Pilot 1 working (right now only pilot 0 is functioning) (catherine shows her gspeak sketch - and it works! and looks awesome)

24 / 01 / 2011 CAMILA: Drawing sketch using 'easing'. Now the idea is to make it work in G-Speak. and make each hand draw in a different color. 29

(jamie gives catherine’s sketch a try) (edit: BRB) catherine: making a plug for simplicity (edit: here here!! i agree. “K.I.S.S.”) in my experience if we’re really particular and specific about how we set up the environment it will take us a long way in making the result / visual more impressive and valuable jamie: and just to be clear, the point is not necessarilly to be “impressive” you came in here, learned and went through a process that is dynamic and interesting and exciting, that = success. that’s why we’re here. that’s what risd’s agenda for the tool is. I think its importatn/ what david wants to see is that there’s something happening taht’s interesting we might think about inviting peter/david since we’re doing agreat job at documentation, part of what could happen is having recordings of our steps along the way documented on the gspeak blog one problem with our last classes is that we have no documentation of previous work done on the system! so it is as if it never existed lucy: your research has been incredibly rich etc. one of my goals in being part of this is to say this is here and lets keep using it and how can we continue to keep using it as a learning tool even if we end up making smaller tools/’sketches instead of something “grand” example of book containing lovely threads/ layers connecting parts of the book: -> http://maria-fischer.com/en/traumgedanken_en.html [niu now talks about his project] niu: (shows us his cityscape sketch on the gspeak system) moving hands creates citys jamie: maybe you should think about creating a gripe to clear the screen lucy: and somehow i think it would be

lovely to record all the data thats being created (points to edge of screen where gripes are being recorded by “println”) olivia: lets make a poster using all the gripe data generated by an interaction with the system!! niu: i want to create an applet whereby with a very subtle gesture one can control the skyline/buildings flying away from the horizon line lucy: where is that? wouldn’t it be interesting to create a sketch of a city that doesn’t have skyscrapers, or that needed them? niu: (shows black and white photo) this is more like what i’d like to use in my sketch jamie: what about asl? potential problem: so much of what is communicated through asl is through emotional moement/facial expression etc. not just the letters/words formed by the hand but HOW they are formed micah: paleography - developing a calligraphic language for the hand? jamie: might be interested to look into the stylus shorthand of palm pilots lucy: if you aer going to do something where you trace two peoples hands in a convo what if one of these hand \-;.. what if we are already making marks with our hands that are somewhere representative of an alphabet? so, what do we want to sort of leave with? continue to read each other’s notes on google docs look back at that nice nugget that catherine expressed do we need to specify things more precisely in the next 20 mins before we walk out of here? catherine: for me our interests seem to lie in somewhat different areas should we agree upon that now, or decide to set it aside entirely? lucy: could we make each of our ideas live in small sketches? and the look to the process documentation

20/01/2011

GSpeak screen dimensions

to show what else we’re thinking about? jamie: so today maybe figure out what you want to pursue and let us know

come back and try to lay out one or two sentences that explain the goal

(1:36-37 - notes on what we want to do over the next two weeks)

then maybe there’;s a way you can each work on different parts of the goal/realizing different parts

lane: ideas for niu about being able to visually traverse a city by moving things with our hands-shoving buildings apart etc. interacting organically witht he cityscape

jamie: its a great idea to think about working as a group to build the gestural motor, and then each can use it potentially differently

lucy: be cognizant of the binoculars - do you need them if you are not using them?

for next time:

niu: the binocs move with the car. not sure if coding ability is up to par with trying to get done what i want to get done

exploring our understanding of code

catherine: devils advocate we don;t have much coding experience either but we;’re getting it done we can make something simple and yet compelling

where the hand(s) is in space, and how the hands relate to one another > how can we translate that information into a compelling visual?

lane: don’t ever underestimate the cumulative effect showing these demonstrations in steps can have

jamie: on interaction: you impact the system and the system impacts you

& keep in mind what catherine stated earlier: “i think that the proposal is to create a visual / a way to visually map (in 3d) gesture whether that gesture be behavioral or emotive with a focus on (spatial) relationship between hands and bodies and what info we may parse from those spatial relationships.”

what about faking it???

QUICK CHECK IN at 3:00pm:

(sorry everyone i am losing steam here a bit when it comes to typing - 3 hrs and my hands hurt!)

olivia: interested in mapping gripes/relationships rather than just hand in space (not just hand:hand but gripe:space). very interested in the REMNANT - what data is left over from a conversation articulated with our hands, and what can we DO with that data?

people have a tendancy to understand the new by things that we already understand today

lucy: good conversation around the big project, tracking (1:47-1:48) get some nourishment

developing the visual basically, we are all interested in very similar visual parsing of the data:

could we make a poster of gripes? do we use the data in a video?

the form of people coming to our final showing)

do we create a 3d prototype?

as lane said, sharing our IDEAS RESEARCH

camila: like that idea - mapping btw two hands

maybe (olivia) begin to brainstorm about how we could visually collate our

catherine: might be interested to map btw behavioral gesture and â&#x20AC;Ś mapping behavior to mode of communication i.e. do we gesticulate more when speaking face to face than we do on the phone? over ichat etc.?

micah: excited to be playing with the visual aspect of this

would like to have outside pressure (in

keitaro: hmmmmmmmmm........hmmmmmmm......want to keep coding and try to figure out how it might be possible to manipulate typeface in 3d through processing. simple prototyping?

24 / 01 / 2011 NIU:

20/01/2011

24 / 01 / 2011 CATHERINE: 33

27 / 01 / 2011 KEITARO: Considering again from the beginning, I made dot fonts on processing. This only works on PC so far, I want to connect with G-Speak and also make it 3D (like spheres).

20/01/2011

27 / 01 / 2011 OLIVIA AND MICAH: We finally had a little breakthrough tonight! micah and i were able to get a bezier sketch running such that each of us could control one of the origin points with a gripe. camila took pictures, which we will post as soon as we have a chance to upload them to the laptop.

WEEK 5 January 31 - February 6 / 2011 Final Work Week

20/01/2011

Micah's sketch

Niu's sketch

Camila's drawing sketch

20/01/2011

group discussion - january 31th, 2011

-We need to update the library on the gspeak system -there is now a second winter-session acct, perhaps we could upload the 2nd library there, so that if there is any problem with one, we can use the other -via email, let’s compile a guestlist containing specific members of gd and d+m to invite to our showing next monday night -kelly, evelyn, beth, dylan, andrew etc.? -probably we should refrain from inviting the whole of each department so that the space is not overwhelmed. -we can also always perform a demo for people who are not there at our “show-ntell” next monday please note: the other g-speak acct info acct name: risdwint2011 password: lines2001 --we might be able to save our sketches when working on this acct, as the problems we’re having with saving on clement’s acct have to do with permissions. everybody, check-in!: keitaro: -working on a sketch called “dotfont” -the size of the dots composing the type are determined by how many times you click on the sketch with the mouse -each of the circles jiggle slightly and leave a trail behind them when you move the word around the screen -next step: upload to gspeak and control dot font through z-position of the hand -also, try to control each letter separately

(one glove:one letter) -the dots grow to a specific size, then return to their original small size -lucy wonders if there could be one row of dots rather than two? if your investigation is about type, perhaps you could start it from the smallest version i.e. start with one dot? then bring in more dots (w magnetic force?) and see the type assemble. that would create a narrative: how the type gets built. -petri dish growth as a jumping off point? -the “jiggling” of the dots suggests molecular movement -how could it be pushed forward? what if the dots were scattered, and went looking for each other? -would it be possible (if each letter were a separate object/function) to manipulate them more carefully? -lucy says: what word you’re teaching seems to matter to me. why risd? what about hotdog? blood? there ought to be some deeper connection between the meaning of the word and the form you’re giving it. -lane says there’s almost something intestinal about the letter-forms, mentions cell mitosis. creation of a creature as a typographic image? -growth is at the crux of this exploration -when things grow, we don’t necessarily think of them as done. we’re just seeing them in one stage of their growth. micah/olivia: -THIS SKETCH (the one we just showed you all):

-the origin points of the bezier curves are controlled by the x and y position of pilot 1’s hands in the g-speak system. -the bezier handles (aka control points) which control the curvature of the lines between the origin points controlled by the x and y position of pilot 0’s hands in the g-speak system. This is functional, but too exaggerated right now, so we need to scale it down. -OUR NEXT STEP: -we would like to map the bezier handles (aka control points) which control the curvature of the lines between the origin points to the x and y position of the hand in the g-speak syste m. -OUR NEXT, NEXT STEP: -get pilot 0 and pilot 1 working in concert such that each pilot’s left hand controls their respective origin point, and each pilot’s right hand controls their respective bezier handles (aka control points).

typographic characters, or perhaps other shapes? -project catherine has been working on alone: understanding how the system follows quality of touch -videotaped an experiment this morning -playing around with the idea of drawing by touch rather than sight -interested in using the computer as a barometer to measure quality of touch -sketch shows catherine running g-speak gloved hands over her own body to produce forms on the screen -looks like duchamp’s nude descending a staircase (cool) -what about incorporating an audio element? -keep thinking about all the possible ways to enrich this artform camila:

--notes--

-has been working on exercises regarding grabbing points and moving them?

-maybe we should incorporate a thin line of light into the bottom of the screen?

-working on a process book for her open research class

-how does having to kneel when interacting with the sketch change the meaning of the sketch?

-it is awesome! incorporates notes, images etc. documenting our processes as a group

-how doe we want to *FRAME* ourselves on “stage”? (or, is it even a stage?) -could the control points of the bezier curves become something other than dots? letterforms? geometric shapes? -micah shows beautiful sketch of rotating white circles catherine:

-please send camila any images + captions you would like to see in the book niu: -demonstrating his sketch wherein the city line is controlled by niu’s gripes as he moves his hands across the screen -there is also a sailboat(?) hanging out in the sky

-feeling a bit out of the loop

-the ship is three dimensional! and it changes!

-b/c we are all new to coding, it is difficult for her to look at the code and understand what’s going on

-perhaps the buildings could be stretched like taffy?

-perhaps we can split up the work as we continue: micah & olivia can continue trying to control the bezier handles? camila & catherine can work on replacing the circular bezier control points with

-visual NARRATIVE -what does this mean? what’s it saying? what’s it about? -also, think details - why upper left box? why nav(?) bar on right? smooth out the details in the graphics etc. so that the city looks like it is growing better.

20/01/2011

Olivia's tree sketch

Olivia's Diffusion Limited Aggregation

WEEK 6

February 7, 2011, Final Presentation

20/01/2011

a n

d e

a M M M

M a a

M a

A collection of images produced through mapping the location and character of hands in space. Written in Processing at the Rhode Island School of Design in the winter of 2011.

a d

d e

Micah's Poster

(1) Two-hand drawing. Hand 1: Micah, hand 2: Camila. (2) Clemen Dancing Drawing

Camila's Poster

20/01/2011

Catherine's Poster