Social Bubble

SOCIAL BUBBLE WORKSHOP 2.0 Apostolos Despoditis Evangelia Magnisali

Chhavi Mehta

Tianyuan Li

Ziyuan Huang

Please watch all the sensor tests and prototypical videos on your youtube channel https://www.youtube.com/channel/UCunnJYyC2fpeEcaOri_NUBw/playlists

CONTENTS

1 Concept

2 Research

3 Performance

• Context

•

Sensors

•

Performance

• Concept

•

Data Research

•

Sensor

• Concept Video

•

Geometry Research

• Workflow

•

Transformation

About Sensor Workflow Data Processing Geometry Transformation

Concept

SOCIAL BUBBLE Context Concept Concept Video Workflow

Context

Concept

The COVID-19 pandemic has become a global emergency. It spreads mainly among people who are in close contact (within about 2 m) for a prolonged period. Spread happens when an infected person coughs, sneezes, or talks, and droplets from their mouth or nose are launched into the air.​ Now as communities are reopening and people are in public more often, physical or social distancing has become a critical part of our everyday lives. Social Distancing helps slow the spread of the disease and our project “Social Bubble” aims to further this agenda.

Concept Video

The social bubble helps maintain social distancing and thus limits the spread of Covid-19. It not only warns a person when they are approaching a 2m distance from another person, but also visualizes if someone is coughing or sneezing. Additionally it displays the persons body temperature, pulse and whether they are walking or stationary, thus providing people the information required to make an informed decision about who they can form a support bubble with and whom to distance themselves from.

Youtube Link: https://youtu.be/C5iwJGJHFUk

Scene 1

Scene 2

Scene 3

Scene 4

Scene 1 : User 1 is a healthy occupant of the social bubble. Two Arduinos (connected to sensors) are fastened to him. The sensors collect data about the users movement, location, health and activity status.

Scene 2 : As User 2 walks, the Social Bubble follows her and in real-time reflects her health and activity status.

Scene 3 : User 3 has a cough and fever. Other viewers can see this through the appearance of his bubble.

Scene 4 : As two users move closer and are about to violate the social distancing rules, the bubble starts warning them through changing its appearance

Scene 5

Scene 6

Scene 7

Scene 8

Scene 5 : Despite the warning signs, if two healthy people walk towards each other, their bubbles start to interact.

Scene 7 : If even one of the users is unhealthy, their bubbles don't integrate.

One of the users should make a conscious decision to walk away.

Scene 6 : If they move further closer, the bubbles integrate into one.

Scene 8 : If neither of them does, the bubble acts as a guidance tool and helps the healthy user navigate to a safe distance away from the unhealthy user.

Workflow

Sensor with Arduino

Arduino Scripting

Filefly

Data is collected through Sensors connected to an Arduino

The Arduino is programmed using IDE software to collect data

Input collected from input into Grasshop Grasshopper plugin

Physical Input

Processing

Data into Grassh

Grasshopper

Fologram

m Arduino is pper through a n called Firefly

Geometry is trasnformed based on collected data through grasshopper

The gasshopper simulation is used to produce a real-time output in Augmented Reality using Fologram

hopper

Geometry Transformation

Augmented Output

Research

SOCIAL BUBBLE Sensors Data Research Geometry Research Transformation

Sensor

Frequency Spectrum

Grasshopper

Arduino Uno

Pulse Sensor

Temperature Accelerometer Sensor Sensor

Ultrasonic Sensor

Sensors

Ultrasonic Sensor Accelerometer Sensor Temperature Sensor Pulse Sensor Sound Sensor Arduino Fologram

Data Research

Sensor

Data

Pulse Sensor

945-965

Temperature Sensor

35-42℃

Sound Sensor

1500-5800Hz

Accelerometer Sensor

≥0 natural number

Ultrasonic Sensor

0 - 5m

Ultrasonic Sensor

Distance between users

Accelerometer Sensor

Direction of movement + Activity Level of users

Temperature Sensor

Body Temperature of User

Pulse Sensor

Pulse Visualisation

Sound Sensor

Detects sound of cough or sneeze

Sensors have been used to collect data about users that can help other users make informed decisions about who to interact with and whom to distance themselves from. Through sensors, we are able to gather information about the user like their pulse, body temperature, direction of movement and distance from another person.

Geometry Research

Geometry Geometry Geometry Research Research Research Geometry Subdivision in Two Dimensions

Geometry Geometry Geometry Subdivision Subdivision Subdivision in 2Din 2Din 2D

Plane Plane Plane Plane

Equilateral Triangle SmallTriangle Triangle Equilateral Equilateral Equilateral Small Triangle Small Triangle Small Extrusion triangletriangletriangle Extrusion Extrusion Extrusion

Geometry Geometry Geometry Subdivision Subdivision Subdivision in 3Din 3Din 3D

Geometry Subdivision in Three Dimensions

Shpere

Equilateral Triangle

Small Triangle Extrusion

Dynamic Surface

Transformation 01 Extrusion

02 Rotation

Transformation 03 Change Color

04 Geometry Scaling

Transformation 05 Deformation

Transformation 06 Resolution (Number of Polygons)

Performance

SOCIAL BUBBLE Performance Sensors About Sensor Workflow Data Processing Geometry Transformation

Performance

Data collected from sensors is then used to change the appearance and performance of the bubble. For example, the accelerometer is used to determine whether a person is walking or not and how active they have been. The sound sensor defroms the bubble around a person when they make loud sounds like coughing or sneezing and the ultrasonic sensor warns people when they are getting closer than 2m to each other.

沮丧 SENSOR

Accelerometer Sensor

Human

BUBBLE PERFORMANCE

Resolution

(number of polygons)

Temperature Sensor

Change Color

Pulse Sensor

Saturation Color

Sound Sensor

Deformation

Ultrasonic Sensor

Extrusion

Mode: Single User

Rotation Accelerometer Sensor

Direction

Ultrasonic Sensor

Intergration Bubbles

Mode: Multi-Users

Sensor - Ultrasonic Sensor About

The HC-SR04 ultrasonic sensor was used to detect the distance between users. It uses sonar(like bats) to calculate the distance and has a range of 2cm to 500cm.

Limitation: Since the Ultrasonic sensor is not supported by Firefly, we had to use IDE (Step 1) to program a separate arduino to detect the distance and the data collected through IDE was then read in firefly using the component Serial Read (Step 3). Serial Read provides the generic data collected from an arduino through IDE as compared to UNO Read which provides the data per output pin of the Arduino. Serial Read only displays in grasshopper what is displayed in the Serial Monitor in IDE (Step 2) , it doesn’t read the arduino itself.

Output: It provides an analog output of the distance detected between the sensor and an object in centimeters. (Step 2 & 3)

Use: We used this sensor to detect the distance between users.

Sensor Workflow

const int TrigPin = 2; // const int EchoPin = 3; // float cm; // void setup() { Serial.begin(11520); // pinMode(TrigPin, OUTPUT); pinMode(EchoPin, INPUT); } void loop() { digitalWrite(TrigPin, LOW); // delayMicroseconds(2); // digitalWrite(TrigPin, HIGH); delayMicroseconds(10); digitalWrite(TrigPin, LOW); //

}

cm = pulseIn(EchoPin, HIGH) / 58.0; // cm = (int(cm * 100.0)) / 100.0; / Serial.print("Distance:"); Serial.print(cm); Serial.print("cm"); Serial.println(); delay(500);

ping: 8 cm ping: 5 cm ping: 11 cm ping: 99 cm ping: 123 cm ping: 136 cm ping: 137 cm ping: 139 cm ping: 0 cm ping: 138 cm ping: 108 cm ping: 139 cm ping: 139 cm ping: 103 cm ping: 0 cm ping: 140 cm ping: 141 cm ping: 112 cm ping: 141 cm ping: 134 cm ping: 142 cm ping: 152 cm ping: 155 cm ping: 158 cm ping: 163 cm ping: 172 cm

Step 01

Step 02

Step 03

Scriting in Arduino Uno

Distance Data in Arduino Uno

Distance Data input to Grasshopper

Processing Data Extrusion

Ex t Le rudin ng g th

Geometry Transformation The ultrasonic sensor detects the distance between 2 people carrying a sensor each. As people move closer, it acts as a means of warning them if they are about to voilate the social distancing rules. When the distance between 2 users is between 2.5 to 5 meters, extusion happens as a first warning. The closer they get to each other, the more extrusion happens.

Data in Sensor (Distance)

≥5.0m

4.0m

2.5m

Extruding Length Parameter

0

10

25

p Mode: Mu

p Data Proc

p Reaction:

p Sensor: U

Rotation

Geometry Transformation The data from the ultrasonic sensor is used to trigger the rotation behavior at greater proximity as a second warning. This happens at the 2.5m mark and exaggerates as users approach 2m.

Data in Sensor (Distance)

≥2.5m

2.3m

2.0m

Opening Parameter

0

10

30

Opening Angle (directly)

55°

50°

0°

Sensor - Accelerometer Sensor About

A GY-521 Accelerometer is used to detect the acceleration of an object and its direction of movement.

Limitation: The accelerometer is not supported by firefly directly either so an arduino is programmed in IDE (Step 1) to read the values which are then input into grasshopper using a Serial Read. (Step 3)

Output: This sensor provides an analog output of acceleration as well as angular acceleration in the x,y and z axis. For our project we have only extracted the acceleration, not the angular values to avoid inaccurate calculations in determining the direction of movement and counting the number of steps.

Use: Accelerometer is used to determine the direction of movement of the user as well as to count the number of steps taken.

Sensor Workflow

#include<Wire. h> const int MPU=0x68 int16_t AcX, AcY, AcZ, Tmp, GyX, GyY, GyZ void setup() { Wire. begin() ; Wire.beginTransmission (MPU) ; Wire write(0X6B) ; Wire. write(0) ; Wire. endTransmission(true) ; Serial begin(9600) ; } void loop() { Wire. beginiTransmission( MPU) ; Wire. write(0X3B) ; Wire,endTransmission(false) ; Wire. requestFrom(MPU, 12, true) ; AcX=Wire. read()<<8 Wire. read() ; AcY=Wire. read()<<8 Wire. read() ; AcZ=Wire. read()<<8 Wire. read() ; GyX=Wire. read()<<8 Wire. read() ; GyY=Wire. read()<<8 Wire. read() ; GyZ=Wire. read()<<8 Wire. read() ; Serial. print("X = "): Seral.print(AcX) ; Serial. print(" I Y= "): Seral.print(ACY) ; Serial. print (" I Z = "): Serial.printin(AcZ) ; delay(333) }

X = -12864 ┃ Y = 3364 ┃ Z = -9760 X = -12876 ┃ Y = 3252 ┃ Z = -9708 X = -12894 ┃ Y = 3424 ┃ Z = -9716 X = -12828 ┃ Y = 3320 ┃ Z = -9666 X = -12924 ┃ Y = 3372 ┃ Z = -9536 X = -12864 ┃ Y = 3440 ┃ Z = -9728 X = -12844 ┃ Y = 3460 ┃ Z = -9668 X = -12932 ┃ Y = 3356 ┃ Z = -9580 X = -12876 ┃ Y = 3328 ┃ Z = -9652 X = -12908 ┃ Y = 3336 ┃ Z = -9656 X = -12756 ┃ Y = 3360 ┃ Z = -9432 X = -12840 ┃ Y = 3372 ┃ Z = -9772 X = -12964 ┃ Y = 3388 ┃ Z = -9684 X = -12920 ┃ Y = 3400 ┃ Z = -9828 X = -12888 ┃ Y = 3472 ┃ Z = -9548 X = -12832 ┃ Y = 3916 ┃ Z = -9620 X = -12896 ┃ Y = 3452 ┃ Z = -9664 X = -12800 ┃ Y = 3344 ┃ Z = -9712 X = -12844 ┃ Y = 3200 ┃ Z = -9568 X = -12880 ┃ Y = 3324 ┃ Z = -9552 X = -12852 ┃ Y = 3476 ┃ Z = -9676 X = -12892 ┃ Y = 3352 ┃ Z = -9708 X = -12760 ┃ Y = 3428 ┃ Z = -9928 X = -12750 ┃ Y = 3500 ┃ Z = -9800

Step 01

Step 02

Step 03

Scriting in Arduino Uno

Data in Arduino Uno

Data input to Grasshopper

Processing Data 1.0 Use: Direction Giver

z y

-x

x

-y -z

The Accelerometer is also used to determine the direction of movement of the user and the distance it travels. The sensor detects movement of a person though collecting acceleration per the x, y and z axis. Example if the person moves along the x axis the values increase in the x or -x axis and the same for the y and z axis. The axis are predetermined and specified on the accelerometer sensor.

Processing Data 2.0 Use: Acivity Tracker

Movement in the Z-axis is recorded

Difference between 2 consecutive movements is calculated

If difference is larger than 1000, it is counted as a half step

The Accelerometer is also used as an activity tracker. It counts the number of steps by a person by setting a threshold value in the z direction and counting it as a step once that value is crossed. Example (diagrams) the value at the beginning is x,y,z. Once you take a step the value in the z direction changes twice. Once when you lift your foot (diagram) and once when you place it on the ground again (diagram). This is counted as one step.

The number of steps taken in a minute are used to determine the users activity level.

If no steps have been taken in a minute, the user is stationary and their bubble has a minimal polycount.If they have taken less than 25 steps in a minute they have a low activity and if they have taken over 25 steps a minute they are highly active and have the most intricate bubble with the highest polycount.

The more active a user is, the more they potentially would have interacted with other users. Also knowing how quickly they are walking will help users

Geometry Transformation Resolution (number of polygons)

Situation

Data in Sensor

Resolution (number of polygons)

Stationary

Low Activity

Normal Activity

High Activity

0 Steps in a minute

＜25 Steps in a minute

＜35 Steps in a minute

＞35 Steps in a minute

20

80

320

1280

Youtube Link: https://www.youtube.com/watch?v=B9fxnFPSE_A&list=PL5cnB09TrD3nyXPefOfbJIah1FAWujVSw&index=2

Stationary

Low Activity

High Activity

- Ultrasonic p Sensor Data Processing: Distance + Sensor Direction

+ Accelerometer Sensor

p About Sensor: Ultrasonic Sensor + Accelerometer Sensor

Accelerometer Sensor

Ultrasonic Sensor

The Accelerometer and Ultrasonic Sensor together are used to trigger an influential point. When users are facing each other and the distance between them is 2 meters, an “influential point” is triggered. The closest point between the surfaces of two bubbles is computed and generated on each bubble; this determines the area where integration of two bubbles happens.

*

* Youtube Link:

https://www.youtube.com/watch?v=UQOcK3pKsCk * indicate location of influential point on the sphere The vidoe shows the impact of an influential point on the bubble

*

*

Geometry Transformation Integration

Situation: Individual

Situation: Extrusion

Distance: ＞ 5.0m

Distance: 2.5 - 5.0m

Data in mapper: 80

Data in mapper: 20 - 60

Different Data Distruibute to Different Behaviors After remap the distance, we seperate the disatance into 4 intervals. In different distance collected from Ultrasonic sensors, bubbles will perform different behaviors with getting close to another. In this case, serveral intervals form 3 Buffer zones in social distance to make sure the security of communication during pedemic.

Situation: Rotation

Situation: Integration

Distance: 2.0 - 2.5m

Distance: 0.0 - 2.0m

Data in mapper: 8 - 20

Data in mapper: 0 - 8

Integration of 2 Bubbles

Sensors collecting data in real-time

View from outside Bubble

Youtube Link: https://www.youtube.com/watch?v=AHZHPmndbsU&list=PL5cnB09TrD3nyXPefOfbJIah1FAWujVSw&index=3

View from inside Bubble

Process of Integration

5m

3m

2.5m

2.2m

2m

1.8 m

1.5 m

0.3 m

Integration of 3 Bubbles

View from outside Bubble

Youtube Link: https://www.youtube.com/watch?v=YMZWWXUSIQo&list=PL5cnB09TrD3kGehF4xVmnaH8Yc98f5fKG&index=2

View from inside Bubble

Sensor - Temperature Sensor About

The 3 Pin NTC Thermistor temperature sensor is used as it is one of the most sensitive ones and can be used to detect the accurate differences in human body temperature. It is used with a potentiometer to adjust the detection threshold.

Limitation: The temperatre sensor is not supported by firefly directly so it has to use an arduino programmed to read the values which are then input into grasshopper using a Serial Read.

Output: This sensor provides an analog as well as digital output.

Use: This sensor is used to detect the body temperature of the user and determine if it is high or low.

Sensor Workflow

const int digitalPin = 4; int analogPin = A0 ; const int ledPin = 13 int Astate = 0; void setup() { PinMode(ledPin, OUTPUT) PinMode(digitalPin, INPUT) Serial.begin(9600); } void loop() { Astate = analogRead(analogPin); { Serial. print("A0: "); Serial. println(Astate); } delay(20) }

A0:375 A0:375 A0:375 A0:375 A0:375 A0:375 A0:375 A0:375 A0:375 A0:375 A0:377 A0:377 A0:377 A0:377 A0:378

Step 01

Step 02

Step 03

Scriting in Arduino Uno

Data in Arduino Uno

Data input to Grasshopper

Processing Data

Color

Transparency

Geometry Transformation Color Change

Data in Sensor (℃)

36 - 37.5℃

37.6 - 38℃

38.1 - 39℃

39.1 - 40℃

>40℃

Color

123, 169, 169

101,124,189

176,146,189

255,191,99

185,91,111

State

normal

low-grade fever

moderate fever

ardent fever

hyperpyrexia

View from outside Bubble

Normal Temperature

High Temperature

View from inside Bubble

Temperature Sensor

Youtube Link:

https://www.youtube.com/watch?v=7Rvho_iS4Yg&list=PL5cnB09TrD3nyXPefOfbJIah1FAWujVSw&index=5

Sensor - Sound Sensor About

The inbuilt Frequency Spectrum from the Firefly plug in is used as a Sound Sensor. It is used to detect and analyse sounds made by the user.

Limitation: The frequency spectrum causes a slightly delayed reaction.

Output: It produces analog outputs of Note Volume, Peak Frequency and Peak Note.

Use: This sensor is used to detect the sounds made by the user.

Processing Data

Geometry Transformation Deformation

Data in Sensor (hertz)

1500

3220

4080

5800

Deformation Parameter (line length）

max:100 min:100 difference:0

max:78.4 min:127.6 difference:49.2

max:67.9 min:141.4 difference:73.5

max:47.8 min:169.3 difference:121.5

View from outside Bubble

View from inside Bubble

Bubble in base state

Bubble when it detects sound

Youtube Link:

https://www.youtube.com/watch?v=JnmT-zwDeJg&list=PL5cnB09TrD3nyXPefOfbJIah1FAWujVSw&index=4

Bubble in base state

Bubble when it detects sound

Youtube Link:

https://www.youtube.com/watch?v=veSVhR839QM&list=PL5cnB09TrD3nyXPefOfbJIah1FAWujVSw&index=7

Sensor - Pulse Sensor About

A plug and play heart rate sensor is used to visualise the heartbeat of a person. It detects the change in volume of capillary tissues through detecting the variation in light transmission and reflection.

Limitation: The sensor is very very sensitive, even minor movements affect its output. Thus the data collected from this sensor had to be remapped very carefully. Additionally, the sensor takes some time to start giving accurate measurements of the heartbeat so the first few values collected had to be nulled out from the data recorder through means of list management.

Output: This sensor provides an analog output of the occurrence of the Q,S and R waves. It can be read directly in firefly through the Uno Read component

Use: It is used to visualise the occurrence of the Q, S and R waves.

Sensor Workflow

int sensorPin=A0; double alpha=0.75; int period=20; void setup() { pinMode(A0,INPUT); Serial.begin(115200); } void loop() { static double oldValue=0; int rawValue=analogRead(sensorPin); double value=alpha*oldValue+(1alpha)*rawValue; Serial.println(value); oldValue=value; delay(period); }

986.42 990.56 988.42 983.57 983.68 982.26 984.94 985.21 988.16 986.87 989.90 990.42 991.07 992.30 993.98

Step 01

Step 02

Step 03

Scriting in Arduino Uno

Data in Arduino Uno

Data input to Grasshopper

Processing Data

Geometry Transformation Saturation Color

Data in Sensor

S wave

Median wave

R wave

Input

720

780

850

Color

237, 238, 240

123, 169, 169

58, 89, 92

View from outside Bubble

S Wave

Median Wave

R Wave

View from inside Bubble

Pulse Sensor

Youtube Link:

https://www.youtube.com/watch?v=u9ajofD2yas&list=PL5cnB09TrD3nyXPefOfbJIah1FAWujVSw&index=6

Interaction between healthy and unhealthy bubble Integration of bubbles is a conscious decision that must be made by the users. When a person has a fever, cough or cold you get to know through the appearance of their bubble. Other users must stay away from the unhealthy person and not integrate their bubbles with each other.

Additionally, the Social Bubble also acts as a guiding tool. If the user is moving close to another user who is not well, after a second warning, the bubble itself starts moving away from the unhealthy person. When healthy users notices this, they must then start following the bubble as it guides them to a safe distance away from unhealthy users.

Sensors collecting data in real-time

View from outside Bubble

View from inside Bubble

Youtube Link:

https://www.youtube.com/watch?v=_6pK1a2JjGk

Two bubbles approaching each other

One bubble exhibits high temperature and its bubble turns red

The same bubble also deforms as its occupant coughs/ sneezes

The bubble of the healthy person starts moving away from the unhealthy person, guiding the healthy occupant to a safe distance away

Performance System

Scene 1

Scene 3

Scene 2

Scene 4

Scene 1 : individual sensors used and the transformations they generate Scene 2 : performance of an isolated bubble with all sensors and performance transformations combined Scene 3 : Sign of an unhealthy person (high temperature, hence the bubble is red)

Scene 5

Scene 6

Scene 4 : Sign of an unhealthy person (bubble deforms as occupant coughs or sneezes) Scene 5 : Healthy person must make a decision to walk away Scene 6 : If nobody walks away, the bubble of the healthy person starts moving, encouraging them to walk away from the unhealthy person and guiding them to a safe distance away

Scene 7

Scene 8

Scene 9

Scene 10

Scene 7 : Integration of bubbles start when occupants of both bubbles are healthy Scene 8 : Two healthy people form one healthy Social Bubble Youtube Link:

https://www.youtube.com/watch?v=WfCgNLotePQ

Scene 9 : Integration of three bubbles start when all occupants are healthy Scene 10 : Integrated Social Bubble of three healthy people