JosephWu_InteractiveHCI_Portfolio

RobotCube This research focuses on examining the interaction between the machine and spatial perception. We are concerned about whether there are possibilities for a new correlation among the space, machine and image. This research combines the large-scale robotic arm with the interactive image, and give full play to the technological and art potential of dynamic projection to create the sensory experiencing combining the virtuality and the reality.

This research is devoted to the field of spatial sensing and human-machine cooperation, and combines the generative art, robot interaction, and real-time 3D mapping. Meanwhile, this research adopts motion and automation as an innovational medium for further development. The works are created through improvisation, enabling the performer and audiences to coexist.

Robotic Solution Cooperative (ROSOCoop) - Fall 2020 Professional Work Adviser: Prof. Yu-Ting Sheng and Shih-Yuan Wang Project Leader: Joseph Wu Group Member: Che-Wei Lin, Tiger Lin, Peng-Hsiu Tung Contribution: image production / robot trajectory planning / real-time mapping / design development / diagram drawing

Master

tx ty tz

Slave

rz ry rx

I.K Solver Analysis Robotic Mapping System

controller

The traditional projection mapping has its limitations. The projected

custom cube

objects must be rigid and stable.

This research developed dynamic projection mapping technology to capture moving cubes installed on the robotic arm. After calibration, the

10%

tcp real-time communication

cubes can coincide with our visual

trajectory control

perception. The Robot movement is

dynamic projection mapping

projector calibration

(sev

collaborated

enth

-axi

with

the

projected

image to express art feelings and

s) li

nea

r mo

tion

create the visual effect that com-

trac

ks

bined virtuality and reality.

Images of this performance are created through the derivative design approach to express the idea that art should be an integration between the natural totem and the technology of the era. Therefore, the algorithm process are integrated into images.

Modern technologies, such as computational design and robot movements, are combined to create a brand-new art experience.

Euler Angle Transformation Grasshopper to Touchdesigner

Z Transform Order : Scale Rotate Translate Rotate Order : Rx Ry Rz

Y Z

3

5

tx = ptx ty = pty tz = ptz

B Y

X Y

4

2

1

3

4

6

rz = prz+90 ry = pry rx = prx

Z X

1

Y

5

4 2

6 1

5

1

2

5

3

2

3

4

5

2

6

X

3

6

6

4

1

N The Euler Angle Transformation Mechanism is used to transform the six-axis point location of TCP on Grasshopper into the six-axis coordinate on Touch Designer so that both the software terminal can achieve synchronization.

Receiving TCP in Real-Time

controller

10%

euler angle transformation

tcp real-time communication

dynamic projection mapping

The TCP/IP is used for sending and receiving information over a TCP/IP connection between two remotely

located

computers.

This performance captures the real-time TCP data from KUKA Controller to touchdesigner to achieve

real-time

projection

mapping.

Virtual image plane

Projector Calibration Projector calibration enabled the projected image to match with

Object plane

the movement trajectory of the cube. We observe the projected image to infer the actual lens position, making the virtual and real closer to zero error.

Rail of the linear motorized stage

Collapse Project This research focuses on the interaction between humans and images. Utilize projection as the medium to create an interactive experience combining virtual and reality. Also, study how to involve the audience in the performance. The final results stages in the annual exhibition of the Institute of Architecture of NYCU. This

performance

focuses

on

spatial

perception

and

human-computer collaboration, combining Generative art, Body Movement, and Mapping. Meanwhile, utilize the sensor to capture the observer's gestures and body movements, making the work interactive in real-time, allowing the coexistence of the performer and the viewer.

NYCU ProjectProject Exhibition - Spring 2021 Group Work Adviser: Prof. Shih-Yuan Wang Group Member: Joseph Wu, Che-Wei Lin Contribution: image production / mapping / human-comptuter interaction / design development / diagram drawing

Interactive Mapping System Reconstruct

the

architecture

image to show the interweaving between media and space in a

touchdesigner

combination of simulated reality.

The dynamic image presents the diachronic

changes

of

the

space. By combining the body of the visitor and the image, it gives immediate

feedback

on

Signal Communication

Mapping Column

the

impact on perception.

Architectural drawing is not only to investigate the feasibility of

Human-Computer Interaction

the construction but also to produce new video texts. The meaning of the image exists in architecture and the process of its performance.

Kinect Motion Sensing & Calibration

Projection Calibration & Mapping

Robot-Controlled Camera The workshop utilizes the IIWA Lady, a human-machine collaborative robot, to explore the spatial perception experience extended by the virtual space and create unique animation effects. This design uses Unreal Engine to create virtual scenes. It also utilizes TACO IIWA software to collaborate with robots, exchanges information with Unreal Engine in real-time.

Eventually, control the rendering effects of its virtual cameras and animations.

In detail, this study developed a custom interface between the ROSOCoop robots and Unreal Engine. Utilize the robot as an intuitive input device for real-time control of virtual cameras within the Unreal Engine scene. It controlled not only camera position/rotation but also focal length, speed, etc.

Unreal Engine Robot-Controlled Camera Workshop SCI-ARC, ROSOCoop, FCU, NYCU, IDF - Summer 2021 Adviser: Prof. Alexin Marfin, Yu-Ting Sheng and Shih-Yuan Wang Team Member:

Han Lin, Chung-Chieh Cheng, Eric Chen, Lee-Yo Chen,

Xiao-Tian Yu, Hsiang-Ting Chien, Tsung-Han Tsai, Ting-Chia Chen, Kuo-Feng Wei, Kang-Chia Wei, Hung-Wen Lu, Zhao-Wei Liu, Joseph Wu, Shih-Kai Fan, Chun-Yu Lo, Ting-Xuan Jiang, Hung-Cheng Chun, Yung-Chi Liu, Yu-Sheng Liu

Medium X light This research is based on the premise of the 3D printing methodology, which incorporates the discussion of the immaterial (light) and the material (transparent Polylactic Acid). Through testing various parameters that enable light to contain inside the medium and also control the refraction of light and the path of the light. This research proposes a new method to revise light trajectories and reflection and also create a unique light atmosphere.

Under process, model an adapter that holds the PLA plastic gun and mounts it to the flange of the robotic arm, operating different control variables such as density, layer height, printing path, and different light sources, to analyze light imaging under different conditions. Thus, integrate these conclusions to generate individual light performance.

As for the main goal in the next stage,we not only want to create pattern or alphabet but a space or atmosphere which affects people’s emotion or vision in a more directly way.

Feng Chia University School of Architecture - Spring 2019 Acadamic Work - Design Studio Adviser: Prof. Yu-Ting Sheng and Shih-Yuan Wang Team: Joseph Wu, Han Lin, Tsung-Han Tsai Contribution: material exploration / robot trajectory planning / end effector design / design development / diagram drawing

Concept

Pattern

Refraction Process

A.

Printing Pattern

Spacial Atmosphere

Material

Light Source

Design

Light Refraction

Screen

Imaging

Point Source Line Source Divergent Source Parallel Source

Robotic printing 3D printing Epoxy casting

Lineaer bending Moiré pattern Irregular waving

Two-dimensional plane Three-dimensional object Three-dimensional space

Light resource Focal length Distance

Polylactic Acid

printing pattern light gathering

B.

Spacial Atmosphere

Light Source Point Source Line Source Divergent Source Parallel Source

Pattern

Infill Pattern

light [Acid]

( refraction )

Design Robotic printing 3D printing Epoxy casting

Observer

Refraction Imaging

Two-dimensional plane Three-dimensional object Three-dimensional space

Opaque part of the object is used as the imaging part

Retention of Marginal Ray

Property

Value

Technical Name

Polylactic Acid (PLA)

Chemical Formula

(C3H4O2)n

Melt Temperature

PLLA: 157 - 170 °C

SLASH TYPE

CONCENTRIC

OCTAHEDRON

size: 100mm*100mm*2mm speed: 20 mm/s infill: 100%

size: 100mm*100mm*2mm speed: 20 mm/s infill: 100%

size: 100mm*100mm*2mm speed: 20 mm/s 50% infill:

TRIANGULAR size: speed: infill:

100mm*100mm*2mm 20 mm/s 50%

CROSS TYPE

HEXAGON

size: speed: infill:

size: speed: infill:

100mm*100mm*2mm 20 mm/s 50%

100mm*100mm*2mm 20 mm/s 30%

Single Light

Single Light

Single Light

Single Light

Single Light

Single Light

Mulitiple Light

Multiple Light

Multiple Light

Multiple Light

Multiple Light

Multiple Light

Curvature Theory

Control Variables Different Method

Radius of Curvature Comparison

Layer Height PATTERN

LAYER HEIGHT

PRINTING PATTERN (1)SLASH TYPE (2)CONCENTRIC

0.05MM/1MM

Speed

Printing Thickness

INFILL PATTERN (1)OCTAHEDRON (2)CROSS TYPE (3)TRIANGULAR (4)HEXAGON

Printing Speed

Small Radius High Curvature Low Radius Low Curvature

CURVATURE

THICKNESS

(1)FLAT SURFACE

0.5MM/1MM/2MM

(2)RANDOM CURVE (2)SIN CURVE

POLY SURFACE

size: 80mm*80mm*1mm speed: 60 mm/s Infill: 100%

size: 80mm*80mm*1mm speed: 60 mm/s infill: 100%

CONVEX LENS-A size: speed: infill:

50mm*50mm*2mm 20 mm/s 100%

CONVEX LENS-B

CONVEX LENS-C

size: speed: infill:

size: speed: infill:

50mm*50mm*2mm 50 mm/s 100%

50mm*50mm*2mm 80 mm/s 100%

a dome. In essence, the radius of curvature tells us how curved a curve is . The

TRANSPARENT

30%/50%/80%

POLY SURFACE

The Radius of Curvature is a number that is used to determine the “flatness” of

COLOR

DENSITY

POLY SURFACE size: 80mm*80mm*0.5mm speed: 60 mm/s Infill: 100%

larger the dome, the less curve, the flatter the concrete.

60° 45°

Single Light

Single Light

Single Light

Multiple Light

Multiple Light

Multiple Light

30° 15°

PLA Extruder Contral Variable

Mulitiple Light

Multiple Light

Multiple Light

Curvature Section 0.65V 5.00V

EXTRUDING SPEED

AIR COOLING TUBE

HEATING CHAMBER TEMPERATURE

0.65V-5.00V

PATTERN TEXTURE

Perpendicular

Imaging

The texture of the pattern is perpendicular to the imagery

190°C

200°C

210°C

220°C

H CHM TEMP : 205°C H GUN TEMP : 2

BUBBLE

EXTR SPEED : 5.00 V

AIR COOL TUBE *2 COOL FANS

PRINTING SPEED

Positive correlation

Transparency

PRG SPEED : 5.0 %

The faster the printing speed, the worse the light transmittance

TRANSPARENCY

PROGRAMING SPEED MAX : 2 M/S

COOLING FAN

focuses

on

the

effects of light performance causing reservation

EVENNESS

1%-100%

research

by reflection, refraction, and light

180°C-220°C

180°C

This

TANGENT SLOPE

Positive correlation

Image Curvature

The positive and negative slope of the tangent affects the direction of the imaging

HEAT GUN TEMPERATURE

through

fabricating

transparent PLA (Polylactic Acid).

Based on several control variables, we aim to control light imaging.

Research the control variables (pattern, thickness, density, layer height, curvature and color) by using FDM 3D printer. We aim to discover the physical

1-9

LAYER HEIGHT

Interval

Light band Dark band

The interval material affects the image of the light band and dark band

factor which infects the light’s refraction and reflection that creates astonishing light pattern.

2

1

3

4

Curvature

5

y=A*sin(2π), π=180°

Wave crest value

6

7

8

10

9

11

Curvature

RefractionXCurvature

A=wave crest y=A*sin(2π)

y=A*sin(2π)

y=A*sin(2π)

A=1

A=2

A=3

y=A*sin(2π)

A=4

y=A*sin(2π)

A=5

Various Points 1

Reflection/

y=A*sin(2π)

y=A*sin(2π)

y=A*sin(2π)

Reflection/

y=A*sin(2π)

2

3

4

5

6

7

8

9

10

11

Reflection/

y=A*sin(2π)

1

2

3

4

5

6

7

8

9

10

y=4*sin(2π)

11

y=4*sin(2π) θ1 =75°

θ2 = 65°

θ3 = 45°

θ3 θ2

θ1

A=6

A=7

A=8

A=9

A=10

θ1

θ2

θ3

Curvature

Layer HeightXDouble-slit interference

POINT FOUR Reflection/

Reflection/

Reflection/

Reflection/

Reflection/

In astronomy, a light curve is a graph of light intensity of a celestial object or region, as a function of time. The light is usually in a particular frequency interval or band. Light curves can

Impact of Layer Height Layer Height

be periodic, as in the case of eclipsing binaries, cepheid variables, other periodic variables, and transiting extrasolar planets, or aperiodic, like the light curve of a nova, a cataclysmic varable star, a supernova or a microlensing event or binary as observed during occultation events.

The study of the light curve, together with other observations, can yield considerable information about the physical process that produces it or constrain the physical theories about it.

y=4*sin(2π)

1

2

3

As for the main goal in the next stage,we Spool holder 2.0 Filament

Reflection

Refraction

Refraction

Refraction

bet but a space or atmosphere which affects people’s emotion or vision in a

Reflection

9

8

7

100mm

6

4滾滾長江東逝水 3

5

80mm

50mm

2

Flange adapter

Spool holder 1.0

1

30mm

Hot wind gun Filament extruder

4

5

Structure board

Curvature

6

Hot end

Air compressor Hot wind fan Reflection

PLA nozzle

Refraction

Refraction

Imaging Are

8

9

We decided to develop a printing system utilizing a welding plastic machine, comBy experience various types of transparent PLA (Polylactic Acid) to get the most suitable parameters for this printing system. Summarize the parameters of fabricating methods and the light imaging

Circle Curve 7

more directly way.

bined with a 6-axis industrial robotic arm.

Various Radius Curve

Refraction

not only want to create pattern or alpha-

210

Rail

while embody the unique light effects

0.65V

205

10%

under this fabricate system.

Laser pen Pen holder Refraction

Reflection

Heater temperature

Rail connector -vertical Rail connector -horizontal

Refraction

Refraction

Reflection

Voltage adjustment of extrusion speed

Program path speed

Experiment Curve Puzzle

Speed Control

Centre Temperature Control / Wind Temperature Control Filament Holder

Cooling System

one l+one mirror c

one c

one c

one c

KUKA KR90 R3100

Filament

Printing Area

Extrusion

one 45° l

one mirror c

one mirror c

one mirror c

Printing Broad

POINT ONE

Object Refraction 31.20°

38.50°

Reflection Light

53.21°

Normal

75.02°

Tangent

In stage one, our experience allows us to obtain the rules in refraction and reflection. The result enables us to control the light as playing puzzle.By seperating alphabet into curves,it makes it possible to image any pattern which is create by curves.

1.54 =

sinθ1 sinθ2

=

sin75.02° sin53.21°

=

sin38.50° sin31.20°

Reflection y=4*sin(2π)

Testing Model 2D Z offset variable

OFFSET 4MM

OFFSET 2MM program speed: filament speed: temperature: cooling fan: nozzle fan: hot wind: size:

level 4.86 5% 205ºc No Yes level 2 level 2

program speed: filament speed: temperature: cooling fan: nozzle fan: hot wind: size:

OFFSET 6MM level 4.86 5% 205ºc No Yes level 2 level 2

program speed: filament speed: temperature: cooling fan: nozzle fan: hot wind: size:

level 4.86 5% 205ºc No Yes level 2 level 2

HOLLOW TYPE

ENGRAVE TYPE-A

ENGRAVE TYPE-B

layer height: size: speed: infill:

layer height: size: speed: engrave: infill:

layer height: size: speed: engrave: infill:

0.05mm 50mm*100mm*2mm 10 mm/s 100%

0.05mm 50mm*100mm*2mm 10 mm/s 50% 100%

0.05mm 50mm*100mm*2mm 10 mm/s 100% 100%

8 sec 6 sec 4 sec 2 sec

WAIT TIME

Dome Model

33mm 100mm

300mm * 150mm * 150mm

A.

B.

A.

B.

A.

B.

A.

B.

PRESSURE

Bright band Dark band Bright band

[01]

[02]

Dark band Bright band Dark band Bright band

RAISING PATH We utilize different printing methods in the printing process so that the [03]

light will be cast on materials to achieve different effects. The light can

[04]

stay in the materials for refraction and reflection. Different changes of 2mm

light and shadow can influence the perception of humans, and create composite materials and a unique light atmosphere. In the future,

6mm

[05]

[06]

PRINTING PATH

these test parameters can be applied to architectural element fabrication process so that “a brick” can include special light effect.

In the final design, we enlarged the scale to a level that humans can perceive. On this basis, the parameters of these control variables can be further integrated to create a special lighting atmosphere. We include the movement of '"rotation" into the final design process. If the light source is projected in a fixed direction, we will find that during the rotation, the position of the projected light source will form a constantly changing curve, and will

experience

spatial

changes

between light and shadow.

The value of this case is to extend the material to the discussion of spatial perception. At the same time, the operation method is based on the integration of materials, robot trajectory planning, and speed.