Portfolio
Design as Democracy Selected Works 2016-2020
Jieyu Zhou Computational Design
Contents Democracy in the design process has always been debated among various
stakeholders. Participatory design is a hands-on action that inspires collective creativity, challenges environmental injustice, and increases the sense of inclusion. Approaches for realizing participatory design are complex and multidimensional, from the urban scale and community scale, to the human scale; from simply informing, to designing and constructing. My passion and goals lie in the intersection of design, urban data, and computational programming. I like to think that these goals share a common thread: using interaction technologies, big data, and deep-learning to facilitate (computationally) understanding, expressing, and engaging during the design process to achieve consensus for all sectors, which are my objectives for graduate studies.
Long-Term Projects //01 Community Co-Generation Platform Rapid scheme generation and evaluation [Deep learning GAN, Grasshopper]
//02 Mechanical Rockery
Traditional rockery simulation and fabrication [Unity 3D, Reinforce Learning, Robotic Arm]
//03 Automatic City
Automatic driving houses [Architectural design]
//04 The Smart Wall
Interactive space creation [Arduino, Emotion Recognition, Interaction]
Short-Term Projects //05 City Context
Classification of Chinese cities’ texture [Deep learning Self-organizing Maps]
//06 Manhattan Workers
Research of housing and transportation in Manhattan [Data Visualization]
Community CoGeneration Platform Team member: Jieyu Zhou Leshan Fu Instructor: Dr. Waishan Qiu
Participatory design is hands-on democracy in action. For over half a century it has guided us in understanding communities, honoring difference, creating vibrant neighborhoods and ecosystems, challenging environmental injustice, and fostering citizenship. However, design team have found it difficult for all the stakeholders to reach a consensus. The platform is intended to solve this problem by using deep learning for quick generating schemes and Grasshopper for rapid evaluation. A community in West Oakland is used as a case study.
Problems in Community Meeting
Difficult to incorporate different needs
The community in west Oakland has a site for urban renewal. However, residents, the government and developers have obstacles to reach a common ground in collective design process. Therefore, a platform for auto-generation is urged to quickly produce design schemes for the community.
Site Social Relationships in a meeting Residents Daily needs Social Media
Cond omin ium
Engagement
Developers We focus on market viability and business opportunities
ial
Big data
Soc
on t uti Sol nmen ver Go
Participatory design
eco no mi De cg vel oal op s ers
Simulation Education
Prototype for the new approach Data
Real-world Problems
Collecting (Smart sensing, crowdsourcing)
Awareness
Presenting (machine learning, augmented reality)
Understanding
Manipulation (Virtual reality, multiagent planning)
Residents
Intervention
Government Build green to support the Sustainable Oakland Program
We need to see this historic station from any angle.
Framework of Co-generation Platform Procedures of Generation
Interactive Module
Composition of the Installation
1 Leisure (Orange) 2 Amenity (Gray) 3 Shop (Blue)
4 Religion (Green) 5 Sports (Red) 6 Greenland (Pink)
Workflow
2. Model
3. Analysis
7 Landmark building in site 8 Site boundary 9 Number of the plan
10 Start and Clean button 11 Radar map 12 S o c i a l e c o n o m i c a l predicted data
13 Environmental analysis simulation
Components Projector
2. Assembling Blocks
1. Different programs of blocks
1. Discussion
The stakeholders can build up models with the pre-set blocks of different programs. The camera on the top will capture what they have built and then sent the image to the mainframe for next generation and evaluation.
Showing the site and analysis images
3. Generating
10
Webcam Capturing which blocks users put 11 9
Picture processing, generating and analyzing
7
6
12
8
5 4 3
4. Analysis
Mainframe
2
Evaluating scheme by environment, economy and ecology indicators
Users assemble blocks on the site model. Results are captured by camera
5. Redoing
Model
1 Not meet needs
Analysis
AI Generated Design Module
GAN ( Generative Adversarial Networks) Algorithm Pix2pix Algorithm In this step, 500 paired images (image A & image B) of appropriate census block groups’ plans(500m*500m) in American cities were collected by Python in Open Street Map. Image A shows the landuse and image B shows the building outline. These paired images are used to train the pix2pix model. Then input the user's sketch into the trained model to generate a plan.
Evaluation Module
Results Testing Catalogue
The evaluation system is created based on different stakeholders' needs, which can be divided to four parts, seven aspects. Some of the evaluation is simulated in Grasshopper in Rhino, others are generated by dataset.
Residents
Job Offering
Government
Developers
View of the historic Station
Sunlight
Revenue
Eco-friendly
Economy
nt nm e o r i Env
Rooftop Solar Potential
Accessibility
CO2 Emission
Environment
Job Offering
Space
2
7 2.9
4
S p a ce
Eco-frie n
dly
7
3.1
1 4.8 2 Form: Data
1 Form: Data
Predicted by: dataset from Google Environmental Insights Explorer
Economy
Rating for Plan 19
3
9
Predicted by: dataset from U.S. Bureau of Labor Statistics and Data USA 3 Form: Model Simulation
Predicted by: Honeybee and Pedsim in Grasshopper
Evaluation Module Results Testing Catalogue
Sunlight
In the West Oakland project, there are 12 plans created by different stakeholders with the aid of the AI tool, pix2pix, and their evaluation is shown below. ① Plan ② Plan ③ Plan ④ Plan
View of Station
Accessiblity
①
②
③
④
1 4 8 11
Sunlight
Overall
View of Station
Accessiblity
Overall
1
7
7
2
8
8
3
9
9
4
10
10
5
11
11
6
12
12
110cm
Accessiblity Score: 7.1 100
12
90
11
10
7
6
5
4
3
2
1
Sunlight Score: 9.5 80
70
60
View of Station Score: 8.3
50
40
30
Paln number, Average score to chose the best one Highway to Sanfransico for commuters Different stakeholders
Ecology data and Economy data
Overall rating using radar map
16th Train Station started in 1912 Simulation Results in Grasshopper Honeybee and ladybug
20
Start Generation and clean bottom
Data Source: OpenStreetMap, U.S. Bureau of Labor Statistics and Data USA
10
1
1
10
20
30
40
50
60
70
80
90
100
110
120
130
140
150
160
170
Mechanical Rockery Team member: Jieyu Zhou
Instructor: Diego Pinochet Xu Zhang
People in the non-design field aided by robotic arms participate more easily in the model making process by the robot performing iteration and arranging complex spatial formation. Users simply place a rough structure and a robotic arm will complete it. Training by reinforcement learning in Unity 3D with a physical engine will ensure the stability of structure completed by the robot, and the layout of the components will just mimic the users’ outline as the robotic arm once be trained according to the datasets.
Mechanical Rockery
Understanding rockery building process Shape Grammar ange its color according to the kinematic velocity and shrink based on the object's location.
Rules
[ Rule 1 ]
Image Source: The Ming dynasty, painter Lin Youlin
[ Rule 3 ]
Computation
[ Rule 4 ]
[ Rule 5 ]
[ Rule 6 ]
ange its color according to the kinematic velocity and shrink based on the object's location. Rules Applied
[
Results
Stone Types
Initial
1
Initial
3
1
5
2
1
1
3
1
1
Initial
2
4
6
4
2
Initial
3
4
3
4
2
+
Triditional process of making rockery
[ Rule 2 ]
]
1
[ ]
3
[
2
]
Rockery Selection
[
Cleaving Rocks Adding small ones Rotation and Addition
4
The rockery is an integral element of Chinese classical gardens. To help the public to understand the space of rockery, the stones are abstracted as dodecahedrons to analyze different forms and combination styles with the method of "Shape Grammar."
]
1 2 3 4
① Zhuo Yu ② Dan Zhou ③ Tai Hu Stone ④ Cang Jian
Mechanical Rockery Building Rockery with Reinforcement Learning Trained by reinforcement learning in Unity 3D with a physical engine will ensure the stability of structure completed by the robot, and the layout of the components will just mimic the users’ outline as the robotic arm once be trained according to the datasets.
Reinforcement Learning State & Reward
Agent
Environment State Transition Action
Reinforcement learning let the agent take suitable action to maximize reward in a particular situation.
Multi-Agent Cooperation Academy External Communicator
Action State & Reward Agent
Action State & Reward Agent
Python Tensorflow
......
......
Action State & Reward Agent
Sharing Instantaneous Information eg. Reward/ Position/ Rotation/ Velocity/State
Cooperative agents share sensation, episodes and learning policies. This sharing speeds up learning at the cost of communication.
Initial Input: Cang Jian
Initial Input: Dan Zhou
Initial Input: Qiu Chuang
Output Training Model using ML
Output Training Model using ML
Output Training Model using ML
Offline Simulation with Reinforcement learnning
Mechanical Rockery
Force: 1
Force: 2
Force: 3
Force: 0.75
Force: 2
Force: 0.2
Force: 1
Force: 4 Force: 0.8
Force: 3
Force: 2
Force: 0.6
Force: 0.4
Results of Reinforcement Learning in Unity 3D ML-Agent and data on Tensorboard Learning Rate
Value Loss
Episode Length
Cumlative Reward
Mechanical Rockery
Simulation of the Robotic Arm in Unity3D
In Unity3D, a six-axis robotic arm is used to simulate the rockery building process, which can help manipulate the robotic arm in reality. The epistemic action with physical models encourage the unprofessional to explore the aura of space and engage in design more deeply. Prototyping and Circuit
[ Kuka R900 ]
[ Gripper ] 3D-printed Grip objects with kuka
Robot Robot to grip objects
[ Kuka RC4 ]
[ Connection ] Connection Adapting piece for gripper
Controller Control Kuka and gripper
Kuka working radius
1
22'65
3
Building Process
4
2
1 Camera 2 Worktable
Interface Input 2
3 4
Kuka R900 Objects needed to load
Mechanical Rockery
Mechanical Rockery in Forbidden City
The mechanical rockery manifests a brand new way to understand the space forms of traditional rockery with reinforcement learning and the robotic arm.
Automatic City Team member: Siming Chen Jieyu Zhou Instructor: Pro. Mark Anderson
This design aims at creating ecosystems that are benchmarks how cities should be in future. Going vertically is a strategy to solve the problem of landuse shortage. There will not be any cars or roads in the city. Instead, with the technology of automatic driving, each housing unit will have an elevator as its owner's car to transport them to everywhere in the city. Elevator shafts make it convinient for "cars" to transport people from their houses to the working place. To push the boundaries of design, this project also used innovative habitat working models, materials, technology, close to zero land costs, a nomadic yet rooted lifestyle.
Automatic City
Self-driving housing units as transportation
This design aims at creating ecosystems that are benchmarks how cities should be in future. With a faster transit and a more connected world the need to stay rooted at one location will go away and futures will be more transit/ mobile.Faster transportation techniques and connectivity + collaboration, has made it possible to look beyond boundaries of cities Live
Plants
Water
Resistance
City Growing Strategy
Expension
Sahara
2.5 mile
Skyscraper
This design aims at creating ecosystems that are benchmarks how cities should be in future one location will go away and futures will be more transite in future one location will go away and
Prototype Habitat
Terrain
Energy
Sandstorm
Rainfall
Transit
For 1,000 people 0.5*0.5mi
Road
Layer for Housing Units 1.Self Driving Home
2.Problems
12 million
Americans are blind or have low vision
3.Computer Vision
4.Construction Tech
Highway
79%
of seniors in car dependent communities
0 100 m
54 hours
wasted in traffic year per person
2.5 mile
500 m
1 km
Automatic City
Self-driving housing units as transportation
Facade
A u t o m a t i c Fa ç a d e , c o o l i n g system and building strategy applying bacteria are introduced to this project.
The structure and module of working space will remain the same with those of housing units. Elevator shafts make it convinient for "cars" to transport people from their houses to the working place. Elevators
Grow human civilization in synchronus with nature + technology + planet
Diagrams of Units
Sahara Skyscrapers
Housing or working Modules
Transportation Module
This design exercise can be considered similar to colonizing a new earth with technology of today.
Linear combination modes
Working sapce
The structure and module of working space will remain the same with the housing units
Ring combination modes
Linear combination modes
Diagrams of Single Skyscrapers
Transit
Form 1
Form 3
Form 2
going vertically is a strategy to solve the problem of land-use shortage. Automatic-driven elevator.
Diagrams of Skyscrapers and Joints
Housing unit
There will not be any cars or roads in the city. Instead, with the technology of automatic driving, each housing units
Transportation Module
Single Building
Connection
Greenspace
It is a space in all skyscrapers that community members can enjoy their time
Automatic City
Self-driving housing units as transportation
There will not be any cars or roads in the city. Instead, with the technology of automatic driving, each housing unit will have an elevator as its owner's car to transport them to everywhere in the city. The structure and module of working space will remain the same with those of housing units. Elevator shafts make it convinient for "cars" to transport people from their houses to the working place.
Module units
Housing Units
Elevator
Working and playing
Cooling System
Automatic Façade
Building strategy
Plans for Working modules and housing units
0
Layer for Housing Units
Layer for Working
1
5
10 m
Automatic City
Programs
Self-driving housing units as transportation
Center
Center
The structure and module of working space will remain the same with those of housing units. Elevator shafts make it convinient for "cars" to transport people from their houses to the working place.
Work Housing Leisure Transit
Transit Housing Work Greenland Leisure
Rendering for the transit
Transit
Transit
The Smart Wall Team member: Jieyu Zhou Baoliang Yu Instructor: Pro. Li Li
Making authentic models and creating space on their own offers a chance for people in non-design field to experience the aura, aesthetic, and mystery of architecture. In other words, it is a heuristic way to understand space, form and style and participate deeply in design process. In my project, the smart wall can change its color according to users’ facial expressions and twist based on users’ body movement. In this way, people less-informed of design knowledge engage in design process more actively and have the opportunity to experience the aura of space through lighting and motion of this wall.
The Smart Wall
How do we perceive space?
In the common sense of people, walls are always still with no emotion. But can we bring a new definition to the wall? That is where the idea of our project began. We want to make the wall talk to people through waving and shining.
1 Design Concept Diagram
2 The Evolution of Space
① ② ③
Classical Modernism Cyberspace
①
②
Different Perception of Space
How do emotion influence we perceive space? Emotional Recognition
The same space but different results
It is a just normal place for me.
Collective memorries and difference
This classroom recalls me the time I spent with friends. I feel nostalgic for the old days.
I feel depressed today. All I see is blue.
A space that can act according to your emotion Different perception system
③
The Smart Wall
Workflow Diagram
Algorithms and Framework
[ Motor End-Effector ]
[ Lighting End-Effector ] 8
9
... ...
6
7
... ...
Prototyping and Circuit
[ LED Strip ]
[ Fiber Optics ]
[ Arduino ]
[ Raspberry Pi ]
Motion Controll motion with grids
Lighting Show colors with fiber optics
Lighting and Motion Colors and enlarge motion
Controller Controll motors and LEDs
Controller Emotion Recognization
8
5 +
[ Servo Motor ]
... ... +
... ...
189
+ -
... ...
[ Digital Control ]
6
4 Power [220V]
+ -
Central Controller 4.Arduino Mega 2560
4
Motion 5.Adafruit 16-Channel 12Bit PWM Servo Driver 6.Mg996 Servo Motor 7.Flexible Gribs
5 2 1
Arduino IDE Fast LED Library Adafruit PWM Servo Driver Library
Lighting 8.WS2812B LED Strips 9.Fiber Optics
3
Serial Communication
[ Raspberry Pi ] 1
2
1 Power 2 Mg996 Servo Motor 3 Adafruit PWM Servo Driver
4 WS2812B LED Strips 5 Arduino Mega 2560 6 Raspberry Pi 4
Input 1.Fan 2.Camera Module Central Controller
3
Emotion Recognition [OpenCV,Dlib] Object Tracking [Optical Flow] Linux
Power [220V]
+ -
The Smart Wall How do we perceive space? Optical flow is the pattern of apparent motion of objects, in a visual scene caused by the relative motion. It is used to detect motion in this project. With the use of OpenCV and Dlib, 68 points we re dete cte d as fa c i a l landmark annotation. Using these points, facial features were analyzed to recognize different emotions.These points, facial features were analyzed different emotions and sent them to Arduino.
The smart wall can change its color according to users’ facial expressions and twist based on users’ body movement. In this way, people less-informed of design knowledge engage in design process more actively and have the opportunity to experience the aura of space through lighting and motion of this wall.
Motion Mode: Optical Flow
Emotion Mode: Emotion Recognition 1
Annotate 68 facial landmarks with the Dlib trained model
Open the camera, Compress image, and grayscale
2
OpenCV Dlib, facial landmark annotation
3
Open the camera, Compress image, and grayscale
4
Annotate 68 facial landmarks with the Dlib trained model
5
Calculate degree of raised eyebrows and frowning, eye opening distance and mouth opening distance
5
Emotion Recognition
6
Recognize the emotion, and transmit the code to Arduino
7
Skin 1. Fiber Optics 2. 3D Printed Grids Structure 3. Wooden Planks 4. Frame with motors 5. Central Controller 6.Arduino Mega 2560 7.Raspberry Pi 4
1
2
3
4
The Smart Wall
① Motion Mode
How do we perceive space?
If there is any moving object detected, the wall will change its color according to the kinematic velocity and shrink based on the object's location. Camera
3
speed ++++ Very Fast The color will be brighter and warmer.
2
speed ++ Fast The color will be less birght and warm.
1 speed + Slow The color will be darker and cooler.
Motion Mode Change color and shrink by the velocity
Chronological Sequence
No moving object, showed preset color Palette
Moving object tracked
More Active
The Smart Wall
② Emotion Mode If there is no moving object but faces are detected, the wall will change its color according to emtion.
How do we perceive space?
Happy
Natural
Facial expression
Facial expression
LED pattern: Purple
LED pattern: Yellow
Natural
Happy
[Facial expression: Happy]
[Facial expression: Natural]
Angry
Amazing
Facial expression
Facial expression
LED pattern: Pink
LED pattern: Blue
Amazing
Angry
[Facial expression: Angry]
[Facial expression: Amazing]
Short-Term Projects City Context Team member: Baoliang Yu Jieyu Zhou Instructor: Pro. Li Li
In the study of commercial environment, I used one of the machine learning methods, Self-organizing Maps to classify all shopping malls’ texture in China, aimed at interpreting sophisticated city contexts to the grassroots and encouraging them to engage in the city regeneration process.
Short-Term Projects Manhattan Workers Team member: Tong Ji Jieyu Zhou Instructor: Pro. Karen Chapple
This research focuses on housing and transportation data in New York, making it easier for the public to understand the complex housing market and transportation situation. With computational analysis tools, Matplotlib, and geographic visualization tools, Carto, complex raw data turned into interpretable, actionable information in the process of participatory design.
