Augmenting Creativity

Augmenting Creativity A methodology for designers and artists to identify their roles and survive in the new age of AI

Master in Design for Emergent Futures Program, Institute for Advanced Architecture of Catalonia and ELISAVA School of Design & Engineering.

June 2020 Author: Ching-Chia Renn

Augmenting Creativity A methodology for designers and artists to identify their roles and survive in the new age of AI

Project Credits Augmenting Creativity is a project of IAAC, Institute for Advanced Architecture of Catalonia developed in the Master in Design for Emergent Future 2019/20

by Students: Ching-Chia Renn and Faculty: Tomas Diez, Oscar Tomico, Mariana Quintero

Abstract Artificial intelligence makes machines to be smarter, act more natural, and become more adaptive to different people. With the development of AI, it also became more "creative" recently, and it would totally change the relationship between humans and machines.

Augmenting Creativity is a project about exploring the opportunities of collaborating with Artificial Intelligence in a creative process and expression. From discussing and reflecting how technology would impact creators think and act to speculate the possible future scenarios of new relationships between human creators with machines. Fining the intersection between AI and the creative field, inviting artists and designers to create with the creative machine and identify their roles in the age of creative AI. The project not only makes creators know about the possibility of applying AI in creative practice, but also provides them actually to learn machine learning tools and have the ability to use them. Aiming to broaden the imagination and augment creativity.

This thesis Augmenting Creativity in the context of the Master in Design for Emergent Program. It includes the main motivations of the project, documentation of design actions, and a series of speculative design exercises to project the project in the future.

Acknowledgements I would like to first thank the Master in Design for Emergent Future studio tutors Tomas Diez, Oscar Tomico and Mariana Quintero for their surpport, guidance and providing references, ideas, and feedback through the whole master year. Also thank to Oliver Juggins, Xavier DomĂ­nguez, Ramon SangĂźesa, Guillem Camprodon, Saul Baesa for giving me support, thoughts, and references that let me keep researching, reflecting, and exploring the relevant field about artificial intelligence and creativity.

Thank to all of the open source projects and platforms that I have been able to use to reflect and develop the project. mBlock, RunwayML, google Magenta and the Wekinator are just some of these.

Finally, I would like to thank all the friends, teachers, and students who have participated in the project. All the small ideas, thoughts, and experience sharing are valuable to me. With the conversations and discussions with you that the project can be formed.

Content

1. Area of Interest and Weak Signals

14 14 16 16 18 19 20

2. Area of Intervention and Description of Interventions

25 25 26

3. State of the Art

30 30 34 36

1.1 Motivation and Problem Awareness 1.2 Area of Interest 1.2.1 AI creativity 1.2.2 AI tools for art and design works 1.2.3 AI literacy for creator 1.3 Atlas Weak Signals

2.1 Research Questions 2.2 Area of Intervention and Description

3.1 AI tools for art and design works 3.2 AI-based art and design research 3.3 AI literacy for creators

4. Design Intervention

42 4.1 First intervention 42 4.1.1 Results 44 4.1.2 Reflection 52 4.2 Second intervention 54 4.2.1 Results 55 4.2.2 Reflection 74 4.3 Third intervention 76 4.3.1 Results 76 4.3.2 Reflection 83

5. Future Scenario

5.1 The future scenario of AI in the creative process

86 86

6. Design in emerging contexts: COVID-19 6.1 My new me 6.2 Mapping my domestic experimental laboratory

92 92 94

7. Hyper-local / Hyper-Global

98 7.1 Experimenting with my project in emergent contexts 98 7.1.1 Intervention 98 7.1.2 Results 99 7.1.3 Reflection 102 7.2 Collaborating with others for my project 104 7.2.1 Intervention 104 7.2.2 Results 106 7.2.3 Reflection 112

8. Project and personal identity

122 122 124

9. Project the interventions into the future

9.1 Emerging narratives 9.2 Looking forward 9.2.1 Reflection on the scalability of the project to other contexts 9.2.2 Development plan for after the masters programme

132 132 134 134 136

10. Final Reflection

140

References

146

8.1 Creating my own biography and constituency 8.2 Reflecting about new weak signals in my future scenario

Chapter 1 Area of Interest and Weak Signals

Chapter 1

Generated by RunwayML styleGAN

1. Area of Interest and Weak Signals 1.1

Motivation and Problem Awareness

Artificial Intelligent (AI) techniques had already applied to many places in our digital lives and they will only become more powerful in the future. However, we are always using the technologies in passive ways and even cannot aware of them. The AI just like a black box in the background of system and somehow controlled our mind that we just follow the decisions they gave us blindly. We have already gradually been controlled by them.

14

Chapter 1

As technology advances, the technology and machine should not just make decisions for us automatically, but should augment and enhance us for having more interactions and communications between the digital and physical world. In addition, technology and machine should become something enhancing the human ability of creativity instead of the things which limit our imagination and creativity. I believe that we can use technologies and machines in more initiative ways and be equal for everyone to explore their creativity. Disrupting the interface, blurring the boundary of digital and physical worlds and create a new relationship with technology and machine.

The development of technology will never stop and the progress will only go faster and faster.

In this rapidly changing world, it’s hard that the people in creative sectors are not influenced by new technologies. When a camera was invented, there was a new way to express the creations which appeared; when computers showed up, there were a lot of different software for creators to choose; when robots and AI came, they could also be used as the creative tools.

I believe that technology would change the perceptions of people. When a creator learned a new technology, it would definitely change his/her thoughts and to have a unique creation in some way. In addition, because of using different technology in the creations, there will be different actions, solutions, and ideas coming out in both processes and expressions. The impacts of technology would be both positive and negative. How can we shape technology to amplify the ability rather than be shaped by them, it should be always a question for us to think about. The question is not just about how technology was designed, but also how do creators use technology and create the values of tools.

The project is an opportunity for me to explore the possibilities of artificial intelligence (AI) and machine learning (ML) for enhancing creative expression and the new ways of using AI/ ML as an inspiring tool in creative processes. Changing the human-machine relationship to be a more collaborative way, shifting the role of machines from automation to augmenting human creativity, and trying to build a new way of creating and expressing creations. Making AI to be a familiar tool for creators to apply in their creative projects and will never be a black box for us.

15

Chapter 1

1.2 Area of Interest 1.2.1

AI creativity

Creativity is a fundamental feature of human intelligence, and a challenge for AI. Creativity is very difficult, maybe even impossible, to define in objective terms. “What is creativity?” is always a question that we can discuss about. It is thus interesting to see how AI can assist in creative domains such as design and art. In the Design Intervention (Chapter 4), there is a survey I did in the second intervention which is called the research about technology and creativity. In the survey can be seen how diverse that people think about creativity.

"Creativity is the ability to come up with ideas or artifacts that are new, surprising, and valuable."

- Margaret Boden

Can AI learn to be creative? this is a question which is hard to answer, but is also interesting to think about. A lot of people feel that ‘creativity’ is an attribute that is solely in the purview of humanity and computers are usually assumed to have nothing to do with creativity.

Ada Lovelace(1815–1852), who was an English mathematician for writing an algorithm for a computing machine in the mid-1800s, is often quoted in this regard: “The Analytical Engine has no pretensions whatever to originate anything. It can do [only] whatever we know how to order it to perform.” If this is taken to mean that a computer can do only what its program enables it to do, it is of course correct. But it does not follow that there can be no interesting relations between creativity and computers.

There are four different questions, which are proposed by Margaret A. Boden in her book The Creative Mind: Myths and Mechanisms, called Lovelace questions, and state them as follows: 1. Can computational concepts help us to understand human creativity?

2. Could a computer, now or in the future, appear to be creative?

3. Could a computer, now or in the future, appear to recognize creativity?

4. Could a computer, however impressive its performance, really be creative?

16

Chapter 1

Figure 1.1: Copy of one of Babbage’s general plans of the Analytical Engine, in Bromley (1982)

There is a field called computational creativity (also known as artificial creativity, mechanical creativity, creative computing or creative computation) which is the study of building software that exhibits behavior that would be deemed creative in humans. Such creative software can be used for autonomous creative tasks, such as inventing mathematical theories, writing poems, painting pictures, and composing music. However, computational creativity studies also enable us to understand human creativity and to produce programs for creative people to use, where the software acts as a creative collaborator rather than a mere tool.

The project would like to discuss about "What is creativity?" and what is computational creativity means for human creators, and how can creators take this artificial creativity into their works to augment the creativity. Through the project, I took first person perspective to experience myself as an artist and designer in this context to use and test differnt Machine Learning tools to develop the possible creations and reflect on the processes. In addition, the project also invites creators to think and experience AI creativity in their creative domain and not only starting aware of the existence of 'creative AI' and being interested in it but also having the ability to apply to the creative process.

17

Chapter 1

"Others have seen what is and asked why.

I have seen what could be and asked why not."

- Pablo Picasso

Generated by the Processing code

1.2.2 AI tools for art and design works Machine Learnng (ML) is a important sector in AI research, which is a subset of Artifitcial Intelligence. Machine learning enables systems to learn and improve from experience without being explicitly programmed. It focuses on developing computer programs that can access data and use it to learn for themselves. Moreover, there are more and more ML and generative algorithms were developed recently for creativity and captured people’s interest which can autonomously create new works of art, music, video, or text. But actually, ML has been used for many years in a wide range of human creative practices. In State of the Art section (Chapter 3) will give a overview and brief introductions about the the existed Machine Learning tools which aims to apply in creative practices.

Creation is made by tools and the tools are operated by a human creator. It’s hard to create without a tool. Thus it can be seen that tools are important for artists and designers to create their creations. However, what kinds of tools they use and how they use it are the questions.

Also, how can AI become a tool even a collaborator for creators and get involved in creative practice?

18

Chapter 1

Inventing and developing a new tool is always not an easy job to do, however, the thing that creators can do is to create the values of the tools and techniques. The project is not pushing creators to be an AI developer, but trying to give them the ideas and basic knowledge of AI and ML to open their curiosity and broaden their imagination. Through learning, using, and experiencing different AI/ML tools in creative pactice to let them know that AI is not only a technique that can give us the recommendation of videos and recognize our faces, but also become a tool for creativity.

1.2.3

AI literacy for creator

There are growing as a result of the “STEM to STEAM” movement, which advocates for the inclusion of the arts and design within the STEM (science, technology, engineering, and mathematics) disciplines that are taught computing skills by using creative applications of technology.

With the idea of the STEAM movement, there are programs combining programming and other computer-science skills (including AI/Machine Learning) with creative instruction is also called “creative computing” and “creative coding”. Such programs typically target students to use computers to make new creative work. However, there is a challege that creative computing programs still focus on a lot of programming, and it requires students a number of the same skills as computer-science students. But whether there is a better way to learn and teach computational skills within creative computing courses.

The project began with an exploration of the uses of Machine Learing in creative work and then was going to develop in the creative computing education and Machine Learning education. Trying to finding a way to adapt computer-science topic especially AI and ML within a creative computing context. Providing appropriate Machine Learning education and tools to creators presents many potential benefits said by Rebecca Fiebrink in her publication Machine Learning Education for Artists, Musicians, and Other Creative Practitioners: by enabling more people to use ML more effectively, ML education can lead to new creative outputs and means for self-expression, and also to economic impact as creative entrepreneurs discover opportunities for using ML in creative technologies. Supporting creators, including artists and designers, to build and create with AI in their creative works is also a way to identify their role as a human creator in the creative process.

19

Chapter 1

1.3 Atlas Weak Signals The Atlas Weak Signals seminar (Joes Luis de Vicente, Mariana Quintero, 2020) was a series of sessions which was part of the second semester of the Master in Design for Emergent Futures programme. The aim of the course was to undertand the present and develop the future scenarios, and identify the larger context and themes that the projects engaged and related to, providing a way for speculating on the areas of intervention.

Every class we had a weak signal topic to discuss, and tried to develop the signal into a future scenario for a possible proposal based on the topic.

Weak signals point out the directions of future scenarios. Through detecting in the present trends and analysing the main change factors to position the subject and set the topic related the future matters.

There were 25 defined topics and themes which formed the basis of searching for “weak signals”. The research of weak signals pointed out the main issues for the project, besides, with the discussions in the seminar helped to set the direction and build the future scenario.

Weak signals related to the project: • Human-machine creative collaboration • Technology for equality The main topics that link to the project include human-machine creative collaboration and technology for equality. The most relevant a weak signal to my project is human-machine creative collaboration, as there are more possibilities that appear recently that machines can collaborate with a human in many sectors and with the progress of Artificial Intelligence, machine also become more smart and creative. There are more ans more connections between machine and creativity, thus should be aware and reflected.

20

Chapter 1

Tangible programming kit Learn the world in different perceptions / To have different perspective of experience

Universal construction kit inspiring machine

play / have fun connect to people

Learn creativity (Tech can help)

Automation

disrupt ageism (inter-generational)

Learn empathy

Augmentation Technology for equality

Human-machine creative collaboration

Age of AI

disrupt discrimination

human behavior adapting

Atlas Weak Signals Fighting AI bias

Figure 1.3: Situating the ideas of the project in the atlas with weak signals

21

Chapter 2 Area of Intervention and Description of Intervention

- A person from architecture sector

24

Style generated by GoArt fotor (https://goart.fotor.com/)

"Creativity is thinking out of the box."

Photoed by author

Chapter 2

Chapter 2

2. Area of Intervention and Description of Interventions 2.1 Research Questions The key research questions of the project addresses are: • How can creators use AI as a tool for inspiration?

• How do creators change their perceptions when they use the AI as a creative tool?

• How can AI enhance opportunities for creative expression?

• How can AI become a collaborator with artists in a creative project?

• What is the essence of art and creativity?

• How can AI intervene in the creative process and create a new possible way of making creations?

• How can designers collaborate with AI in a design process? How to communicate with AI? What interface will be?

• How can designers and artists redefine the value of their roles in the creative sectors when they collaborate with AI? • What if a design process changes from finding solutions to identifying problems and rule-making?

These questions are what I’m considering and they drive me to keep thinking and researching about the relevant topics or issues in my area of interest.

25

Chapter 2

2.2 Area of Intervention and Description The development of technology will never stop and the progress will only go faster and faster.

In this rapidly changing world, it’s hard that the people in creative sectors are not influenced by new technologies. When a camera was invented, there was a new way to express the creations which appeared; when computers showed up, there were a lot of different software for creators to choose; when robots and AI came, they could also be used as the creative tools.

I believe that technology would change the perceptions of people. When creators learn new technology, it will definitely help their creations in some way. In addition, because of using different technology in the creations, there will be different actions, solutions, and ideas coming out not only in the processes but also in the expressions.

The impacts of technology would be both positive and negative. How can we shape technology to amplify our ability rather than be shaped by them, it should be always a question for us to think about. It’s not just how creators use technology, also how technology was designed.

The area of intervention of the project is around the issues about emergent technology especially Artificial Intelligence and its applications in the creative field, which are also come from my area of interest. In order to respond to my area of interest and the research questions, I started the design intervention and tried to find questions and settle them. Design intervention is a real action for me to explore, research, and require the issues from my area of interest, it helps me to clarify my thoughts, question them and think repeatedly the topics that I am interested in and care about.

The three interventions were taking different perspectives to approach to the things about creativity, technology, and AI, and also trying to speculate the future scenarios of applying AI in the creative process. All the interventions are the means of inquiry for myself to clarify my thoughts and also shape the project in the process. Through the intervention, trying to find the essence of the project.

26

Chapter 2

Tangible programming kit Universal construction kit

Learn the world in different perceptions / To have different perspective of experience

inspiring machine

play / have fun connect to people

Learn creativity (Tech can help)

Automation Augmentation

disrupt ageism (inter-generational)

Technology for equality

Learn empathy

Human-machine creative collaboration applying AI/ML in creative field

Age of AI

disrupt discrimination Fighting AI bias

human behavior adapting

RunwayML Magenta

State of the Art

Atlas Weak Signals

Navidia GauGAN

Augmenting Creativity

Autodesk Generative Design

Teachable Machine performance workshop

Activity

middle school students

People

Wekinator

Creative Professionals

Visual impairments

Oliver (ex-MDEF student who was working Cooperator on the AI field) Santi (learning future unit) Learning Future Unit (Fab Lab Bcn) Lucas / Ramon (AI expert)

Intervention

Tools /

Microbit Materials Arduino mBlock

Figure 2.1: Mapping the elements of intervention with weak signal and ideas

27

Chapter 3 State of the Art

Chapter 3

3. State of the Art In the state of the art, there is a collection of a number of projects which are taking AI and machine learning into the creative field, and pushing the boundaries of creativity and investigate the implications of AI on society, art and culture, including the software, platforms, research, etc.

Augmenting Creativity is a project about researching AI tools and applying them into the creative field, and taking creators on a guided tour to disrupt the status quo of the traditional art world, and explore our complicated relationship with machines.

3.1 AI tools for art and design works RunwayML

July 2019 | By RunwayML RunwayML is a platform for creators of all kinds to use machine learning tools in intuitive ways without any coding experience. It is a powerful and easy-to-use application that makes machine learning more accessible and inclusive to creators. It lets anyone with a computer start to explore and create using the latest AI and machine learning models. (https://runwayml.com/)

Figure 3.1: Runway ML software workspace

30

Chapter 3

ml5.js

July 2018 | By NYU’s ITP/IMA program ml5.js is an open source project developed and maintained by NYU’s Interactive Telecommunications/Interactive Media Arts program and by artists, designers, students, technologists, and developers from all over the world. It is machine learning library for the web in the web browser. ml5 is not just a project about developing machine learning software, it is about making machine learning approachable for a broad audience of artists, creative coders, and students. The library provides access to machine learning algorithms and models in the browser, building on top of TensorFlow.js with no other external dependencies. (https://ml5js.org/)

Wekinator

2009 | By Rebecca Fiebrink The Wekinator is free, open source software originally created in 2009 by Rebecca Fiebrink.

It allows anyone to use machine learning to build new musical instruments, gestural game controllers, computer vision or computer listening systems, and more. It allows users to build new interactive systems by demonstrating human actions and computer responses, instead of writing programming code. (http://www.wekinator.org/)

Figure 3.2: Using the Wekinator and Processing to interact with the graph

31

Chapter 3

AI Generated Images / Pictures • Deep Dream Generator – Stylize your images using enhanced versions of Google Deep Dream with the Deep Dream Generator. (https://deepdreamgenerator.com/) • DeepArt.io – Upload a photo and apply different art styles with this AI image generator, or turn a picture into an AI portrait of yourself. (https://deepart.io/) • GoArt – Create AI photo effects that make your photos look like famous portrait paintings with this AI image generator (Web, Android and iOS. Made by Fotor). (https://goart.fotor. com/) • Deep Angel – Automatically remove objects or people from images (Web. Made at MIT). (http://deepangel.media.mit.edu/) • ArtBreeder – Merge images together to create new pictures, make hybrid AI portrals and create wild new forms that have never been seen before. (https://www.artbreeder.com/) • AI Painter – Turn your photos into AI paintings or create abstract art with this neural network painting generator. (https://www.instapainting.com/ai-painter)

Figure 3.3: Stylizing the photo by Deep Dream Generator

32

Chapter 3

Photoed by author Stylized the photo by Deep Dream Generator

33

Chapter 3

3.2 AI-based art and design research Nvidia GauGAN

2019 | By Nvida GauGAN creates photorealistic images from segmentation maps, which are labeled sketches that depict the layout of a scene. Artists can use paintbrush and paint bucket tools to design their own landscapes with labels like river, rock and cloud. A style transfer algorithm allows creators to apply filters — changing a daytime scene to sunset, or a photorealistic image to a painting. Users can even upload their own filters to layer onto their masterpieces, or upload custom segmentation maps and landscape images as a foundation for their artwork. (https://www.nvidia.com/en-us/research/ai-playground/)

Magenta

2015 | By Google AI Magenta is an open source research project exploring the role of machine learning as a tool in the creative process. It is distributed as an open source Python library and JavaScript API for using the pre-trained Magenta models in the browser, powered by TensorFlow. (https://magenta.tensorflow.org/) There are some relevant projects from Magenta: • NSynth Super – NSynth Super is an experimental physical interface for the NSynth model which is one of Magenta model. It uses deep neural networks to generate sounds at the level of individual samples. (https://nsynthsuper.withgoogle.com/) • AI Duet – Play with a piano that responds to you. (https://experiments.withgoogle.com/ai/ ai-duet/view/) • NSynth Sound Maker – Create your own hybrid sounds and instruments. (https://experiments.withgoogle.com/ai/sound-maker/view/)

34

Chapter 3

Project Dreamcatcher

2016 | By Autodesk Generative design is a design exploration process. Dreamcatcher is a generative design system that enables designers to craft a definition of their design problem through goals and constraints. This information is used to synthesize alternative design solutions that meet the objectives. Designers are able to explore trade-offs between many alternative approaches and select design solutions for manufacture. The system allows designers to input specific design objectives, including functional requirements, material type, manufacturing method, performance criteria, and cost restrictions[5]. The software explores all the possible permutations of a solution, quickly generating design alternatives. It tests and learns from each iteration what works and what doesn’t. (https://autodeskresearch.com/projects/dreamcatcher)

Figure 3.4: Autodesk research project Dreamcatcher possible chair designs

35

Chapter 3

3.3 AI literacy for creators AIArtists.org

AIArtists.org is acommunity of artists exploring the impact of AI on art, culture and society.

In their website showcases pioneering artists who are using Artificial Intelligence to push the boundaries of creativity and investigate the implications of AI on society, art and culture. (https://aiartists.org/)

AI meets Design

AI meets Design is a website which is building a bridge between the disciplines of AI & design and exploring how to design human-centered AI applications. It also provides design toolkit which is a set of tools for each step of the design (thinking) process to help designers turn AI into social, user, and business value. It’s an invitation to designers and innovators everywhere to design with and for machine intelligence to create human-centered applications and meaningful user experiences. (http://aimeets.design/)

Machine Learning for Musicians and Artists - Kadenze

July 2019 | By Rebecca Fiebrink This is an online course which was taught by Rebecca Fiebrink in Kadenze. It is a creative machine learning course for artisits and musicions to learn fundamental machine learning techniques that can be used to make sense of human gesture, musical audio, and other real-time data. The focus will be on learning about algorithms, software tools, and best practices that can be immediately employed in creating new real-time systems in the arts. (https://www.kadenze.com/courses/machine-learning-for-musicians-and-artists/info)

IDEO’s AI Ethics Cards

July 2019 | By IDEO IDEO’s AI Ethics Cards are a tool to help guide an ethically responsible, culturally considerate, and humanistic approach to designing with data. The deck is made up of four core design principles and ten activities, all meant for use by teams working on the development of new, data-driven, smart products and services. (https://www.ideo.com/post/ai-ethics-collaborative-activities-for-designers)

36

Chapter 3

Figure 3.5: The AI meet Design webpage of showing the AI design toolkits

Figure 3.6: IDEO’s AI ethics card

37

Chapter 3

Experiments with Google (AI Experiments)

Experiments with Google is a platform made by Google Creative Lab since 2009 with a collection of experiments communicating ideas behind different technologies. The website includes a section on "AI Experiments” which is a showcase for simple experiments that make it easier for anyone to start exploring machine learning, through pictures, drawings, language, music, and more. (https://experiments.withgoogle.com/collection/ai) There are some relevant projects in AI creativity: • Teachable Machine – It is a web-based tool that makes creating machine learning models fast, easy, and accessible to everyone. It was developed by Google to make Machine Learning and AI accessible to the wider public, without requiring any specialized training, knowledge in computer science or coding. (https://teachablemachine.withgoogle.com/) • Sketch-RNN Demos – Draw together with a neural network. (https://magenta.tensorflow. org/sketch-rnn-demo) • AutoDraw – Turn your sketch into clip art with this computer generated AI drawing tool. (https://www.autodraw.com/) • Quick, Draw! – A game where a neural net tries to guess what you’re drawing. Draw along with AI and neural networks with this Google draw app. (https://quickdraw.withgoogle.com/) • Cartoonify – Turn your portrait into a computer generated cartoon drawing. (https://www. kapwing.com/cartoonify)

38

Chapter 3

Figure 3.7: Sketch-rnn predicting possible endings of various incomplete sketches

39

Chapter 4 Design Intervention

Chapter 4

4. Design Intervention 4.1 First intervention How can Machine Learning be used and applied in everyone’s daily life as a more creative and equal user interface?

How can Machine Learning help us to set our own rules to control almost everything?

In the first intervention, I focused in exploring and experiencing the possibilities of using and applying Machine Learning to a user interface in more creative ways. It was a kick-starting activity for me to test and feel the Machine Learning technique in the real world.

I used the Teachable Machine which is a supervised machine learning by using image recognition technique operating interface based on mBlock software. The software helped me to make prototype rapidly for testing the different ways of interacting with the interfaces. Using the first person perspective in this intervention and trying to find the differences by comparing with the operations of traditional flat user interface.

Objectives

• To have different perspective of experience with machine learning (ML) • To explore the possibilities of applying ML in an user interface

Methods / Activities

• Exploring the machine learning in different interface-using situations • Discussing with IAAC students, tutors and others who know more about AI / HCI / UI / UX • Having online courses and tutorials about machine learning and the programming software

Tools / Materials / Techs

• mBlock (Scratch 3.0 based) • Teachable Machine (mBlock based)

42

Chapter 4

I designed a series of diverse situations that we normally can achieve by using our phones.

There are different situations of operating a user interface with image recognition technique which is one of Machine Learning approach. I used the Teachable Machine (Figure 4.2) and mBlock (Figure 4.1) software to make prototype rapidly for testing the different ways of interacting with the interfaces.

Figure 4.1: mBlock software interface

Figure 4.2: Teachable Machine extension in the mBlock interface

43

Chapter 4

4.1.1

Results

There are the situations I tested in the mBlock down below were divided by the things are detected by the webcam with the ML algorithm, which are “Motion” and “Object”.

Motion (body/hand sign) • • • • • •

On / off Direction Sweeping page Adjusting (size / color) Body music / sound To know the temperature

Object

• Operating the level (of light) • Music tape • Unfolding music

Motion - On / off

Figure 4.3: The demo of using gestures to turn on and off the light

There are two gestures to turn on and turn off the light. I think a switch is the most basic thing in an interface and every electronic device. So the test of on / off was my first test in this series of experiments.

44

Chapter 4

Motion – Direction

Figure 4.4: The demo of using gestures to control the charactor

Using a hand to show the directions to control the character’s movement. This test was about switching the typical controlling system of a game to control the character’s movement.

Motion - Sweeping page

Figure 4.5: The demo of sweeping out and sweep back the black page in the screen

45

Chapter 4

Sweeping a hand in air to sweep out and sweep back the black page in the screen. As a screen-based electronic device, it’s a very common operation that we sweep out another page in the original page.

Motion - Adjusting (size / color)

Figure 4.6: The demo of changing the color by waving hand in one direction

Figure 4.7: Opening a hand to get bigger the square; making a hand as a roll to get smaller the square

If there were multiple choices displaying on the same page, it would be hard to use a gesture to select a specific choice. To achieve multiple choices, every choice should become a yes / no question.

From this test, I found a problem with the webcam detecting as an operating input that I need to think about it which is how to ensure the accuracy of detection. Actually it’s easy to have an accidental motion to change something that you don’t want.

46

Chapter 4

Motion - Body music / sound

Figure 4.8: The demo of using gestures to switch images and sounds

There are switching to different images and sounds by different hand gestures (/motions). In this one, I tried to make some sound base on possibly relevant hand motion. From this test, I imagine that music can be generated by motions. Maybe in the future, there will have different ways of composing which is we can dance to a music not just dance follow the music.

Motion - To know the temperature

Figure 4.9: The demo of using motions to get highest and lowest temperature

47

Chapter 4

Setting the motions to let machine know what temperature information I need.

Imagining that we can get different information by setting and using different motions. However, the problem here is that there is so much information that how we can set every information by motion.

Object - Operating the level (of light)

Figure 4.10: Using he physical objects to interact with the machine

48

Chapter 4

Figure 4.11: The light levels are represented by numbers of brick

In this test, I started feeling that the objects in the physical world can really control the digital objects. I can use the physical objects to interact with the machine that changing the numbers of brick would have levels changing respond in the display.

49

Chapter 4

Object - Music tape

Figure 4.12: The machine recognition window and tapes

With the Machine Learning, we can generate music by writing or drawing. These tapes somehow transform the notes into letters or even symbols that allow everyone can play the music with the computer (machine/device).

50

Chapter 4

Object - Unfolding music

Figure 4.13: The machine recognition window

There is a piece of paper which have different colors patterns in the both sides. When the paper was folded, there were looking like another pattern. Each pattern input for the machine recognized and learned, and each pattern was represented in different sound. Therefore, from the folded status of paper to the paper was unfolded, every folding or unfolding step would change the pattern and the sound was changed along with as well by recognizing the different patterns.

51

Chapter 4

4.1.2

Reflection

The first intervention was a warm-up activity for me. Using first person perspective and exploring user interface with image recognition (ML) to prototype the real interfaces and experience the scenarios of operation in direct feeling. Even though these just some small pieces of test, I also learned, thought, and experienced something from the intervention and had more imaginations for the ways of operation by using image recognition. I felt that my ideas were started moving forward and started having more ideas.

From the results I prototyped, it can be found that the applications of image recognition (ML) can not only recognize the images, but also objects and motions. The biggest different from the traditional ways of operation (or the ways of using things) I think is that there are more variations and it’s more flexible and more intuitive. The machine can from detecting your motions and the stuff you use to know your behaviors and then give you the reactions which are fit to you. You don’t need to learn machines, but machines learn from you. We will never need to face lots of trouble buttons, whether they are tangible or virtual, and the troublesome steps of operation process. Besides, with the image technique, I feel that the physical and digital worlds are closer than before, there are no obvious gap between two of them. However, there are still have some problems with the image recognition technique in applications.

It’s hard to guarantee the accuracy of results of image recognition because it’s easy to have a misjudgement from the recognition if the image, object, or motion between each other does not have a large difference. It’s a big disadvantage of this technique. In addition, I found that there is another problem from the application of image recognition which is how to integrate setting mode and operating mode. If the user interface would like to base on the image recognition technique for the user to not only operate the machine in their need, but also set what they want to recognize and what kinds of response that want the machine to react, the problem here is how the operation mode and the setting mode could be well dealt. Before operating, there should be a setting page to setup all the operation first, how to deal with that?

And how to get back to the setting from the operation mode?

52

Chapter 4

Although I think there are more variations and it’s more flexible and more intuitive operation from the application with image recognition or machine learning technique, how people can shift from the traditional operation to a new one, how do people know what should they do in the new operation. I don’t think it’s that much easy to just let the machine learn from us and we don’t need to change any behavior from our lives of interactions with machines and technology.

53

Chapter 4

4.2 Second intervention The second intervention is a bit jump from the first one. In the second intervention, I would like to understand more about the relation between technology and the creativity from people so that I tried to plan questionnaires and experiments for people to engage. Thus, I had two main actions for the second intervention. The first action was a questionnaire about the relation between creativity and technology for creators from different creative sectors to understand more about creativity and the impact of technology in a creative process. The other action was an experiment for some MDEF students (creators) with a questionnaire to see how different tools would drive or influence creativity. In this intervention, I was not focusing on the AI technique, but researching the technologies in general to see their impacts on creators.

The second intervention was about the exploration of the different influences of technologies for creators in their creative processes and expressions.

Objectives

• To understand more about creativity and the impact of technology in a creative process.

• Trying to think about how technology can get involved in creative process.

• From understanding how creativity might change from using different tools or technologies and then imagine how new technology (such as AI technique) can be shaped for creators and what is the possible relationship between creators and technology.

Methods / Activities

• The questionnaire about creativity and technology • The experiment with questionnaire about how creativity might change from using different tools or technologies

Tools / Materials / Techs (in the experiment) • • • •

Papers, pens Photoshop / Illustrator (computer software) Grasshopper / processing (programming) Sketch-RNN (Neural network interactive sketching system from Google Magenta)

People

• MDEF students 54

Chapter 4

4.2.1

Results

Action 1

Questionnaire – The research about creativity and technology (Link for questionnaire: https://forms.gle/nW43LDAhHAC6g5139) The first action for the second intervention was about using a questionnaire which has a series of questions to understand about how much technology impacts creators to do their works in different creative sectors, and what could be the possible ways that technology can stimulate their creativity.

In this questionnaire, there are 21 people in total from different creative sectors. There are 11 people from architecture, 5 people from design, 3 people from advertising and marketing, and 2 people are other.

The following is the analysis of the questions and responses which would be divided into 6 parts to explore and discuss from the questionnaire.

Part 1. To understand how creators find their ideas

55

Chapter 4

From the first response can be seen, most people’s creativity and imagination are inspired by images. And from the second response, creators prefer to find inspirations from other works and projects (art, books, films…), and on-line resources.

These results show that most creators thought that it’s easier to get ideas from visual things, however except images, a creative project also includes content, concept, or intention, so there are one third of people find inspirations from "Art, books, films…” which are something full of messages and information. Besides, from the results of second pie chart, it can be found that searching for resources online have already been a common and convenient way to find some idea for creators.

From the results in general, it could be speculated that references and templates might be a better way to inspire creators.

56

Chapter 4

Part 2. To understand how much do creators think that technology might help them in the creative projects

(0: Not at all; 5: Extremely)

(0: Not at all; 5: Extremely) From the graphs above, most people think that their creative possibilities have increased over the past few year. However, not so many people agree that their creativity and imagination are delivered by technological tools. It can be understood that the development of technology could increase the possibilities of creativity in some way, but technologies seem to still have some limitations for creators. In the other way to say, creativity was not entirely decided by technology.

57

Chapter 4

Part 3. To understand how much do creators think that technology helps or limits them, and how technology helps or limits them nowadays.

(0: Not at all; 5: Extremely)

58

Chapter 4

(0: Not at all; 5: Extremely)

59

Chapter 4

From the results above, most people are positive about the impact of technology on creativity, there are almost half of people think that technology helps them in "Discovering possibilities".

I was surprised that "Increasing efficiency" was not the option that most people think technology helps them, instead, most people agree that technology can help them in "Discovering possibilities" and "Exploring new expression".

However, there are still some possible reasons make people still keep negative attitudes toward technology that are "Difficult to learn", "Unintuitive", and "Inflexible". There is a noteworthy response which is “Too much possibilities to focus on�. There would be more possibilities because of more freedom. However, too much freedom sometimes makes more complicities, just like facing a piece of blank paper, it’s hard to start from empty. I think ways of stimulating creativity should be taking the balance between freedom and limitation.

Part 4. To understand how creators think about AI

(0: Not at all; 5: Extremely)

60

Chapter 4

(0: Not at all; 5: Extremely) The graphs illustrate that most people are not worried about their work being threatened, instead, they may be happy to collaborate with AI. It shows that most creators think the creativity will not be limited by technology or threatened by AI, but instead becomes a tool for enhancing creativity.

Part 5. To understand are there any problem from a creative process

61

Chapter 4

The pie chart demonstrates that there are up to 60 percent of people are afraid of technical issues during the creative process. Form the result, it can be speculated that using technologies in creations nowadays has already become a common thing, but creators still have relatively big problems in technical issues. Although many people would love to try new technology, there seem to be many limitations which might be “Difficult to learn”, “Unintuitive”, and “Inflexible” that from the previous results creators thought.

Part 6. To know how creators think about creativity Q: What does creativity mean for you?

Response: • Freedom • Ability to follow some crazy ideas in order to make changes • Freedom where I don’t have to follow the usual patterns but create my own • Joy • Relating things in unconventional ways • Life and what you experience • thinking out of the box/smart • Everything • Something makes by myself • Liberty of expression • Purposeful + aesthetic • Being able to express yourself • The ability to make various possible connections of relatable or unrepeatable things in order to create something new or find a solution to a problem • A different approach to solve a problem • Challenging your pre conceptions about things and knowing that everything is part of a story that you can question and change with your thoughts. It’s daring to play and have fun without minding the rules I guess • Creativity is response to the world • Coming up with something new • The ability to use everything you can use and create something that no one had ever seen

62

Chapter 4

Q: 3 words to capture the essence of creativity.

Response: Beauty / function / revolutionary / Expression / senses / imagination / Imagination / ultimate tool / lifelong learning / magic / senses / surprise / Research / intuition / culture / Make / imagination / unique / Execution / intuition / preposterous / imagination / interaction / invention / Read / travel / love / communication / passion / curiosity / Play / Question / Free-minded /lateral problem solving / ideas / ingenuity / invention / Love / Solution / Red-Hot / The animation of imagination / Transforming personality / Shifting points of view / Changing the degree of freedom / multi-disciplinary / open-mindedness / curiosity.

From the responses of two questions above, everyone has their definitions of creativity and it’s really hard to evaluate creativities. In my opinion, creativity is including originality and influence, and it’s a skill that combines the abilities of decision-making, problem-solving, and intention-expressing.

In order to research more carefully about the creativity, the plan of the second action was based on the abilities of decision-making, problem-solving, and intention-expressing.

63

Chapter 4

Action 2

Experiment and questionnaire – Creativity in the age of AI (Link for questionnaire: https://forms.gle/Cj6jmYBuT7eUaYMd7) What different tools influence people in the creating processes? And the way of giving creators possible sketches during the drawing process like Sketch-rnn project from Google Magenta could really inspire creators?

The second action for the second intervention was about making an experiment with a questionnaire which was about a series of tasks for presenting things (images/graphs) with different tools. I found 6 people from MEDF to do the experiment I planned because I think that people in the Master are all creators just in different creative fields and I can take advantage of them to be my resources in my project.

EXPERIMENT In the experiment, I would like to know how a creative process or design process changed when using different technologies to achieve the same goal.

The methodology in the experiment was using 4 different kinds of tools which were pen and paper, Photoshop/Illustrator (computer software), Grasshopper/Processing (programming), and Sketch-RNN (Neural network interactive sketching system from Google Magenta) to present the same task. There were 3 tasks I prepared and each task had different condition.

There are the plan of experiment (Figure 4.14) with the tasks and tools I prepared here.

QUESTIONNAIRE for the Experiment The questionnaire was a series of questions which were based on the experiment (3 tasks) that participants in the experiment need to answer.

The questions in the survey are divided into 3 parts by tasks.

64

Chapter 4

square

chair

mood (abstract stuff) ?

sketch

Tools

Tasks

Pen and paper Photoshop / Illustrator (computer software) Grasshopper / Processing (programming) Sketch-RNN (Neural network interactive sketching system)

computer software

#1 Try to present a precise square (in the center) x x

programming

sketch-rnn

#2 Try to present a chair

#3 Try to present a bright mood (happy)

x x

Choose a tool to present.

x x

Figure 4.14: The plan of experiment

65

Chapter 4

Task #1 – problem-solving Describing each process. The first task was a problem-solving task.

In the first task, I would like to see how the process and the thinking logic change when switching from one tool to another. The following are the process descriptions of drawing a square.

#1

#2

#3

Pen and paper I drew 4 lines in a circular manner, but in square shape.

Pen and paper I drew a circle, cross inside and then connected the intersections.

Pen and paper Fold twice in vertical and horizontal directions > Point the 4 intersec-tions (corner of the square) > Connect the point to point

Photoshop / Illustrator (computer software)

In Illustrator, I added a rectangle in square shape.

Grasshopper / Processing (programming)

Photoshop / Illustrator (computer software)

Pressed rectangle and hold shift to make square.

(Illustrator)

--

Grasshopper / Processing (programming)

I made rectangle with rectangle command, adding x and y size and defining plane.

(Grasshopper)

66

Photoshop / Illustrator (computer software)

Click Rectangle tool > Click the work space > Type length 200 mm and width 200 mm > Type XY position (Illustrator) Grasshopper / Processing (programming)

Use rectangle 2pt tool > Find XY plan > Find 2 diagonal coor-dinates of a square (Grasshopper)

Chapter 4

From the results of task 1, it can be found that rectangle in computer software have been defined by some parameters which are Length, width, and position (x, y) already. The logic of using the computer software to draw is unlike sketching, it’s more like to describe or calculate a square rather than draw a square.

#4

#5

#6

Pen and paper I tried to find a similar distance between the border and the lines I drew.

Pen and paper I put a little dot on where I thought the middle of the paper was and then I started drawing the left side line of the square then the top the right and bottom line.

Pen and paper I draw small square (7%of paper) in the cen-ter it’s not perfect and not balance.

Photoshop / Illustrator (computer software)

New doc > 200x200 > square tool > hold shift > try to see it is centred (Illustrator) Grasshopper / Processing (programming)

Rectangle tool > define value slider > 10 > attach value slider to X and Y (Grasshopper)

Photoshop / Illustrator (computer software)

I drew it from the rec-tangle tool using shift to make it square al-most at the center of the A4 l. Then moved it from the center to check if it matched the center of the paper.

(Illustrator) Grasshopper / Processing

Photoshop / Illustrator (computer software)

I make same scale and position by us Illustra-tor and use rectangle tool it look perfect square and use less time.

Grasshopper / Processing (programming)

I try to use rectangle tool and it’s taking much time to make it (Grasshopper)

(programming)

I used the rectangular component with a 10 value on x y.

(Grasshopper)

Figure 4.15: The results of task #1

67

Chapter 4

Task #2 – decision-making The second task was a decision-making task.

In the second task, I would like to know how people think and imagine a chair when they were using different tools to present. Try to find out how the creators decide how to present a chair by using different tools from their descriptions. The following are the descriptions about the chair.

#1 Pen and paper I used 2 rectangles from a certain perspective, and 3 lines as the legs of the chair.

Photoshop / Illustrator (computer software) I used 3 lines: 2 verticals and one horizontal.

Sketch-RNN (Neural network interactive sketching system) --

Photoshop / Illustrator

Sketch-RNN

I used pen and holding shift to draw straight lines in perspective.

It was very similar to the one in illustrator.

Photoshop / Illustrator

Sketch-RNN

A front perspective view, only straight line.

Sofa, perspective view.

#2 Pen and paper I made a curvy chair in perspective and used slight shadowing.

(computer software)

(Neural network interactive sketching system)

#3 Pen and paper Perspective view, more curve.

68

(computer software)

(Neural network interactive sketching system)

Chapter 4

#4 Pen and paper It is a square with a little perspective, then I make the legs, and I have in mind the one that is covered by the perspective. In the end I do the backrest with the same perspective.

Photoshop / Illustrator

(computer software)

I drew the bow of two of the legs and the base and replicated it for the hind legs, and also for the back.

Sketch-RNN

(Neural network interactive sketching system)

Same process as the hand drawn one, but added more details in the end.

between the back and front legs.

#5 Pen and paper It's a simple chair with solid back and seat with 4 legs. I perspective and then the back and then the legs.

Photoshop / Illustrator

(computer software)

I followed the same process as before but using the rectangular tool and rotating it. Then drew on line from one corner of the seat copied it to the other so it has the same height and connected them with a line.

Then drew one line for one leg and then copied it 3 times.

Sketch-RNN

(Neural network interactive sketching system)

Did the same thing as when I sketched with paper.

#6 Pen and paper I draw the wood chair in 45degree position (3D) it take 1 min to draw.

Photoshop / Illustrator (computer software)

I use rectangle tool to make shape of chair and use direct selection tool for adapt it make Figure it look like 4.16: chair (it look like metal sheet).

Sketch-RNN

(Neural network interactive sketching system)

I draw easy simple chair but it not perfect chair that I make in paper it look organic.

Figure 4.16: The results of task #2 - 6 cases of description about chair drawing

69

Chapter 4

(0: Not at all; 5: Extremely)

70

Chapter 4

(0: Not at all; 5: Extremely)

Q: In general, which chair you like most (Self-assessment)? Why? Response: • Paper drawing. It gave me more freedom • Illustrator one, it’s neat • Hand-draw one, more natural • Sketch RNN, because the flow of the brush was nicer. Less formal, more childish.

• Illustrator • In paper because it is perfect chair that Match with my perception Q: What did you consider when you draw a chair? Or you just follow your intuition?

Response: • I tried not to be too abstract • For curves it was easier to do it by hand, for illustrator, it’s easier to work with geometry in perspective • Form • Perspective and the basic form.

• Just my intuition and the perspective • I consider how to communicate people to understand that this is the chair

71

Chapter 4

From the results of task 2, it can be found that what people decide to present and what are the things they considered. There seems to be more details and curves in using pen on paper compared to using computer software. In computer software, geometry and line are the easier things to operate.

It was interesting that more people changed their images of chair when they changed the tools they used. It may be because of different tools have different advantages. From changing a tool to another, the logic of operation changed as well. That’s why creativity would be influenced by tools or technology easily.

Task #3 – Intention-expressing The third task was an intention-expressing task. Participant should choose a tool to express a bright mood. In the third task, it’s more abstract and expressive. I would like to know how people choose a tool to present a thing which has more freedom to present.

#1

#2

#3

Tool: Illustrator

Tool: Illustrator (Because I always use it for drawing)

Tool: Photoshop (Because there are brushes and it was easier to have a hazy effect)

Description: A sun.

72

Description: Abstract shapes circles and curvy lines because I think rounded objects are happy.

Description: Different sizes and light colors of dots.

Chapter 4

The task was drawing a bright mood and having description about the drawing. People can choose the tool they would like to use.

From the results of task 3, it can be found that people would prefer a tool which is easier and more intuitive way to use. It would be really hard and complex that using computational thinking to present just a mood. However, to think in different perspective, if you want to program a mood or emotion, what would be the parameters in the code? (The decision of the parameter would be related to what’s the expression.)

#4

#5

#6

Tool: Illustrator (Because it was easier)

Tool: Pen and paper (Because it helps me when I am trying to illustrate an abstract thought. Gives me more freedom)

Tool: Sketch-RNN

Description: that kind of looks like the sun.

Description: I draw a part of sea with

Description: Angle I draw head before because that part it easy to start and then body wing that time this program help me to make it.

That is what I consider a bright mood coming from memories.

Figure 4.17: The results of task #3 - 6 cases of description about mood drawing

73

Chapter 4

4.2.2 Reflection In the second intervention, I thought I focused more on the research part which was about making experiments to understand how people think and act with different tools. I found it is interesting that there are different relations between creators and the tools they use. For example, a pen as a tool for creators to present their thoughts has already seem to become part of human body, I feel that this kind of relation is more intuitive in an interaction. However, our relation with computer software is not like the relation with pen, so many people would feel they are limited by the new technologies or tools.

After doing some research and having the classes from Ron Wakkary and Ariel Guersenzvaig in the master course, I knew that there some different relations between human and technology.

From Don Ihde’s research – The Technological Lifeworld, there are 4 kinds of human-technology relations, which are embodiment relations, hermeneutic relations, alterity relations, and background relations. Human-technology relations is Don Ihde’s post-phenomenology approach to technology is an analysis of various types of relations between human beings, technologies, and the world. Ihde investigated in which ways technologies play a role in human-world relations, ranging from being “embodied” and being “read”, to being “interacted with” and being at the “background”. The relation between human and pen is an embodiment relation that technologies form a unity with a human being, and this unity is directed at the world.

In the age of AI, artificial intelligence would also become a tool for creators. What will be a better relation to co-create with AI? How will operation logic or process change? And how technology can be designed as an intuitive, easily-using tool for creators to achieve their imagination and creativity, just like a pen?

Technology can be designed as a creative tool, or can be designed as a non-creative purpose.

No matter the technology is designed as a creative purpose or not, it cannot limit personal creativity. People can decide what they want to use the stuff for, so how to train personal creativity is an important thing. However, we still use technology by following their normal usage usually, so if technology can be designed well, it will help people and have a better experience.

If technology would like to be designed as a creative tool, its purpose is for enhancing human creativity, what things should be concerned?

74

Chapter 4

Figure 4.18: Images from the second intervention experiment

75

Chapter 4

4.3 Third intervention How can designers or artists collaborate with generative AI in the creative processes?

Imagining that in the future, generative AI will become a main tool in the creative processes for creators to make their works. And the way of creating projects will become as farming, creators give the generative AI system resources (fertilizing), and the creative project will start growing. In the growing process, creators also can graft as they want to intervene the process. When the growth almost finish, creators can start cutting and subtracting something too much based on their sense of aesthetic and feeling.

The third intervention is a speculative process of making a creation. In the third intervention, I made an experiment with a questionnaire for creators to do. It was a drawing task which is subtracting (erasing / removing) something from a generated image based on the topic I set.

The generated image in the task was myself as a generative AI to create an image based on the same topic. Trying to simulate a scenario that creators collaborate with AI in the creative process.

Objectives

• To know how do creators react and think with making creation in a “subtractive way” • Try to speculate the possible scenarios of the creative process which is collaborating with AI

Methods / Activities

• Experiment, a speculative process of making a creation • A questionnaire after doing the experiment

Tools / Materials / Techs (in the experiment) • Photoshop • Myself as a generative AI

4.3.1

Results

Task: To express your own feeling about the situation (virus, quarantine, project...) recently by an image. Here is a generated picture and there is a PS (.psd) file for this pic. Try to subtract (remove) something from the picture (ps file) by using PS to fit your feeling. Note: subtracting only!

76

Chapter 4

#1 Subtracting from the original image

#2 Subtracting from the original image

#3 Subtracting from the original image

#4 Subtracting from the original image

Original image

#5 Subtracting from the original image

#6 Subtracting from the original image

#7 Subtracting from the original image

#8 Subtracting from the original image

Figure 4.19: 8 results of subtracting from the generated image

77

Chapter 4

Questions: 1. Are you inspired?

2. How do you feel the process?

3. How would you imagine this “subtractive” way of creation in the future, what the new creative process will be?

#1

Subtracting from the original image

1. I got inspired by deleting things and therefore getting different visuals.

2. This made me feel more relaxed because I didn’t have to think too much about creating something new (which I am doing for design dialogues) generally I feel weird.

3. I imagine it with a machine learning algorithm that creates new images informed by each subtraction. So maybe deleting something creates a chain reaction that moves the components of the image, rearranges them or even adds some new.

1 my inspired is from tangible virus and pollution that we saw in micro camera.

2 I’m feel good it not too difficult for me to use it.

3. Subtractive in my perception is about reshape and make from 2D to 3D. Another perception is about the way to keep it simple, move to minimalism.

#2

Subtracting from the original image

78

Chapter 4

#3

Subtracting from the original image

1. Yeah, it was fun to try and sort the abstract shape of it, into something that made sense for me.

2. I felt confused and thought that it was going to be hard to make something out it, but then I started erasing and saw a shape that looked like a person and decided to go with that.

3. The process reminded me of when you’re looking at the clouds and interpret stuff out of it (or also like the Rorschach Test) were you interpret a personal vision of something that is already there. Which could be an interesting approach as a creative process.

1. Yes, I was thinking maybe the blanks would become the visual focus rather than other elements in the image.

2. I still or maybe more use the elements I preferred and was used to.

3. I think this way of making creation can not only use in graphic design or drawing, but also can become a creative form in highly engaged and data-building.

#4

Subtracting from the original image

Figure 4.20: The responses from 8 cases

79

Chapter 4

Questions: 1. Are you inspired?

2. How do you feel the process?

3. How would you imagine this "subtractive" way of creation in the future, what the new creative process will be?

1. I imagined more different forms and patterns in the process.

2. There was too much black, I was scared, so I tried to make it less.

3. Maybe it could become a new form of making creation in a more interactive way, it would be fun.

#5

Subtracting from the original image

1. Yes, I feel that it’s very suitable for doing psychotherapy during the quarantine.

2. I feel that it’s like Gestalt psychology. I was confused at the beginning, but soon it gets easier and easier.

3. I would like to use this kind of activity for people to reflect their lives, remove something you pursue, then see what’s left.

#6

Subtracting from the original image

80

Chapter 4

1. Not so much. It’s like removing some stuff from complexity I saw to fit my feeling, but I think I would be inspired more by the process is from simple to complicate.

2. Uncertain, but then have been more stable.

3. It might be like making sculpture.

#7

Subtracting from the original image

#8

1. Yes, I have been trying to make doodles daily so this way a good exercise to jump from my regular.

2. I kept wanting to add things instead of only subtract. I felt a little limited to what I personally like to create.

3. I’m not really sure. I feel like if the focus continues to be on reusing then it might be addition rather than subtraction. Actually, maybe it’ll become multiplication or “Boolean 2 objects” instead of subtraction.

Subtracting from the original image

Figure 4.20: The responses from 8 cases

81

Chapter 4

#1 A chance to add something

#2 A chance to add something

#3 A chance to add something

#4 A chance to add something

Original image

#5 A chance to add something

#6 A chance to add something

#7 A chance to add something

#8 A chance to add something

Figure 4.21: 6 results of adding something

82

Chapter 4

4.3.2 Reflection From the results, I found that there are three types of thought from the people in the process: one is feeling more relax by erasing something, another is feeling confuse and unstable at the beginning but got feeling well after, and the other is feeling a bit limited. Although there are different types of thought, it’s undeniable that this process is somehow helping people to find their ideas. It’s hard to start from nothing. Comparing to drawing something from the empty space, it’s easier to be inspired from the subtractive process because there was something already for people to reference. And after the process of subtracting, they seem to be more eager to add something. There is a person also saying that she kept wanting to add things in the subtracting process.

It also can be seen from the results that actually creativity can also show up from the limitation. Most people can be inspired by the generated image, and from the subtractive process, people would get more ideas in their mind from really doing something. So maybe the next question would be How to amplify the creativity from the limitation? What is the interface would be? How to convert the personal experience and emotional elements to the algorithm?

And how can designers (and artists) redefine the value of their roles in the creative sectors in the age of AI?

83

Chapter 5 Future Scenario

Chapter 5

5. Future Scenario Speculating future scenarios is a means of reflection from the interventions. With the iteration between doing intervention and building future scenarios, really helping me to think broader and let me have more concrete thoughts. I think that future scenario is also a kind of vision of the project but said in a more clear way which is showing the possible future that the project can get involved and try to achieve.

"We are moving from things that are fabricated to things that are farmed and moving from things that are constructed to which is grown. " – Maurice Conti

5.1 The future scenario of AI in the creative process In the future, collaborating with AI will become an ordinary work for all kinds of creatos in their creative processes. With collaboration with AI, creatos will focus on how to farm their projects and the projects will grow up by themselves.

In the age of AI, Artificial Intelligence will not only become part of our daily lives, which will be applied almost everywhere in every industry and become a very powerful technology to give us more personalized options, more flexible operations, and more diverse experiences, but also be a tool to inspire and train our imagination and creativity. In addition, AI will make machines moving from being more efficient, productive, and smarter to being more creative as well. In the creative projects, AI will be a design tool and a creative for designers and artists to bring out creativities. And creators have AI literacy to use the AI tools and apply to the projects, and they will be very used to collaborating with AI in the creative processes. And the creative process will become a series of interaction with AI.

86

Chapter 5

Figure 5.1: The imagination of a design growing as a plant from the seed

87

Chapter 5

The role of the artist and designer will be no longer the same as before. AI will shift the human creator to become a creative director that will change the focus from creating to decision-making. With the collaboration of AI which will empower creators in the projects to enhance their creativity and imagination, and the creators will never be limited by any technological problem.

Humans will have a new relationship with machines. Being augmented by AI, using them in a more initiative way, enhancing creative expression, and learning creativity with them. AI will be the power to build the ability of creativity, and provide people more freedom to express their ideas. But more importantly, it will be easier for everyone to become a creator without any technical limitation and we have already been used to how to collaborate with AI. There is nothing important with skill anymore, however, in order to be augmented by AIs, rather than controlled or influenced by them, creativity should be an ability for everyone to have, and it will be the only thing we need to learn and train because creativity is a powerful ability of humans to survive in the age that objects become smarter and smarter.

88

Chapter 5

89

Chapter 6 Design in emerging contexts: COVID-19

Chapter 6

6. Design in emerging contexts: COVID-19 This was a special master year. We are in the context fo COVID-19 that everything was changing so fast. It’s a challenge for all of us to face and try to adapt to the circumstance although the recent situation was hard to deal with, as so many limitations. The COVID pandemic is a wild card that appears in the design space, and it’s too influential to ignore. So that it should be included in the consideration.

The following is the introduction of my experience adapting the project and showing the changes during the COVID juncture.

6.1 My new me Because of the COVID pandemic, our actions were limited and we were all hidden in our own domestic spaces. When the quarantine started, the biggest change for me was moving all the activities, things in the same place which was my room. All the classes, sessions, and works were getting on in digital.

My new normal was staying in the same place all days, the figure 6.1 is showing the drawing about my a day in quarantine, and kind of be isolated from the outside-world, but hyper-connected with everything in the world. The challenge for me in the new normal was to organize time and get focus on the works. Staying at home every day really made me feel too relaxed to work.

This crisis influenced how we think, feel, and act in so many different aspects. It changed our lives. We had everything remotely, having classes remotely, got information remotely, connected to others remotely, everything was happening online and we were moving from the physical world into digital.

At that moment, the project was not having a certain shape. The project was growing slowly, so in the new normal, it was shaped by the situation and also naturally led to being in digital and moving online.

92

Chapter 6

Figure 6.1: A day in quarantine

93

Chapter 6

6.2 Mapping my domestic experimental laboratory In the new normal, in order to figure out the situation and what I can do with the context, I was rethinking what’s the new opportunities appeared and what resources that I can get, so that I started trying to map my new hyper-local and hyper-connected design space in my domestic experimental laboratory, which is showing from Figure 6.2.

In my new domestic laboratory, the space is so tiny, not very rich with stuff, and my workplace is basically only on my desk in my room. However there were many resources coming from the digital, and the new normal situation was also pushing me even to find more things online.

There are tools and materials I have in digital which are software and online platforms that I can do many works there. I can also find tutorials and learn, have courses, and find almost anything online. Besides a bunch of digital stuff, the physical stuff I have are not that much which are a laptop, a phone, Arduino kit, a tool box, notebooks, and Wi-Fi. I notices that WiFi is the most important stuff in this hyper-local and hyper-connected design space. Without Wi-Fi/internet, we cannot do almost anything (fortunately, I have).

In this hyper-connected world, I can connect to people through social media platforms and many different communication software. I feel that this made it even easier to contact people in different places or countries because there was no longer a distinction between near and far. So I was planning to connect to the teachers from my previos school. With powerful internet connections, social media, and online resources, I was also starting to discover the possibility of the project working online. In the next chapter Hyper-local/Hyper-global (Chapter 7) would show more clearly what I did and plan to do.

94

Chapter 6

Figure 6.2: Mapping the digital and physical stuffs in my domestic experimental laboratory

95

Chapter 7 Hyper-local / Hyper-Global

Chapter 7

7. Hyper-local / Hyper-Global 7.1

Experimenting with my project in emergent contexts

After a period of quarantine, I had more used to the situation now. So I started looking for and trying out some stuff that was meaningful for the project in this emergent context.

In order to find the opportunities that the project can continue in the next two months, I tried to do something to discover and to approach the project. I had two actions here as my first intervention after the COVID situation. First action I did was to try out some software and online platforms which were related to machine learning, and the other action was connecting with two teachers and having a chat about the art, creative computing, and physical computing education with their teaching experiences.

7.1.1

Intervention

In this intervention I used first person perspective to do my two actions. I took the opportunity to try out different tools in order to find the possiblities of ML in creative field and reflect about the AI technique, also prepare myself to have ability to teach others. The following are details of the two actions I took.

Action 1

There are lists of software and online platforms I tried in the first action: • mBlock (with Arduino) • Teachable Machine with p5.js (and ml5.js) • Wekinator with Processing (and Arduino) • Runway ML

Action 2

There are basic information of two teachers that I contacted with: • Teacher 1 (Art and Design department in the college) – teaching Computer-aided Design and Manufacture and Fundamental Computer Programming • Teacher 2 (senior high school) - Teaching Fine Art

98

Chapter 7

7.1.2

Results

Action 1 - Trying out the software and online platforms mBlock with Arduino

Figure 7.1: Arduino build-in LED turns on (left) by putting a hand in front of webcam

I used the teachable machine which is an extension in the mBlock software that I can use it to build an image classification to recognize different images by the camera and to trigger the case that you want it to do. In here, I connected mBlock with Arduino to control the LED turning on and off by the classification.

Teachable Machine with p5.js

Figure 7.2: Teachable Machine and p5.js online platform workspace

Teachable Machine is a great online platform for everyone to know and experience machine learning easily. I was curious about how can I do after collecting the image data from the Teachable Machine, so that I found the classification model which was built by Teachable Machine can put in the p5.js - which is a creative coding platform - online workspace to program a bit to do what you want it to do such as showing images, generating sound, etc.

99

Chapter 7

Wekinator with Processing (and Arduino)

Figure 7.3: Changing the colors by changing different actions

Wekinator is a machine learning software for creative computing. I also used it to build a image classification which can change the processing window’s color showing by recognizing different images or actions. It’s more difficult to operate compared to the Teachable machine, but it can combine more different software and devices to have variety of implementations.

Runway ML

There are many things can do in Runway ML software that you can use its pre-train model to generate new images, stylizing photos or videos, and you also can train your own models.

I tried many differet stylizing models in Runway, and I was really surprised by its powerful function and pretty satisfied with the results. As figure 7.4 which is showing a photo I took that was stylized by Runway.

I also tried to train my generating model in Runway. I collected the 220 chair images (left sife of figure 7.5) online as my dataset then train it in Runway. I would like to know what it can generate and what’s the possible things might help me in the creative process. After the model was built, I generated new chair images in Runway workspace as figure 7.5 (right). I was suprised by the results again. Although some chairs might look strange and surreal (that is because there are only 220 images in the training dataset), it’s kind of helping me to imagine new chairs in some way.

100

Chapter 7

Figure 7.4: The stylized photo by Runway ML

Figure 7.5: Training images (left), and generated chair (right) by Runway ML

101

Chapter 7

Action 2 - Connecting to the teachers With asking and discussing to the teacher who is teaching physical computing and fundamatal programming in art and design college, there are a list of key problems about teaching technology-related knowledge and tools to art and design students: 1. Higher technical threshold 2. limited imagination for connecting their works with new technology 3. lack of knowledge and skills 4. lack of motivation 5. lack of systematic learning methods The biggest problem is that art and design students don’t really want to learn programming and don’t think it’s related to them. It might be the problem of their learning willing, however, I was thinking if there is a possibility of teaching methodology to motivate interest students or learning methodology for students to learn these complicated things easier?

7.1.3

Reflection

These two actions helped me to reflect on the machine learning tools with creative computing and physical computing tools now, and to understand the circumstance of the digital and computational education in the art and design college.

I think that there are already have many creative machine learning tools for creators to apply the ideas. However, there are not that many programs or courses teaching about them properly. So I was thinking that maybe I can contribute a bit in the education to think about the better way to teach creative comutation in art and design. Because considering the COVID context, I was thinking the things I can do could be a online course, video tutorials, website, toolkits, etc.

After piloting myself in the two actions, I had a plan for the project. I planned the things I should do including Researching/learning, Doing, Interacting, and Reflecting & modifying. I also tried to plan the next few weeks until the final presentation that can be seen in figure 7.6.

102

Chapter 7

Researching / learning

18 May

Creative methods / Design Methodologies Research - Design Thinking & Computational Thinking - Human-centered Design - Speculative Design

Objective -To understand and know more problems and ideas from different people -To develop a methodology for artists and designer to learn new technology(/AI)

Learning - Machine learning technique - Craetive coding - Physical computing

25 May

Doing

Action -Sharing the methodology in the media (Website/video/instagram...)

-Online activity Objective -having collaborative idea to rich and modify the methodology

Planning to DO - Methodology - Toolkit - Media (Website/video/instagram...)

Action -Methodology research -Tool learning -ML workshop for art and design students

01 June

Action -Modifying the methodology -More online activity Objective -having collaborative idea to rich and modify the methodology

Interacting with people - Artists & designers (technology-related & non-related) - Art and design teachers / educators in technology / engineering-related domain - Art and design students / learners

Reflecting & modifying

08 June

Action -Another ML workshop for art and design students Objective -To test the methodology

15 June

Reflecting & documenting

22 June

Design Dialogue

Figure 7.6: The plan for the project in the next two months

103

Chapter 7

7.2 Collaborating with others for my project After experimenting with my project in emergent contexts, I created a new intervention in this new normal which was my second intervention in the COVID context. Because of trying out many different tools and contacting with the teacher in college, I planned a course for art and design students and coordinated the time with the teacher.

7.2.1

Intervention

In the intervention, I prepared an online course for art and design college students to learn about the artificial intelligence and creative machine learning tools through zoom video call with two surveys to understand the thoughts from the students. The following is the coure I planned:

Machine Learning for Human Creative Practice - A ML course for art and design students

• Providing art and design students more imaginations and capacities to find possibilities for applying AI/ML in art and design practice. • Introducing machine learning tools and examples for art and design students to operate and apply in their works.

A survey before the course

Course Structure

1.

Discovering AI Experiencing AI 2.

Understanding AI / ML 3.

Operating ML tools 4.

Inspiring

(from some AI creative practices)

Reflecting

(about the relation with AI and new techn ology tools)

A survey

after the course

Figure 7.7: The plan for the course

104

Chapter 7

Objectives

To know what art and design students understand about AI and ML. • Providing art and design students more imaginations and capacities to find possibilities for applying AI/ML in art and design practice • To reflect the value of AI literacy for art and design students • To develop the methodology for teaching new technology in art and design practices

Methods / Activities

• A machine learning course for creative practices • Two surveys, one is doing it before the course, the other is after

Tools / Materials / Techs (in the experiment) • mBlock • Teachable Machine (with p5.js & ml5.js) • Wekinator (with processing)

People

• Art and design third-year students (16) • Their design studio teacher (CK Lim)

Input Collect

Train

Training Data

Learning Algorithm

DecisionModel making

Sensor

Computer decides what action to take, based on input

(Classification, regression…) if (value > 500) { print(“High”); } else { print(“Low”); }

Output

Figure 7.8: A slide talking about the supervised learning working pipeline

105

Chapter 7

7.2.2 Results Pre-survey In the pre-survey, there are three questions to briefly understand what students think about AI: 1. How much do you understand about AI? 2. How much can you imagine the possibilities of AI in creative practice?

3. Have you ever think that AI can be a tool for creative practice?

(0: Not at all; 5: Extremely)

(0: Not at all; 5: Extremely) 106

Chapter 7

From the results that most of people are not very understand about AI, but can imagine using AI in creative practice, and would like to take AI as a tool in their creative practices.

107

Chapter 7

Post-survey In the post-survey, there are 70 percent of people agree or strongly agree that they learned a lot from the course, and there are also 70 percent of people agree or strongly agree that they changed their impression of AI/ML. Surprisingly, all the people found more possibilities of AI/ ML in art and design practice. There are relatively fewer people who agree that they found more ideas and imaginations about connecting AI/ML with their works. And most of people think this AI literacy can help them in their creative practices.

108

Chapter 7

109

Chapter 7

Q: Any new thoughts about AI? What does it mean for you?

Responses: • It’s interesting, but I’m not interested in.

• Using it in the life is very convenience, using it for creations might be fun, but it’s still hard for me to practice and need more ability to apply in works.

• Although I didn’t really understand about AI very much, but I believe it can bring unlimited possibilities to creations. It would be very delicate in art. I believe that the combination of AI and art can bring contemporary art to be a very dramatic and very different form.

• I have always known that the application of AI, but I didn’t know many types. Today I saw more functions of AI. I think AI can be used as an assistant or tool or it will be like a human.

• I feel that it could be a tool that can be operated and not just reply me 0 or 1, might react more like human, also can decide for me.

• New possibilities for creativity • It might be a template for creating new things (gallery) • An essential tool in future life Q: If AI/ML can help you with your creative practice as a co-creating assistant, what will it look like? How will it help you?

Responses: • I have to learn and understand it first.

• It can help me accomplishing massive processes fast and conveniently • Media for creation • I think it can help me in early ideation thinking. I always feel that finding out idea is the hardest part for me when I do a creative project, so I think if AI can help me linking different ideas, for example, I have an idea then it can connect and show me the things such as some elements, images, etc which are related to my idea. It can inspire me more creativity out.

• Can help me to think about the shapes and forms I might like (? Giving me opinion • It can help me counting data and classifying judgment. Creators can collect data by themselves and define the output they want.

• Would become a holographic image appearing in the space, cooperate with the robot arm to achieve a fully automatic living environment • A simulator for design work

110

Chapter 7

Q: If you collaborate with AI/ML in your creative process, what will your works to be different than before?

Responses: • To be more digital • Might be able to jump out of my own circle of thinking. Maybe I can do more than I think as long as having the tools, the message I want to convey may be handled in a more expressive way.

• I really don’t know, but if it’s an interactive installation or interactive art, the things we talk about will definitely become very different.

• Would be digitalized or might be more functional and more complete • More able to adapt to people or replace people (?

• Breaking away from the result of 0 and 1, and makes creations have more possibility of reacting and feedback, and make it seem to have emotions and vitality • Can be able to analyze huge data and likely possibilities. Good and bad.

• I can present what I think more completely, and will not limit my thoughts because of the lack of manual skills.

111

Chapter 7

7.2.3 Reflection When I taught in the course, I found that the students were very unfamiliar with AI. It’s a new domain for them and there was the first time for the art and design students to really learn the Machine Learning in a more practical way. AI is a fuzzy existence for the students, as the results of pre-survey, it can be seen that they can imagine AI apply in the art and design projects, but have limited understanding of how to use or operate it.

In the course, I started with a little lecture to give the students a basic knowledge about AI and Machine Learning, and then I gave them some examples and experiments to feel and better imaging and understand. This way worked kind of great to attract their eyes. However, the most important part of the course, which was teaching how to really use the Machine learning tools in creative practice, didn’t work very well. I found that I still didn’t know how to teach the tool operation better because I just taught step by step and it can very easily become too technical. Although the operation example in which I had already tried to pick the one that was not very hard, there were some students still feel it’s hard to understand.

In order to let the students understand better about the operations and have more complete learning, after the course, I was thinking maybe to film a series of videos and post them in their group on facebook (figure 7.11) might be a better way for them to learn so that they can watch the video repeatedly. Therefore I tried to record some of my teaching and make videos for the students, and the feedbacks from them were quite good. After a couple of weeks, I saw some students was using and applying the Machine Learning tools in their design projects (figure 7.12) which were shared in their facebook group. I was happy to see that the students can use new tools they learned and try something different in their project. With changing the format of teaching from online courses to videos, I started to think that perhaps different formats of teaching also can influence the effectiveness of students’ learning.

After thinking about the formats, channels, social media, and also considering the emergent context, at the end of the reflection, I decided to build a website and social media platform (figure 7.13) to contribute what I learned and thought about the essential knowledge of using Machine Learning in the creative field. And the site trying to not only share the possible future of the relationship between human and creative AI, but invite creators to think about and build the future with me.

112

Chapter 7

Figure 7.9: Photos from the online course on zoom (teaching Teachable Machine)

113

Chapter 7

Figure 7.10: Images of teaching Wekinator with Processing and Arduino

Figure 7.11: Images of posting the operating videos on their class group in facebook

114

Chapter 7

Figure 7.12: The practical implementation of ML tools in the students’ projects

115

Chapter 7

ABOUT

About Experiencing Learning Applying Imagining Reflecting

Resources Contact

Machine Learning for Creativity A series of machine learning practices for artists and designers to augment the creativity. In this site, providing different tools, tutorials, and issues for all kinds of creators to achieve the imaginations. 一系列的機器學習練習，讓創作者創造作品新的可能性。

Learn more

Figure 7.13: The website and the instagram account of

Machine Learning for Creative practice

116

Chapter 7

The website I built was called Machine Learning for Creative practice. It’s a website for people to explore the possibilities of AI in creative practices, broaden the imagination, reflect on the relationship with AI, and augment creativity. When I was building the website, I planned a structure of AI learning based on the plan of the course for art and design students (figure 7.7) including five parts which are “Experiencing AI”, “Learning AI”, “Applying AI”, “Imagining AI”, and “Reflecting on AI”. People would start from experiencing AI by playing some AI creative projects like “quick draw”, “sketch-rnn”, deep dream generator”, etc, then learning the fundamental knowledge of AI and machine learning. After that, people can choose that they want to go ‘Imagine AI’ guided by the toolkits and card deck to imagine the new possibilities of AI in creative practices, or go ‘Applying AI‘ which is learning practical part of AI in creative practices from the tutorials and videos (would be more about technical operation). And finally “Reflecting on AI” would lead people to rethink what they learned, built, and imagined at the moment.

The structure of the five parts is shown in figure 7.14, and there are some images of the website as figure 7.15 shows.

Imagining AI

Experiencing AI

Reflecting on AI

Learning AI

Applying AI

Figure 7.14: Five parts of AI learning structure

117

Chapter 7

About Experiencing Learning Applying Imagining Reflecting Resources Contact

About Machine Learning for Creativity Artificial intelligence (AI) has already been in our daily life and become ubiquity. There are more and more people starting discussing not just artificial intelligence but “artificial creativity”, and researching how machines can be creative. There have been some implementations in the creative field and art industry that used AI techniques in the creations, there are also some machine learning (ML) tools that were developed for creativity.

Have you ever been curious about whether machines can have creativity or not, been worried about AI could replace your role in the works, or thought about using AI in your creative practices, but it’s like a black box that seems to be complex and hard to understand? This is the right place for you!

There are five parts in this site, you can explore AI/ML from experiencing which lets you aware of its existence consciously, and learning the basic knowledge to applying the machine learning tools and having the ability to imagine the future and reflect. Entering the world of collaborating with AI in the creative field is not as hard as you think. You don’t need to have a background of coding, the only thing you need to have is your curiosity, then you can unfold the black box called AI. 人工智慧已經在我們的生活中，無所不在，而愈來愈多人開始討論人工智慧可能會擁有的創造力，創意產業、藝術產 業也紛紛有不少的應用，而現在也已經有許多用用於創作的機器學習的工具被開發出來，讓創作者發揮創意。你是否 有曾經好奇過，機器到底能不能有創造力？亦是否曾經擔心過人工智慧可能會取代你的工作？或是希望將AI技術應用 到你的創作中，但AI卻像是黑盒子，好像很複雜很難懂？那麼你來對地方了！ 此站分為五個部分，你可以從體驗AI開始，有意識地認知到其存在，並學習AI基本知識，到會應用一些機器學習的工 具，並有能力想像及反思，希望創作者在學習及應用AI的過程中，也培養對其的未來想像及反思能力。踏入與AI創作 的世界其實並不困難，可以不需要有程式基礎，只需要擁有一顆好奇心，你將揭開名為AI的黑盒子。

Who is this site for 適用於什麼人 People who are interested in interactive art / design and creative industry, also who would like to try / learn different types of creative expressions / processes. Or people who are interested in artificial intelligence and its future development in the creative field.

Whether you are designers, artists, musicians, engineers, etc., you can combine what you learn in this site to expand your profession and create the future. 對互動設計與藝術產業有興趣，想嘗試不同創作形式的人，亦或對人工智慧以及其在創意領域的未來發展有興趣的人。 不論是設計師、藝術家、音樂家、工程師……等，都能結合此站的內容，拓展專業，並創造未來。

What you can learn in here 可以學到什麼 - Know the applications of AI / ML in the creative field, and broaden your imaginations to AI / ML - Learn how to use the machine learning tools like Wekinator, Teachcble Machine, Runway ML etc.

- Reflect on the relationship between yourself as a creator and machines, AI or other tools in a creative project - 認識機器學習在創意領域的應用，開啟對人工智慧及機器學習的想像 - 學會使用Wekinator、Teachable Machine、Runway ML等機器學習的工具，並與機器共創作品 - 反思自身作為創作者與機器、人工智慧亦或其它工具在創作中的關係

Figure 7.15: Images of Machine Learning for Creative practice website

118

Chapter 7 ABOUT

About Experiencing Learning Applying Imagining Reflecting Resources Contact

About Experiencing Learning Applying Imagining Reflecting

Resources Contact

Future Scenario

Tutorial Here providing a series of tutorials about machine learning for creative practices. It’s welcome for everyone

In the rapid development of technology era, machines will be not only smarter, but also become creative which

who don’t have computing science background to learn and play with machine learning tools in your next

provides creators more possibilities in creative process and expression.

creative project.

There are different possible future of relationship with creative machine. It’s what you are expected?

這裡將提供一系列關於機器學習工具應用於創作的教學。非常歡迎

What do you think the better relationship would be? How do you imagine that future? 在科技發展快速的時代，機器不僅變得更聰明，還可能變得更加有創造力， 以提供創作者更多在創作過程及表現上的可能性。

What might be used in these tutorials. 教學中可能會用到的工具。

這裡有關於許多與創意機器的關係之不同可能未來的想像，這些是你所期待發生的嗎？ 你覺得什麼才是創作者與創意機器之間該保持的關係？你或如何想像這樣的未來呢？

ALL

ALL

Image

Sound

Runway

Text

Classification

Teachable Machine

Regression

Visual Art

Design

Music

Generating

Wekinator Automating Aesthetics 美學的自動化

Interactive expression 互動表現

Interactive creative process 互動性的創作過程

Generative design 圖像生成

Generative design 生成設計

Generative music 音樂生成

Copyrights © 2020. Created by Ching-Chia Renn

ABOUT

About Experiencing Learning Applying Imagining Reflecting Resources Contact

Copyrights © 2020. Created by Ching-Chia Renn

Experiencing About Experiencing Learning Applying Imagining Reflecting Resources Contact

ABOUT

ALL

Visual Art

Design

Music

What is Artificial Intalligence?

Sketch-RNN Demo

Deep dream

What is AI? Artificial intelligence has been given a significant amount of attention in the last decade. Both public and private interests in AI have risen exponentially since its break from machine learning in the mid-1980s. Like machine learning, AI is comprised of a collection of algorithms, which are executable mathematical functions that process data through a series of rules and step by step instructions. AI algorithms make use of data and rule-based logics to model and simulate environments, with little human interference.

Autodesk Generative Design

Nvidia GauGAN

NSynth Super

25 moments that have defined AI Look through some of the key dates and developments in mathematics, computing, science and culture that has led to the artificial intelligence that we know today.

“The field of study that gives computers the ability to learn without explicitly programmed.”

— Arthur Samuel (an American pioneer in the field of computer gaming and artificial intelligence)

The spring spyre

What is intelligence Write your own definition of creativity.

What is creativity?

Submit

Copyrights © 2020. Created by Ching-Chia Renn

Copyrights © 2020. Created by Ching-Chia Renn

119

Chapter 8 Project and personal identity

Chapter 8

8. Project and personal identity In this chapter - project and personal identity, there would be including two parts. The first part is about creating the biography and constituency as developed for Ron Wakkary’s session, which is understanding myself and what’s my capability. The second part is the reflection of new weak signals in the future scenario as developed for Laura Cléries’ session, which is about reflecting on two previous experiments and analyze the opportunities for reframing the future new normal that I would like to achieve.

8.1 Creating my own biography and constituency I was coming from the art and design school and having a year of experience of interactive design in a fablab, so I was more familiar with designing stuff itself and researching the tools. Before I came to the Master in Design for Emergent Future program, the designs did were tangible stuff either physical, digital, or mix, and I didn’t have any experience of project planning, people engaging, or educating (such as hosting a workshop). However, during the master’s program and doing the project this year, I was approaching the education aspect more and designing the things about methodology and system, not a product or stuff. It’s not a thing that I’m used to, but going through the process of the master project, it riched me with different elements. There is a timeline of the brief development of the project (figure 8.1) shows the pathway that how the project grows and I also grew with the project. I was going from thinking about interactive design and physical computation to focusing on education.

The future of me will keep researching and doing the works that are related to physical computing, creative computing, and interactive art and design as a designer and artist. At the same time, the future of the project will keep growing and have an integral online platform that provides a collection of machine learning tools, practical courses, toolkits and card deck, future scenarios of applying AI in the creative field, and a space that people can upload and share their works built by ML tools. The resources in the online platform will be rich not only by myself but also by many people from different creative sectors. There will be an AI creativity community that the people (like artists, designers, engineers, art and design backgrounds’ students or teachers) who are interested in AI/ML applications in the creative field will appear. And the community will also grow and the people in the community share their experiences in social media (online platform, Instagram, facebook etc). When there are more and more AI tools are developed for inspiration, creativity, or interaction with a machine, there will 122

Art and Design

Chapter 8

Art and Design department in the University - Digital Media and Art Design Program - Art, Design & Engineering Creative Integration Credit Program

Work

Fabrication lab / maker space - Human-Computer interaction workshop (assistant) - Developing assistive device (design) - Managing the machines the fab lab

DD term 1

- Human-machine creative collaboration - Technology for equality

1st Intervention - Exploring and researching AI / Machine Learning - Exploring and researching interactive interfaces

2nd Intervention

MDEF

Weak signals, Interventions, Project construction Technology + machine with art + design, Creating my hybrid profile

- Exploring the impacts of technologies in the creative field

3rd Intervention - An assumption of collaborating with generative AI to make a creation in subtractive way

DD term 2 - Human-technology collaboration relations in the creative fields - A methodology for designers and artists to identify their roles and survive in the age of rapid development of technology

Re-thinking the project during the COVID pandemic - Machine learning & creative computing (first person perspe ctive) - Preparing remote courses / video coures - Creating website, social media to share the methodology

Now DD term 3 - Reflecting

Inviting creators to re-think the relation with new tech & machine - AI-related project (art / design) - Providing AI courses / workshop for art and design students - Keeping uploading the do cumentations to the website

Getting job - Interactive design - Digi-art / tech-art

Artist-in-residence

Future

Promoting multi-disciplinary collaborations in the creative industries and art&design education

after life - New structures of AI and creative computing edu cation in art&design - New types of collaboration with tech&machine in the creative field

Creations

Figure 8.1: My evolution (me and my context) from before, during the master, and after

123

Chapter 8

be a new series of courses in the art and design college which become a fundamental course for art and design professional students to learn relevant knowledge about AI in creative applications. And the project - Augmenting Creativity - is a beginning of the new kind of revolution in creative computing education that rethinking about not only the relationship between human creators with machine and technology as creative tools in the creating process or expression, but also the proper way to educate the art and design professional students to integrate the new technology in their creative projects.

The future of the future of the project will relate to the whole art and design education and the creative industry. There will be a new structure of AI and creative computing education in art and design discipline, and a new type of collaboration with technology and machine in the creative fields. In this future, everyone has the knowledge of using AI, and collaborating with AI will be a common thing for creators. In this kind of powerful computer future, everyone works with computers, creative machines, technology, and there is almost nothing they cannot do. At this moment that humans almost forgot how to draw and craft, what can we recap the values of traditional skills or revive the past creations?

8.2 Reflecting about new weak signals in my future scenario From the reflection on two previous experiments, which are ‘Experimenting with my project in emergent contexts’ and ‘Collaborating with others for my project‘ sections, I tried to analyze the opportunities for reframing the future new normal that I would like to achieve. After reflecting on the two experiments, I found that the project was going from human-machine creative interaction to education about creative computing, physical computing, and ‘AI creativity literacy’.

The weak signals I chose at the beginning were ‘human-machine creative collaboration‘, ‘technology for equality’, and ‘fight AI bias’. However, after the whole world changed into the new context, and after doing and reflecting the previous experiments in the context and some iterations, there were some weak signals fading, some were augmented, and there might be new weak signals appearing at the moment. In the project, ‘technology for equality’ and ‘fighting AI bias’ was fading, and the one signal which was augmented was ‘human-machine creative collaboration‘ because I was becoming more focusing on the human-machine relationship in the creative field. In addition, I think there were also some new weak signals showed up here for my project around new forms of creation, new types of relationships with 124

Chapter 8

What I chose at the beginning

Augmented

Faded

New

Weak Signals Life in the times of Surveillance Capitalism - Attention protection - Dismantling filter bubbles - Circular data economy - Truth wars - Redesigning social

Designing for the Anthropocene

Life after AI - the end of work

- Climate conscience - Inter-species collaboration - Long-termism - Carbon neutral lifestyles - Fighting Anthropocene conflicts

- Technology for equality - Fighting AI bias - Imagining new jobs - Making universal basic income work - Human-machine creative collaborations

After the Nation-state

Kill the heteropatriarchy

- Making world government - Rural futures - Pick your own passport - Refugee tech - Welfare state 2.0

- Non-heteropatriarchal innovation - Imagining futures that are not western-centric - Reconfigure your body - Gender fluidity - Disrupt ageism

New Weak Signals (from Laura ClÊries) 01 Optimistic futures 02 Taming tech’s influence 03 Protective tech, protective everything 04 New digital communities 05 Privacy era 06 Legacy preservation 07 Disaster-proof destination 08 The new super-creatives 09 Unconventional brand action

10 Future-proof ingredients 11 Regenerative agriculture 12 Skincare 2.0 13 Anti-excess consumerism 14 The new superstore 15 Health concierges 16 Wellness architecture 17 Digital spas 18 Engineering companionship

19 New payment gestures 20 Gen Z finances 21 The new language of advertising 22 The gaming multiverse 23 Novel dining formats 24 Renewed faith 25 Gamescape travel

Macrotrends

Market opportunities

- Technology - Culture & Education - Social & human behavior

- Digital services & physical UX - Mobile spaces and services - Entertainment - Emotional tools

Figure 8.2: The new weak signals, microtrends, and market opportunities in the future scenario of project

125

Chapter 8

machines, and a new structure of education and digital community about art, design and technology. In the Laura Cléries’ session, I explored the new weak signals from Laura and found that "optimistic futures", "taming tech’s influence", "new digital communities", and "the new super-creatives" could be my new weak signals in the project. From the new weak signals, I discovered that the present opportunities for the project I haven’t explored before. First of all, the "new digital community of AI literacy for creativity" will be built, and in the future, human creators will have ability to tame technologies better and will not be influenced by technology because of having basic literacy. In addition, there will be a new relationship with technology and machines. With collaborating with the creative AI, human creativity will be augmented and there will be a "new super-creatives" appear. I think the appearing new opportunities shaped my future new normal scenario to become not just focusing on how people interact and collaborate with machines in a creative process, but also about how people would learn about machines and new technologies and what is the better way to teach technology and machine in the creative field.

After rethinking the weak signals, it’s also time to exploring possible macrotrends and market opportunities for the project. The macrotrends of the project I think would be mixing "Technology", "Culture & Education", and "Social & human behavior" that technology will integrate into the culture and art & design education, and will change the social and human behavior.

Furthermore, the market opportunities for the project could be "digital services & physical UX" that AI will become a powerful tool for designers to get ideas and explore new forms of chair designs (Figure 8.4) for instance and there will be an online learning platform with the Instagram community about Machine Learning for creativity for creators to learn and explore the new possibilities of creations with AI and others. There are also the possible alternative markets for the project which are "mobile spaces and services", "entertainment", and "emotional tools".

126

Chapter 8

Culture & Education

Technology

Digital services & physical UX

Social & human behavior

Macrotrends Entertainment

Emotional tools

Creators can create their own style with AI creative tools

Market opportunities Taming tech’s influence

Mobile spaces and services

Technology for equality

Optimistic futures

Human-machine creative collaboration

New digital communities Augmenting Creativity

The new super-creatives

Fight AI bias

Artists and designers will be familiar with AI

New expression

AI creative literacy AI become a common tool in the creative field New creative computing education

What I chose at the beginning

Augmented

Faded

New

Figure 8.3: The relation of new weak signals, microtrends, and market opportunities in the future scenario of project

127

Chapter 8

Figure 8.4: The possible future of collaborating AI system to get insparation in a chair design process and an online learning platform with instagram community about Machine Learning for creativity

128

Chapter 8

Tangible programming kit Universal construction kit

Learn the world in different perceptions / To have different perspective of experience

inspiring machine

play / have fun connect to people

Augmentation

disrupt ageism (inter-generational)

Technology for equality

Learn empathy Testing different ML tools

Develpoment & Testing

Learn creativity (Tech can help)

Automation

Human-machine creative collaboration applying AI/ML in creative field

Age of AI

disrupt discrimination Fighting AI bias

human behavior adapting

Magenta

ML course for art and design students

Speculation & Projection

Creative computung education

State of the Art

Atlas Weak Signals

Building website AI literacy in art and design education

RunwayML

Navidia GauGAN

Augmenting Creativity

Autodesk Generative Design

Teachable Machine Toolkit and methodology for learning AI in creative applications

performance

middle school students

workshop

New relationship with machines

Oliver (ex-MDEF student who was working Cooperator on the AI field) Santi (learning future unit) Learning Future Unit (Fab Lab Bcn) Lucas / Ramon (AI expert)

Activity

People

Wekinator

Creative Professionals

Visual impairments

Intervention

Microbit

Tools / Materials

Arduino

mBlock

Figure 8.5: Mapping the development & testing and speculation & projection into the project

129

Chapter 9 Project the interventions into the future

Chapter 9

9. Project the interventions into the future 9.1 Emerging narratives Vision

Co-creating with the creative machine, and the human creativity can be augmented by new technology.

Communication Strategy planning Audience engagement Who

Identify audience

Objectives

Reason for communication with them

Barriers

Possible barriers to their engagement or interest

Key messages

What you will tell them, what them need to hear

Actions

Channel

What you want them to do, a call to action

Where will you be in contact

IAAC faculties

To give them a better understanding of my project

The content of the presentation and the thesis are too much diverse information and hard to get the main idea

- The structure of my project - My vision and mission

- Get feedback from them - Giving me suggestions

- Presentation (a video around 6 min) - Thesis

Art & design students (Target)

- To make them interesting about AI application in their creative practices - AI creativity could be new trend in art and design

- Lack of motivation - Limited imagination

- AI could be a creative tool (for creative expressions or apply in processes) - AI is a big issue in creative field - The necessity of AI literacy for creators

- Applying the ML tools in creative practices - Imagining different future scenario of applying AI/ML in creative processes - Reflecting the future scenario of AI/ML

- The online platform - A short video

Art & design teacher (Collaborator)

To give them a better understanding of my project

-Might not have the same vision -Might not be interested about new technology

- The structure of my project - My vision and mission

- Sharing me the teaching experiences and methodologies - Giving me suggestions

- Presentation - The online platform

Engineers (Collaborator)

To give them a better understanding of my project

--

- The structure of my project - My vision and mission

- Helping me the technique part - Giving me suggestions

- Presentation - The online platform

General public

- To make people interesting about - To let people think about possible future scenarios of AI applications in the creative field

- Slow information spreading - The posts are not attractive

- The possible future scenarios of AI applications in the creative field

- Responding me the posts to having collaborative imaginations of future scenario of AI applications in the creative field - Discussing the issues

Social media (IG, FB...)

Figure 9.1: The plan of audience engagement (using the table template from Kate)

132

Chapter 9

Mission

Change the human-machine relationship to be a more collaborative way, shift the role of machines from automation to augmenting human creativity, and to build a new way of creating and expressing creations. Make AI to be a familiar tool for creators to apply in their creative projects and will never be a black box for them.

Stage 1 Building an online AI creativity discussing learning platform for all kinds of creators: • Documentations and videos tutorial with project planning canvas: Applying the ML tools in creative practices • Images and text of imaginations of future scenario with imagining canvas: Imagining different future scenario of applying AI/ML in creative processes • Questions with reflecting canvas: Reflecting the future scenario of AI/ML Stage 2 Building collaborative future scenarios of a better relationship between human and machines /tools (AI): From the online platform, the community will be formed, creators will have "AI creative literacy", and the future scenarios of a better relationship between human and machines/tools (AI) will be built.

Stage 3 Make the scenarios come true in the future: The better relationship between human and machines/tools (AI) from the online platform which are discussed by the community can be developed/created in the future.

133

Chapter 9

9.2 Looking forward This section would be a reflection on the scalability of the project to other contexts and possible professional, and a development plan for after this year of the master’s programme.

Outlining strategies and trying to reach interconnected global scales from Hyper-Local (to Hyper-Global).

9.2.1

Reflection on the scalability of the project to other contexts

There are three elements in the spread and scalability of initiatives which are "elements of the scaling and spreading process", "elements of the target socio-technical context", and "intrinsic elements of a given initiative". In figure 9.2 indicates the composition of components with each element.

AI tools collection

Online courses Workshops New tech art movement

Tutorials

Generative design Community & Champions

AI creativity literacy

Tutorials

Toolkits

Knowledge Sharing & Transfer Resources

Social media (IG, FB, Youtube...)

Narratives & Communication Creative computing in art and design education (STEAM)

Elements of the scaling & spreading process

Toolkits

Ease of Use and Understand

Openness

New kinds of creative process

AI Literacy

Alignment of Matter of Concern

Augmenting Creativity intrinsic Elements of a given initiative

Knowledge

Online platform (website)

Elements of the target socio-technical context

AI Creativity

Social media (IG, FB, Youtube...)

Copyright for AI-Created Visual Artwork

Alignment of Social Values

Proof of Value Creations from people Responses from people in social media and online platform

Creativity will be augmented

Ownership

Alignment of Legal Norm

Toolkits & AI tools Online platform (website)

New types of expression

Unfold the AI black box Never be limied by new technology

Figure 9.2: Spread and scalability of initiatives (using the framework from Mara)

134

Chapter 9

Elements of the scaling and spreading process

Community and champions There will be a community that provides AI creative literacy and all the creators have their AI creative literacy in their creative sectors. For instance, in the product design sector, the generative design will become a main way of design. Human designers will have the capacity of communicating with generative AI which will be an important role in the design process. On the other hand, collaborating with AI will be a normal thing for artists, not only in the creative process but also in the expression, and there will be a new tech art movement called "AI collaboration revolution". AI will be not just a tool, but also a collaborator with artists and a new form of creation will appear.

Knowledge sharing and transfer resources Knowledge sharing and transfer resources would go through the resources putting in the online platform (website) which are "tutorials", "toolkits", and "AI tools collection". Also throughout social media that can share the ideas and information. In addition, the offering of workshops and online courses are ways to share knowledge and transfer resources.

Narratives and communication The way for the project to put the narratives and communicate with others would through the online platform (website) and social media.

Elements of the target socio-technical context

Alignment of matter of concern The project was around the matter of "creative computing education", "AI literacy", and "AI creativity". Aiming to provide a methodology and tools for creators to explore AI in creative practices, and create ‘new kinds of the creative process’ and ‘new types of expression’.

Alignment of legal norm The legal norm of the project relating is "ownership" and "copyright for AI-created visual artwork". When AI can generate an infinity of images, who is the one that owns the images? Also, taking other’s works as a training dataset to generate new works, will it be a new kind of copyright problem here?

Alignment of social values The project provides the tools and structure to learning AI in creative practices that help people unfolding the black box of AI. Creators will never be limited by new technology, and creativity will be augmented by understanding and using AI. 135

Chapter 9

Intrinsic elements of a given initiative

Ease of use and understand The toolkits and the tutorials for guiding people to build, imagine, and reflect on AI creative projects should be the things that are easy to use and understand.

Openness The online platform (website) and social media should be open for everyone to access including the knowledge, toolkit, and AI tools on the website.

Proof of Value The proof of value from the project can be found in social media and courses or workshops.

The responses from people in the social media responding to the posts or sharing the ideas there, and creations from people applying the ML tools as well.

9.2.2 Development plan for after the masters programme The plan for after the master programme shows as figure 9.3 is starting from the point of the master’s project - Augmenting Creativity. Then at the time, the project is split into four aspects that I will have multiple futures based on my areas of interest and the master’s project. The four aspects are “education”, “human-machine interaction and physical computing”, “product/system development”, and “tech-art”, but all of them are concerned about the similar issue which is around the relationship between humans and machines/technology. I think I was always interested in finding ways to collaborating with technology and creating values out of technology. The four aspects seem to have their own path, however, I think they are not the four parallel lines. There will be different intersections and combinations with each other in some points.

136

Chapter 9

MDEF project

Reflecting on the relationship with Creating a better relationship

}

Augmenting Creativity

technology machine

}

Future

Now Education

Human-Machine Interaction & Physical comuting

Product/system development

Tech-Art

Build the website for learning and creativity (Machine Learning for Creative practice)

Research the field of human-machine interaction and physical computing

Interactive design project

Creative computing

Enhancing UI/UX skills

AI-related creations

Develop the AI-assistant tools for creative practices

The creations for reflecting on the relationship with technology

Build the social media and create the community AI education in art and design

Keep making and building electronic stuff Enhancing coding skills Exploring human-machine relationship

Open online courses about AI for creative practices

Collaborate with artists, designers, and engineers

Exihibition to experess the ideas Artist-in-resifence

Figure 9.3: Development plan for after the masters programme

137

Chapter 10 Final Reflection

Chapter 10

10. Final Reflection The whole process of the project was changing all the time. But in the overview point, it’s always looking for finding out the relationship between humans and tools – machine and technology in a creating aspect, figuring out the meaning of the tools for creating, and trying to create the new values out of the tools. Tools can help us fulfilling the works easily, efficiently, or conveniently, and they seem to have their functions to solve the specific problem or do a specific task. I think tools are also playing an important role in creative practices for the creators, because of the help of the tools that we can achieve our ideas easily. The role of the tool for human seem to just play its functional and technical job that helps human to complete the works or achieve the ideas, and human is the idea-maker. However, what if there is a tool that is assisting us to come out with the ideas, what will be a new relationship between us? And What if machines play the role as an idea-maker (generating different results) in a creative process, how can human creators shift the role to become a creative director, and have the ability to communicate with the machines?

In the process of the project, I started exploring different machine learning tools, from image and sound recognition to image generative algorithms. It’s my first time getting in touch with AI and machine learning, and I was totally fascinated by it for no reason. I found that how powerful machine learning is that everyone can train their own models which are meaningful for them. And there already been many open source tools, software that people can actually use them. Because I was coming from the art and design background which is more working on designing the real stuff or creating a personal art piece, I tried to think about how to implement these tools in creative practices, how can they make creation become different than before, and trying to find out what are their differences with other tools. After trying out the machine learning tools, I found that with machine learning, the creative process and expression could be changed. And there are so many possibilities that apply AI and machine learning in creative practices.

In the middle of the process, the project also combined the educational aspect of it. As I explored and see many possibilities of applying AI and machine learning in creative practices, but most of the people from creative sectors have nothing to know about it that just like me at the beginning. However, I think that using machine learning in creative practices will be the future because we are in the age of everything becomes smarter and smarter and tools will also become more “creative”. Thus, I started to connect teachers, art and design students in 140

Chapter 10

my previous school to discuss the ideas and make a course for the students. When I was preparing the course for the art and design students, I was coming out the questions about how to teach the people who don’t have the background of computer-science to understand how machine learning works and how to provide them the practical applications of machine learning that they can apply in their works. I was taking so much time to prepare before teaching machine learning to others because I didn’t have much confidence as I also felt that there are still many things I don’t know. However, from the experience of teaching the students from art and design school, I also learned some ideas from them. It makes me feel that in the educational environment, teachers maybe can also learn with students, so that what is the thing that teachers can provide or lead, and how to teach the unknown.

In the end, in order to benefit and connect more people, I built an online learning platform which is a website for providing machine learning knowledge, tutorials, and digital learning kits to creators to learn the machine learning tools in creative practices. I was coming out of the idea of “AI creative literacy” which is the fundamental knowledge about AI in creative practice that the creator should know, and the platform would also like to follow the idea.

So the questions here I was thinking are: what is a proper way for creators to learn about AI and machine learning, and how to start exploring the world of creative AI and applying machine learning in creative practice? To respond the questions, the five parts of the learning path which are experiencing (the existing examples), learning (basic knowledge), applying (in practical with tutorials and toolkits), imagining (possible futures of the applications), and finally reflecting (on that future) are the proposal. I was also thinking about not just providing the digital materials, but also can have physical toolkits (figure 10.1) with Arduino and camera module for people can build up their own image recognition devices to integrate physical computing skills and have more tangible experiences.

That is what I am now. The project at the beginning I was thinking to build a kind of product things to now be building up a methodology or a system for learning technology tools. The program is ending, but I don’t think the project was not ending up here. I feel that it’s more like a beginning because there are so many possibilities that I still can explore but I haven’t.

The whole journey of the master project is what I never experienced before (although it’s my first master). It’s so different from my previous studying and learning experience. I was so confused about the design intervention at the beginning and I didn’t really know how to make development plans, organizing things, and engaging people, etc. But now, although I cannot say that I’m good at these things, I learned from the whole process of the project was building 141

Chapter 10

up from nothing. In the end, the project was still having a lot of space to improve, but I would like to believe that all the actions and works I did have values and leads me to see the different futures.

Figure 10.1: The image of the toolkit of image recognition and the guide book

142

Chapter 10

143

References

References

References Adobe Follow. (2018, October 10). Creativity & Technology In The Age Of AI. Retrieved June 24, 2020, from https://www.slideshare.net/adobe/creativity-technology-in-the-age-of-ai-119007383 AI Art Generators: How to Make AI Art (2020 GUIDE). (n.d.). Retrieved June 24, 2020, from https://aiartists.org/ai-generated-art-tools AI Experiments | Experiments with Google. (n.d.). Retrieved June 24, 2020, from https://experiments.withgoogle.com/collection/ai Boden, M. A. (2004). The Creative Mind. doi:10.4324/9780203508527 Conti, M. (2016, April). The incredible inventions of intuitive AI [Video file]. Retrieved from https://www.ted.com/talks/maurice_conti_the_incredible_inventions_of_intuitive_ai Fiebrink, R. (2019). Machine Learning Education for Artists, Musicians, and Other Creative Practitioners. ACM Transactions on Computing Education, 19(4), 1-32. doi:10.1145/3294008 Interactive Demos from NVIDIA Research. (2019). Retrieved June 24, 2020, from https:// www.nvidia.com/en-us/research/ai-playground/ Ml5js·Friendly Machine Learning For The Web. (2018). Retrieved June 24, 2020, from https:// ml5js.org/ Project Dreamcatcher. (2016). Retrieved June 24, 2020, from https://autodeskresearch.com/ projects/dreamcatcher Ramón López de Mántaras Artificial Intelligence Research Institute (IIIA), Mántaras, R., & (IIIA), A. (n.d.). Artificial Intelligence and the Arts: Toward Computational Creativity.

Retrieved June 24, 2020, from https://www.bbvaopenmind.com/en/articles/artificial-intelligence-and-the-arts-toward-computational-creativity/

146

References

Schwab, K. (2019, February 07). See a chair co-designed by AI. Retrieved June 24, 2020, from https://www.fastcompany.com/90297220/see-a-chair-co-designed-by-ai Vicente, J. L., & Quintero, M. (2019). Atlas of Weak Signals – MDEF 2019-2020. Retrieved June 24, 2020, from https://mdef.gitlab.io/mdef-2019/term2/S03.html Visnjic, W., CreativeApplications.Net, F., Visnjic, F., & CreativeApplications.Net, E. (2018).

Learning To See – Making deep neural network predictions on live camera input. Retrieved June 24, 2020, from https://www.creativeapplications.net/openframeworks/learning-tosee-making-deep-neural-network-predictions-on-live-camera-input/ What is Machine Learning? A definition. (2020, May 29). Retrieved June 24, 2020, from https://expertsystem.com/machine-learning-definition/

147

Augmenting Creativity June 2020