Extended Interaction Space

Page 1

Extended Interaction Space

Felix Schwamkrug

Research questions:

¬ Why do we need to change the current way of interacting with the digital world, in the particular case of working from home and which situations would benefit the most from a more refined interaction?

¬ Is the incorporation of gamification elements capable of improving the adaptation or the overall experience of a service?

¬ How can a remote workers’ home be transformed into an interactive space where rooms can be altered to fit new purposes as well as perceiving the presence and actions of coworkers, all using a universal solution?

Extended Interaction Space

Examining working models to find a solution to achieve a more sociable and effective workday

Felix Schwamkrug

Superviced by:

Prof. Carola Zwick

Prof. Judith Glaser

Prof. Dr. Patricia Ribault

Master of Arts in Product Design

Kunsthochschule Berlin Weißensee April 2022

77 Concept definition. 78 The decision-making process laid out. 80 Expert interviews. 82 A Day In A Life. 83 The Takeaway. 85 The detailed chart of daily smartphone use. 87 Comparing the methods of digital interaction. 91 Inputs & Outputs. 93 Daily situations sorted by working models. 95 The conditions that emerged. 100 The interaction mechanics. 103 List of hardware requirements. 104 Moodboard. 105 Design notes. 106 Design Process. 108 1:1 scale model. 109 Working model. 111 Information on screen. 112 Interaction in HOMELY OFFICE broken down. 114 Gamification in HOME LY OFFICE. 118 The three main modes. 120 A day of work at home with HOME LY OFFICE. 128 Dock explained. 132 Interface. 136 Discussion. 137 Research questions revisited. Research & Theory 3 Research questions. 6 Contents. 8 Preface. 10 The status quo. 14 Introduction. 16 Preliminary problems. 18 Work. 20 Working from home. 22 COVID–19. 28 Working together. 30 Trends for basic needs. 32 Communication. 34 Motivation. 36 Gamification. 40 Flow. 44 Psychology. 48 Smartphones. 50 Screens. 52 Interacting with tech. 56 Sensors. 62 Wearables. 64 Computing and intelligence. 66 APIs. 68 Timeline of Smartphone Milestones. 70 Connecting the dots.
Concept Development

Preface.

Identifying problems

I’ve always been someone who either finds technological solutions incredibly inspiring or sees so many problems and opportunities for improvement that I cannot stop myself from thinking up new ones. Most of the time, however, I am both of those people at the same time, trying to figure out what I find so brilliant about these things and applying them to other areas and issues. I grew up in a designers environment, surrounded by timeless and practical things. School did not help me with creative expression or critical thinking, my studies at the University of Wuppertal, internships, workshops, and sponsored projects were all the more rewarding as a result. I eventually followed in the footsteps of an industrial designer, using my own drive and enthusiasm to improve the world of products and services.

From then on, resolving each problems by applying logical processes, I found myself having to take a step back again by getting new impressions, meeting people and starting new studies - to get away from problem solving and questioning established practices and situations and understanding shifts in society, technology and resources.

This interest in many technological and engineering principles, trend-setting technological developments, sociology, and why

2 Antonio Damasio. Descartes’ Error: Emotion, Reason, and the Human Brain, 1994

3 ericssoners.wordpress.com, GS88 ‘Penelope’: Smartphone Concept by Ericsson, received 17.02.22

4 Michael Polanyi. Personal Knowledge: Towards a Post-Critical Philosophy, 1958

people do things the way they do in recent years, combined with education that enabled me to use this tacit knowledge (4) in a reproducible and comprehensible process of ideation, product development, and solutions. Having to make a difficult decision with numerous ties to other aspects would be impossible or extremely slow without this implicit knowing emerging from the subconscious library of general deductive knowledge, past experiences (inductive knowledge), and biases (abductive inference). This means that if a paragraph contains information that is not common knowledge, it will be marked with a number and a footnote linking to the source – or it is based on my differentiated experiences with the subject.

In my research on how to use technology to ease the transition from work to retirement, I’ve spent a lot of time looking at different generations and their use of technology. It was the research into the interface between man and its digital world that finally led me to focus on the smartphone, as it precisely represents the state of the art for that purpose from a technical point of view. In face, it is so high-tech, some of it is capabilities, sensors and data gathering possibilities never get utilized in a lifetime of such a device — more on that later.

I learned a lot in the past months about interpersonal and interaction possibilities, artificial intelligence possibilities; about hardware currently accessible to us and technologies changing our future; about the various hybrid and non-hybrid work models, practices and sociological changes that are taking place partly slowly in the background, partly right before our eyes. Then I formulated preliminary questions about work, technology, and communications, compared contemporary and common work models, their benefits and drawbacks, and began to carry out further research into each topic that emerged on the way to my solution to some of the findings.

I will begin with a prologue of scenarios from the various work models currently in use, and then transition into the research with the preliminary questions adding to the main research questions. Subsequently of the research, there will be a conclusion where all dots are connected to create a profound structure for creating a concept resultant of that analysis.

8 felix schwamkrug masterthesis extended interaction space 2021 2022 9 theory and research

The status quo.

The modern workplace – the proven reference model

Working at your own workstation in a room that accommodates people of different disciplines.

But first, you have to get there. For some, the workplace is located in the neighborhood, while for others it is a long daily trip. Employees are provided with good conditions and all the necessary tools for effective work.

The work schedule is regulated; most colleagues start and finish their daily work at a similar time.

There is always the possibility to exchange work-related issues with colleagues. This can be done by phone, chat, or face-to-face conversation with a colleague that starts with a simple hand gesture or by going to the colleague’s workplace. Pinning thoughts on a collaborative white board adds to the spirit, and seeing others work passionately is encouraging.

People meet in the hallway, at the water dispenser, at the coffee machine on the balcony, at the front door, or in the parking lot. These are different settings with different retention times and thus bring diverse contexts to the conversation. These encounters come along the way; the actual focus at the locations is on the main

activity, like getting water from the dispenser, or getting coffee from the machine.

At the workplace, in order to be able to focus more effectively, one can wear headphones with supportive music, let colleagues know that one does not want to be disturbed, and possibly even redirect phone calls to a colleague. People in the vicinity, on the other hand, may interfere with the effectiveness of work – it is hard to get in the zone with people running in your peripheral vision.

Some modern offices offer separate rooms or areas where you can work all on your own.

Lunch breaks are scheduled every day and are spent with colleagues in the cafeteria or nearby lunch restaurants. On these occasions, you get to know each other and meet new people.

The rest of the lunch break, if you are lucky and depending on the office layout, can be spent alone in the recreation area of the office. Napping is rarely an option.

The end of work is time-bound for most of the regular employees, and through a chat, overlapping hobbies are discovered and lead to a joint activity after work.

The end of the workday usually marks the start of personal life, with only the most urgent concerns being addressed through private communication channels.

10 felix schwamkrug masterthesis extended interaction space 2021 2022 11 theory and research

The status quo.

The remote home worker – standard in recent years

Being self-disciplined is the first item on the agenda as soon as you get up. Keeping the daily starting point of the work the same is possibly not checked or needed, especially when self-employed.

Depending on the situation, one may wake up in the office and start the day in the in-house cafeteria – the kitchen.

A short walk in the neighborhood provides you with fresh impressions – or at least, fresh air.

The workplace is in the living room, and for some, it is in a room of their own. It is flexible and can vary, but it needs to be set up for high efficiency. A day at home is usually filled with video conferences, phone calls, and exchanges of files and screenshots. In collaborative work, part of the effort always goes into preparing and sending out work that has been done in the form of presentations or annotated files.

Breaks are taken at times that fit the agenda. Informal communication is possible via phone and chat. If you have already met and exchanged contact information, you can set up a time to talk. However, the time spent on break tends to be used for private conversations with people outside of work.

During downtime, you can recharge without being disturbed. Whether you are alone during this break or other people elsewhere are having loud conversations, you would not notice it anyway. You can take a short nap, engage in a hobby, or do errands at home.

While working, there is very little opportunity to indicate to colleagues that you are busy. A project has to be completed in only a little time, and you do not want to be disturbed in any other way for a specified time. Setting the status to “busy” in the instant messaging platform and replying to calls with a “Can not talk right now” quick response is the usual approach.

In addition to colleagues, you often get in your own way. At home, various entertainment sources are in close proximity, as are other distractions irrelevant to work, which disrupt the flow.

The end of the workday can be as ceremonial as closing the laptop and leaving the apartment, or less than ideal, like closing the work document on the computer and opening something else. In between business hours, personal and work-related matters are often discussed on a variety of channels, which can make it difficult to finish the day.

Comparing two parallel working model status quos.

When comparing these different work models, it shows that both being in the office at a scheduled time and being at home building their own workflows have their own set of characteristics. Differences are apparent, especially in the collaborative and social aspects. While working from home provides a more flexible and calm environment, it requires self-discipline and strong communication skills to achieve the efficiency of a dedicated workplace. Meeting new people and socializing remotely is much harder and less spontaneous than at the office, too.

12 felix schwamkrug masterthesis extended interaction space 2021 2022 13 theory and research
remote working (noun)
the practice of an employee working at their home, or in some other place that is not an organization’s usual place of business.
(Cambridge Dictionary)

Introduction.

It is time to redefine work and life

The way we work has changed. The lines between work and life are blurring. The standard of 9-to-5 is also melting away.

Societal, technological, and administrative shifts will need to take place to make these transitions happen. The groundwork for the new normal had already been laid, albeit involuntarily hastened by a worldwide pandemic. Now there are different types of alternative models, spanning from hybrid office-centric presence requirements with a negotiated Friday working from home to full-time remote employment without ever getting to know your co-workers. There is not one model that fits all and it probably never will be one, but flexible work hours and locations are here to stay, and that is a good thing. Later on, I will get into the ups and downs of that new lifestyle, if imposed or earned, in detail.

How can a remote workers’ home be transformed into an interactive space where rooms can be altered to fit new purposes as well as perceiving the presence and actions of coworkers, all using a universal solution – like the smartphone.

Aside from the changes in work structures, the smartphone as used today has created the greatest shift in technology since the Internet did in the 90s. With this shift came a lot of new possibilities. Both the personal and commercial landscapes have changed dramatically, with things like apps, voice and video calls, and powerful computational power outrunning personal computers. Next to that, there are built-in cameras, which are also more capable than compact cameras, and sensors like microphones,

“In a few years remote work will just be work.”
(Prithwiraj
Bloomberg Business 2022)

geolocation, and several wireless connection transceivers. The latter is sometimes used to make calls. The “phone” part of the smartphone’s name is almost becoming misleading, as it allegedly has its origins in a 1997 Ericsson phone concept that could do some of the things mentioned above. (3) On my journey, I was more interested in the “smart”-part of the smartphone, and you will find the answer later in the thesis. The pseudo-anglicistic German term “handy” is quite fitting and timeless, though it may be a little too general when used in an English context.

Many things became more efficient, pleasant, cheaper and easier with the smartphone. Others got more annoying, more invasive, more stressful more addictive. With every technical upgrade, simplification of high-tech and new applications this device became more of a part in our life.

In the following pages the topics leading up to the final research questions are being outlined and concluded. Complicated information framing this thesis is untangled and the general base of knowledge is being laid out.

Aside from the main questions of why we need to change the current way of interacting with the digital world and which situations would benefit the most from a more refined interaction I am elaborating on other questions too.

What technology helps amplify the benefits of working from home and minimize the downsides?

Is the implementation of elements of gamification able to enhance the adaptation or the entire experience of this service?

14 felix schwamkrug masterthesis extended interaction space 2021 2022 15 theory and research

Preliminary problems and opportunities regarding the status quo to working remotely:

¬ Spending too much time in front of the screen

¬ Excessive amount of distraction

¬ Time-consuming daily planning

¬ Insufficient communication

¬ Solitude when working from home

¬ Lack of separation of work from personal life

¬ Untapped smartphone’s whole potential

¬ Making use of various spaces and behavioral patterns

Three main subjects

Subsequently these are the three major areas I will focus on over the course of the work and the development of the concept.

16 felix schwamkrug masterthesis extended interaction space 2021 2022 17 theory and research
“The office during the day has become the last place people want to be when they really want to get work done.”
(Jason Fried and David Heinemeier Hansson 2013)

Heavy automation is responsible (and is being continued to be blamed) for loss of so many jobs in various industries that were highly dependent upon humans, just a decade ago. In the future as well, automation will continue to rise, and continue to make lives easier for the people, and harder for those looking for jobs that can be automated. (4)

Remotely.

Few people will say “the office” when asked where they go when they really need to get work done. If they say “office”, they will add a qualifier like “super-early in the morning before anyone gets in”, “I stay late at night after everyone’s gone”, or “I work on the weekends.” (1)

They are trying to tell you that they are unable to complete tasks at work. When it comes to getting work done, the office has become the last place people want to be during the day, states Jason Fried in his book dealing with the topic. (1) In fact, offices have turned into a factory of interruptions. A crowded office works like a food processor, chopping up your day into tiny pieces. 15 minutes here, 10 minutes there, 20 minutes here, 5 minutes there. A conference call, a meeting, another

meeting, or some other institutionalized unnecessary interruption fills each segment.

When your workday has been cut into smaller task, it is incredibly difficult to get meaningful work done. (1) Work that is meaningful, creative, or important requires long periods of uninterrupted time. Long stretches of time are hard to come by in today’s workplace.

One of the major benefits of working remotely is the ability to be alone with your thoughts. When you work alone, away from the busy office, you might find your own productive zone.

Working from home, a cafe, or a co-working space is what remote work entails. (2)

Remote workers use digital tools to manage activities, complete projects, and communicate with their team, rather of coming into the office and meeting with coworkers face to face. If you have a laptop and a reliable internet connection, you can work remotely from anywhere.

Employees might work fully remote or partly remote. Partially fully remote employees may be obliged to visit the office at least a few times per month, whereas fully remote employees may not be required to visit the office at all.

Partially remote employees are required to stay

presence (noun)

the fact that someone or something is in a place or the feeling that someone is still in a place although they are not.

(Cambridge Dictionary)

close to the office to make commuting easier. Fully remote employees, on the other hand, can work from anywhere in the world. (1)

People are booking working vacations, moving their remote offices to be near parents they have not seen for more than a year or to tropical locales, and thinking about what the ideal work-life balance should look like. (5)

18 felix schwamkrug masterthesis extended interaction space 2021 2022 19 theory and research
Work. At the office or
somewhere else?
Jason Fried & David Heinemeier Hansson. Remote: O ffi ce Not Required, 2013 2 kissflow.com, Remote Work vs Work From Home vs Flex Work vs Telecommuting: What’s The Di fference?, received 21.02.22 3 William Gant. Remote Work, 2021 4 Mehek Kapoor. uxdesign.cc, The future of UX design is not on a screen, received 12.04.22

Working from home.

Why should people consider

Working from home does not have to be limited to the fully remote working employees. It can also be for people who want to take a few days off from work to focus on a few important tasks. Working from your living room, bedroom, or even a dedicated home office is an ideal solution. You could even set up your laptop on the kitchen table while preparing lunch.

Working from home, in general, represents a significant shift in your normal work pace, which for many people can be beneficial. It is not your typical way of working, but it is also very efficient. (2)

For those who enjoy working from home, it may be a good fit for their long-term goals. Others benefit from it by saving money, time, and stress, which improves their quality of life. Others, on the other hand, find that it allows them to have a job at all.

Many companies are now enabling employees to work remotely one or two days per week, which is a good balance for both the employer and the employee.

Remote work, on the other hand, has its critics. For some of those people, especially those who tend to be more social, remote work sounds like a terrible level of isolation. Working remotely sounds like a recipe for job loss to those who lack discipline or are concerned about how management will perceive remote workers.

Opportunity Cost.

While there is a hidden, insidious cost to wasting time and money driving to and from work, commuting to work can also limit your future success in a variety of ways.

Given approximately half a dozen hours a week of wasted time in a commute, one would be wise to consider what else they could be doing with just the drive time and money spent on commuting.

If you do not have to go to work on a regular basis, you may be able to relocate to a less expensive, more relaxed area. Housing expenses account for a significant amount of practically everyone’s living expenses, so lowering them is immensely useful. (3)

Anyone can easily surpass coworkers who are not able to work remotely if they work an extra five hours a week in a more pleasant setting. If handled properly, this can have a significant impact on the individual.

Of course, during the COVID-19 pandemic, when people were suddenly forced to work from home, workers and management optimized infrastructure for this workflow, significant progress was made. More on that in the next chapter.

Tips for working from home:

¬ Communicate expectations with members of your home.

¬ Make a dedicated (clutter-free) workspace.

¬ Interact with other humans face-to-face.

¬ Take time for “water cooler” chats.

¬ Prepare meals the night before.

¬ Be easy and kind on yourself.

¬ Pick a definitive finish time.

¬ Set and stick to a routine.

¬ Take clear breaks.

¬ Eat and sleep.

(3), (4), (5), (6)

20 felix schwamkrug masterthesis extended interaction space 2021 2022 21 theory and research
2 kissflow.com, Remote Work vs Work From Home vs Flex Work vs Telecommuting: What’s The Di fference?, received 21.02.22 3 William Gant. Remote Work, 2021 4 Erik Devaney. hubspot.com, How to Work From Home, received 12.04.22 5 inc.com, Essential Tips for Working Remotely, received 21.02.22 6 nhs.uk, simple tips to tackle working from home, received 21.02.22 7 net.nz, simple wellbeing tips when working from home, received 21.02.22

COVID–19.

Since the beginning of the COVID-19 pandemic in March 2020 in Germany, many companies allow their employees to work from home. (1) In addition to social contact restrictions, working predominantly from home also reduces social contacts among colleagues. Thus, with increasing duration and intensity of home working, employees may be affected by adverse factors such as the increasing risk of social isolation. (2) Currently, no robust occupational psychology evidence is available in this regard from the COVID-19 pandemic period. Therefore, for the design of a work organization that is as health-promoting as possible and counteracts social isolation of this target group, conclusions can only be drawn from previous research findings on virtual teamwork and teleworking. However, these do not fully represent the current complex situation.

Social isolation in general can be associated with various negative mental health consequences, such as depression and anxiety disorders, for both older and younger people. (3)

However, working in a home office, in addition to some health-promoting aspects such as flexible working hours or increased freedom of action, which can lead to higher job satisfaction, motivation and performance (4), (5), also has disadvantageous factors, such as the increased risk of

social isolation.

Close contact with the manager, information exchange, feedback and trust play a decisive role. (6) The latter forms the basis for successful virtual collaboration and has a greater impact on performance in location-independent teams compared with traditional on-site collaboration.

Deutscher Gewerkschaftsbund. Corona und Homeoffi ce: Wann darf ich, wann muss ich zu Hause bleiben und von dort arbeiten? Fragen und Antworten zum Arbeiten von zu Hause, 2020

2 Tammy D. Allen, Timothy D. Golden & Kristen M. Shockley. Psychological Science in the Public Interest Journal, How e ffective is telecommuting? Assessing the status of our scientific findings, 2015

3 Susanne Röhr, Felix Müller, Franziska Jung, Christian Apfelbacher, Andreas Seidler & Ste ffi G. Riedel-Heller. Psychiatrische Praxis, Georg Thieme Verlag KG, Psychosocial impact of quarantine measures during serious Coronavirus outbreaks: a rapid review, 2020

4 Ravi S. Gajendran & David A. Harrison. Journal of Applied Psychology, The good, the bad, and the unknown about telecommuting: meta-analysis of psychological mediators and individual consequences, 2007

5 Sumita Raghuram & Batia Wiesenfeld. Human Resource Management, Work-nonwork conflict and job stress among virtual workers, Summer/Fall 2004

6 T.A. Bentley, S.T.T. Teo, L. McLeod, F. Tan, R. Bosua & M. Gloet. Applied Ergonomics, The role of organisational support in teleworker wellbeing: a socio-technical systems approach, 2016

Digitization refers to the technical process of converting analog or traditional paper-based tasks or processes to digital form so that computers can help in accessing, storing, and transmitting information. (7) By contrast, digitalization refers to “the sociotechnical process of leveraging digitized products or systems to develop new organizational procedures, business models, or commercial offerings”. (8)

The Covid-19 crisis has opened senior leaders’ minds to the idea of adopting work-from-anywhere for all or part of their workforces. We got our answer during the pandemic lockdowns. We learned that a great many of us do not in fact need to be co-located with colleagues on-site to do our jobs. Individuals, teams, entire workforces, can perform well while being entirely distributed — and they have. However, concerns persist regarding how workfrom-anywhere affects communication, including brainstorming and problem-solving; knowledge sharing; socialization, camaraderie, and mentoring; performance evaluation and compensation; and data security and regulation. (9)

Prithwiraj Choudhury from Harvard business school states that synchronous communication gets increasingly challenging when workers are

separated. Zoom, Skype, Microsoft Teams, and Google Hangouts can benefit individuals who work in the same or comparable time zones, but not those who operate in other time zones. Work-from-anywhere organizations must become accustomed to asynchronous communication, whether through a Slack channel, a customized intracompany portal, or even a shared Google document in which geographically dispersed team members write their questions and comments, trusting that others in different time zones will respond as soon as possible.

7

22 felix schwamkrug masterthesis extended interaction space 2021 2022 23 theory and research
How it changed how we work, …
Jason Bloomberg. Forbes, Digitization, digitalization, and digital transformation: confuse them at your peril, 2018 8 Ted Saarikko, Ulrika H. Westergren, Tomas Blomquist. Digital transformation, Five recommendations for the digitally conscious firm Business Horizons, 2020
Prithwiraj Choudhury. Harvard Business Review 98, Our Work-from-Anywhere Future, (No.6) 2020 accelerated digitalization …
9

Many people were unprepared for the new work situation because COVID–19 swept the globe without warning, leaving them overwhelmed and inefficient. and left many people unprepared in the process.

Working hours.

In our society today, characterized as it is by economic crisis and downsizing, the workload of the employees who have managed to keep their jobs tends to increase. In addition, technological advances have resulted in an increase in non-standard work schedules, including evening, night, and weekend work. (7)

Work-home interference.

Most participants in a study on working at home mentioned that they were struggling with hometo-work interference and work-to-home interference. First, working at home means more interruptions from family, which may negatively influence work effectiveness. Notably, schools had been shut down during the COVID-19 outbreak, working parents, therefore, faced a bigger challenge in balancing work and family roles. (1), (13) In addition, individuals’ work invaded their life domains during the period of working from home. These interferences from work domains could make people feel exhausted. A project manager shared her experience of being “always online”: “I’m basically always online…my supervisors and colleagues may come to me whenever they need something from me, and you have to give immediate response.” The need to be always online affected this project manager’s ability to meet her family obligations. (1)

“Colleagues are less likely to become close friends because their face-to-face interactions are less frequent.”
(Cecily D. Cooper and Nancy B. Kurland 2002)

Procrastination.

Procrastination, defined as the irrational delay of behavior (2), is one of the biggest productivity killers at work. Procrastination is common in the office-based workplace (3) and it can become even worse when people work from home. Most workers are committed to working productively as usual, they are sometimes struggling with self-regulation failure. The studies indicated procrastination as a challenge whilst working from home. (1) They found that participants delayed working on their core tasks via spending time on non-work-related activities during working hours, such as using social media and having long breaks.

Monitoring.

A study done in China of 39 full-time employees who were obliged by their employers to work from home until further notice was done. The participants worked in a variety of sectors and jobs. Ten participants reported that they experienced different forms of monitoring from their supervisors, including daily reports, clocking in/ out via applications such as DingTalk, and being required to have a camera on whilst working.

They also suggest that work-related social support can aid in the reduction of procrastination. Procrastination can be used to relieve stress. (4) In this unique situation, social support is especially crucial since it can operate as a “negativity buffer” (5), assisting workers in coping with stress and focusing on duties. Furthermore, prior research has demonstrated that social support can contribute to increased organizational commitment. (6) As a result, employees who receive greater social support prefer to reciprocate their employers by focusing more on their work.

24 felix schwamkrug masterthesis extended interaction space 2021 2022 25 theory and research

Ineffective communication.

Remote workers rely heavily on information and communications technology (ICTs) to communicate and collaborate with colleagues, supervisors, and clients. Especially during the pandemic, ICT-mediated communications almost become the only option because workers were not able to engage in face-to-face meetings. Twenty-one participants identified that they suffered from low productivity caused by poor communications during this period. Workers experienced lower efficiency in ICT-mediated communications as opposed to face-to-face communications in the office. Online communication supposedly has a time cost.

Loneliness.

Remote working means fewer face-to-face interactions with colleagues and supervisors. Given the restrictions on non-essential social gatherings during the pandemic, people also lost social opportunities to meet their friends or colleagues, which inevitably contributed to the feeling of loneliness. Workers indicated loneliness as a challenge. Although individuals can connect with colleagues via ICTs, conversations with colleagues were more task-focused, which cannot meet her psychological needs for belongingness or relatedness. (1) B. Wang, Y. Liu, J. Qian & S.K. Parker. Applied Psychology, Achieving e ffective remote working during the COVID-19 pandemic: A work design perspective, 2021

2 P. Steel. Psychological Bulletin, The nature of procrastination: A meta-analytic and theoretical review of quintessential self-regulatory failure, 2007

3 J. Kühnel, R. Bledow & N. Feuerhahn. Journal of Organizational Behavior, When do you procrastinate? Sleep quality and social sleep lag jointly predict self-regulatory failure at work, 2016

4 Jennifer A. A. Lavoie & Timothy A. Pychyl. Social Science Computer Review, Cyberslacking and the procrastination super-highway, 2001

5 Yuen Lam Bavik, Jason D. Shaw & Xiao-Hua (Frank) Wang. Academy of Management Annals, Social support: Multidisciplinary review, synthesis, and future agenda, 2020

6 V. Rousseau & C. Aubé. Journal of Social Psychology, Social support at work and affective commitment to the organization: The moderating e ffect of job resource adequacy and ambient conditions, 2010

7 M. Härmä. Scandinavian Journal of Work, Environment & Health, Workhours in relation to work stress, recovery and health, 2006

26 felix schwamkrug masterthesis extended interaction space 2021 2022 27 theory and research

Working together.

when being apart

People clearly require socialization, particularly in the monotonous environment of working from home. This deficiency is clearly visible in today’s organizational structures, according to studies (1). Some technology is already available, and the COVID-19 pandemic has made great strides in addressing this problem. Many business meetings are now held using video conferencing services like Zoom, Microsoft Teams, Google Meet, and others. Some of them even host digital breakout sessions and even Friday sundowner meetings.

Digital substitutes.

Communication encompasses not only verbal communication, such as spoken words, but also nonverbal communication, such as facial expressions, contexts, and body language, all of which are difficult to convey via digital means. Later in research, I will go into deeper levels of this topic. I’ve already discussed the “Clubhouse” platform, which provides people with a stage, shares this communication space with listeners and interlocutors, and thus provides context to the

topics of conversation. This is important in any social setting to help frame conversations - for both listeners and speakers. A dditionally this kind of low-threshold connection of diverse individuals of life while making the talks relevant was very creative for that time.

There are also a few startups working on solutions to bring social office life home, such as “Kosy O ffice,” “Twine for Zoom” and “SpacialChat”. Virtual interfaces, digital spaces, customizable avatars and proximity chat are all part of the solution. This means that you move around in a digital space as a digital character that is personalized to the user’s cosmetic preferences, meet other coworkers, and can communicate with them when you are in virtual proximity. That sounds like something out of a video game. That is because all of these elements come from popular games that have enabled communication and socialization among players for years – particularly during the pandemic, when more people got into casual gaming and missed out on socialization.

2

dos Santos. ncbi.nlm.nih.gov, Association between COVID-19 and anxiety during social isolation: A systematic review, 2021, received 12.04.22

Human collaboration.

Working on a particular subject is also greatly benefited by having all participants in the same room. What is it about collaborative work that makes it so powerful and enjoyable that it is so difficult to recreate when teammates are working in different locations? It is all about the human element. This human element is what is lacking in remote cooperation and what brings us together when we work in a physical place. It is a type of energy felt when people are working together passionately, and it is highlighted in studies and results from workshops.

Other people can, of course, be disruptive, particularly in the workplace. People are sometimes under pressure, feel insecure, and certain structures even encourage competitive behavior amongst members. Working from home helps you push this unpleasant aspect to the background. Furthermore, people lose some of their ability to engage with others, or they develop communication concerns and even fears. (2)

However, I will focus more on the positive elements of working together, since this should be the norm and the ideal environment.

28 felix schwamkrug masterthesis extended interaction space 2021 2022 29 theory and research
1 B. Wang, Y. Liu, J. Qian & S.K. Parker. Applied Psychology, Achieving e ffective remote working during the COVID-19 pandemic: A work design perspective, 2021 E. R.

Trends for basic needs.

New tricks for old matters

People no longer need to leave their homes to communicate with friends or meet new people. Traditional motivation methods that rely on faceto-face meetings are being impacted by this type of social connection. Weight Watchers’ success, for example, can be attributed in part to their online meetings. Getting support from a group of people who are going through the same change is inexpensive and simple thanks to social networking sites like Facebook. (3)

This app delivers a welcomed break from monotony, boredom and screen time with the power of voice. (1)

The app creates a sense of intimacy that is similar to sitting next to someone and having a chat - it is the human touch that social media has been missing. Members can listen in real time, participate in a chat, and register to speak in front of potentially thousands of people. However, for the time being, the hype has declined.

Podcasts & Clubhouse.

A podcast is the streaming age’s answer to radio. You can listen in on your preferred people’s conversations whenever you want. Since both forms are audio-only, Clubhouse has been compared to a live podcast. However, there is one key distinction between the app and podcasts: spontaneity. Podcasts need preparation and planning. Clubhouse is a spontaneous gathering. Members can go from room to room, and those who want to stay in one club or room for an extended period of time are nevertheless exposed to dynamic change as subjects alter and listeners, participants, and presenters arrive and depart. Because events are not recorded, you either heard it or did not.

Valerie Lalonde. deptagency.com, received 15.03.22

2 mayoclinic.org, Napping: Do’s and don’ts for healthy adults, 2020, received 28.02.22

3 William Gant. Remote Work, 2021

4 bettersleep.org, Sleeping on the job, August 2021, received 20.03.22

5 insidehook.com, The Pandemic-Induced Rise of the Secret Workday Power Nap, September 2021, received 20.03.22

6 sleepadvisor.org, How Long Should It Take To Fall Asleep? –Tips and Advices, 2020, received 20.03.22

7 Junhong Zhou, et al. nlm.nih.gov, Pink noise: e ffect on complexity synchronization of brain activity and sleep consolidation, 2012

8 Nelly A. Papalambros, et al. frontiersin.org Acoustic Enhancement of Sleep Slow Oscillations and Concomitant Memory Improvement in Older Adults, 2001

Naps.

Sleep deprivation affects perspicacity, patience, and creativity.

Younger workers are more likely to take a midday nap. Millennials are 2.5 times more likely than Baby Boomers to nap, while Generation Z is 3.4 times more likely. It is also worth noting that Better Sleep’s poll is relatively new. (4) These naps are not mischievous attempts to shirk their duties — they are pre-planned blocks of time intended to find some restoration in the middle of the day. The NeuroLeadership Institute contends we can only produce six hours of legtimitate “work focus” a week. (5)

People typically take upwards of seven minutes to fall asleep, and this is in their own beds, in the dark, at the end of the day. (6) What if you’ve convinced yourself that you need a power nap to finish an afternoon assignment, but you cannot sleep?

Surprisingly, a good nap necessitates a less-thanideal sleeping arrangement because the ideal power nap, according to studies, is between 20 and 30 minutes, and sleeping on the desk or the couch prevents deep sleep. The longer the nap, the more likely it is that individuals will wake up tired and lethargic. Young

adults, on the other hand, may be able to tolerate longer naps. Naps should be taken early in the afternoon. Napping after 3 p.m. can disrupt nighttime sleep. Individual factors such as sleep need, sleeping schedule, age, and medication use all influence the best time of day to nap. The atmosphere should be relaxing. It is preferable to nap in a quiet, dark place with a comfortable room temperature and few distractions. (2)

Many people enjoy facilitating “sleeping aid” noises. White noise and pink noise have been shown to have positive effects. Researchers discovered that constant pink noise reduces brain waves and that there is a positive link between pink noise and deep sleep, which enhances stable sleep in a 2012 study published in the Journal of Theoretical Biology and a 2017 study published in Frontiers in Human Neuroscience. Rain, a constant sound of a highway, or the sound of the ocean are also popular noises to sleep to. (7) (8)

30 felix schwamkrug masterthesis extended interaction space 2021 2022 31 theory and research

Communication.

Understanding the basics gesture (noun)

Any given situation may involve many interacting contexts, including the retrospective context and the emergent context. The retrospective context is everything that comes before a particular behavior that might help understand and interpret that behavior, while the emergent context refers to relevant events that come after the behavior. (1) The meaning of nonverbal cues appear to persist across cultures. (2) That is true for inter-human communication. With artificial assistents and data collection on the rise, we nowadays expect devices to know and understand the context where conversation and interaction take place and interact accordingly.

Communication can therefore divided in verbal, nonverbal and context. I will also ignore other forms of communication for this matter like written or other sensory communication.

Verbal.

Verbal communication, often known as communication, is a type of oral communication in which the message is conveyed using spoken words. I will not go into extensive detail, but in general, the success of verbal communication is determined not just by the speaker’s capability, but also by

a movement that you make with your hands, your head or your face to show a particular meaning.

the receiver’s ability to listen. The effectiveness of communication is determined by how well a human or a device listens to the subject’s matter. (5)

Non-verbal.

Nonverbal behavior, such as facial expressions and body movement, appears to reveal more information about what one is thinking or feeling than words, according to research. (1)

Vocalics:

A person’s speech and voice patterns can provide a lot of information. Nonverbal cues might include tone, pitch, volume, length, intonation, and pace, voice quality, speaking style and speech intelligibility, and accent.

Touch:

The study of touching as a kind of nonverbal communication is known as haptics. Handshakes, holding hands, kissing, back slaps, high-fives, shoulder pats, arm brushing, and other motions like that can be classified as

haptic communication. Each of these sends nonverbal clues about the intentions and emotions of the person who is being touched. They also trigger emotions in the receiver, both pleasant and negative. (7)

Eye contact.

When two individuals gaze into one other’s eyes at the same moment, this is called eye contact. It is a nonverbal type of communication that has a significant impact on social behavior. (7) Sustained eye contact, without the need to look at other people’s eyes, indicates that a person’s attention and interest is focused on the subject in front of them.

Body language.

Body language is a broad term that refers to forms of communication that use body movements or gestures in place of or in addition to other forms of communication.

Proxemics.

Is a type of nonverbal communication. Personal space norms differ significantly from those of other nonverbal communication channels such as facial expressions. The distance between two people conversing can reveal a lot about their relationship or the context in which they are having a conversation. (6)

Context.

Meanings of certain facial expressions are

universal, it is argued that people derive meaning from facial expressions by relating them to the context in which they occur, including both verbal and nonverbal behaviors. (3)

Recurring criticism is that the terms “verbal” and “nonverbal” are themselves outdated misleading and obsolete and no longer useful. It makes no sense to speak of verbal communication and ‘nonverbal communication because there is only communication. (4)

2 Elfenbein and Ambady, On the universality and cultural specificity of emotion recognition, 2002

3 S. E. Jones & C. Lebaron. Journal of Communication, Research on the Relationship Between Verbal and Nonverbal Communication: Emerging Integrations, 2002

4 A. Kendon. American Journal of Psychology, Review of the book Kinesics and context by Ray Birdwhistell, 1972

5 businessjargons.com, Definition of Verbal Communication, 2019, received 26.03.22

6 umn.edu, Nonverbal Communication in Context, received 26.03.22

7 psychologistanywhereany- time.com, Non-Verbal Communication, 2005, reiceived 26.03.22

32 felix schwamkrug masterthesis extended interaction space 2021 2022 33 theory and research
Mark L. Knapp & John A. Daly. Handbook of interpersonal communication, 2002

Motivation.

Where can you buy it?

Motivation arises from the interaction of three elements: Autonomy, Mastery and Purpose.

Autonomy.

Autonomy is linked to the independence of practical reason and the freedom that reason grants persons to govern themselves independently of obedience to others, including the state. (5) Immanuel Kant’s concept of personal autonomy as rational self-government has, in turn, become increasingly expanded beyond adherence to universal law to include governing yourself according to your “own” authentic desires and impulses. (1)

„In effective gamified solutions, players opt in to participate, and once they do, they make choices about how they will proceed through the challenges to achieve their goals. Players are given the opportunity to discover and learn using different paths through the solution.“ (2)

Autonomy can also be diminished and empowered through the quality of the choices available to us. When we are given more control over important aspects of our lives and access to a diverse range of meaningful choices, then our autonomy is increased. Further, having some degree

of control over how our choices are realized, and not being subject to excessive oversight or control in their pursuit, is also important for our sense of autonomy. (3), (4)

We can increase a person’s autonomy by helping them to make more authentic choices, both in the sense of more choices that are authentic and choices that are more authentic.

Job autonomy.

In a study done in China during COVID–19, thirteen participants suggested the role of job autonomy to be important in remote working. Job autonomy means employees can decide when and how to accomplish their tasks. Individuals’ performance and well-being benefit from job autonomy, as those with higher job autonomy can balance work and rest and choose the most productive ways to do their work. Job autonomy was also identified as beneficial for work-family balance. For example, participant 14 stated: “I can control the rhythms of work and rest. If it is not during the meeting, I can have a short break, around ten to thirty minutes, and then continue to work. That also means more time to spend time with my family.” (6)

“Motivation arises from the interaction of three elements: Autonomy, Mastery and Purpose.”
(Brian

Machine autonomy.

Quick excourse: Machine autonomy can be understood as “the ability of a computer to follow a complex algorithm in response to environmental inputs, independently of real-time human input”. (7) More advanced forms of machine autonomy typically depend upon the use of AI.

Mastery.

„We all have a deep-seated need to improve in aspects of our lives, but often we lack the motivation to take the first step.” Brian Burke states ins his book “Gamify: How Gamification Motivates People to Do Extraordinary Things”. (2)

Gamification offers positive feedback and simple onboarding, which can inspire users to improve their performance within that area. However, mastery is a journey, not a destination. There are several progress markers along the journey, but there is never a finish line. There is always another level in almost all of life’s endeavors, whether it is running, painting, or learning a language. (2)

Purpose.

A purpose is the reason why work is done or why a specific goal is intended to be reached. Gamified solutions, by definition, differ from typical games in terms of their goal. Gamification is used to achieve one or more of three goals: changing habits, increasing skills, or stimulating innovation. Gamification must begin and end with a goal in mind: to help players achieve meaningful objectives. (8)

2

3

4 R. M. Ryan & E. L. Deci. Self-determination theory: Basic Psychological Needs in Motivation Development and Wellness, 2017

5 J. B. Schneewind. The Invention of Autonomy: A History of Modern Moral Philosophy, 1998

6 (B. Wang, Y. Liu, J. Qian & S.K. Parker. Applied Psychology, Achieving e ffective remote working during the COVID-19 pandemic: A work design perspective, 2021)

7 AI assisted ethics Etzioni & Etzioni, 2016

8 Melinda Jacobs. Presentation at Gamification EuropeGamification & UX: The Big Picture, 2019

34 felix schwamkrug masterthesis extended interaction space 2021 2022 35 theory and research
Onora O’Neill. A Question of Trust: The Bbc Reith Lectures, 2002
Burke, Research Vice President at Gartner, 2014)
B. Burke. Gamify: How Gamification Motivates People to Do Extra-ordinary Things, 2014
Richard M. Ryan & Edward L. Deci. Self-Regulation and the Problem of Human Autonomy: Does Psychology Need Choice, Self-Determination, and Will?, 2006

Gamification.

Triggers peoples emotions

Gamification is not a gimmick — it works because it triggers powerful emotions. By using game mechanics in a non-game context, gamification makes tasks more fun and engaging.

The sole purpose of games is to entertain the players. To accomplish this, video games employ elaborate storylines, graphics, and animation to provide players with realistic experiences. The goal is to immerse players in the game world and the role they play within it. Customers with higher value and repeat transactions are rewarded through rewards programs, while employees are rewarded for meeting goals. Airline, hotel, and retailer loyalty programs are the most common, but there are also employee incentive programs and other types.

At the moment, there are dozens of gamified applications available to change personal behaviors, but they are often simple reminder systems with some points and badges slapped on. In order to reinforce motivation with social acknowledgment, more successful systems integrate with social networks. Gamified apps to help people lose weight, quit smoking, improve fitness, correct posture, manage personal money, take medication, and boost memory are just a few examples.

Gamification is not about having fun, despite what it may appear. Many people assume that by adding points and badges to activities, we can make them more fun, just like in a game. (2)

Gamification has three primary goals: it changes behaviors, it develops skills, and it drives innovation. People need to be reminded to change their behavior by a trigger until it becomes part of their routine. Breaking down the change into small, manageable steps is the key to achieving larger goals.

Over time, complexity must be built. When something is done repeatedly, it develops into a new habit.

By providing a clear transformation route with simple milestones and encouragement along the way, gamification can play an essential role in implementing change. Goal-setting, triggers, and baby steps are all used in gamification to assist people change their behaviour. With social sharing, players can meet like-minded people and enlist the help of their friends. Gamification encourages people to repeat activities until they become second nature, to keep the process interesting, and to evolve through time.

trigger (noun)

an event or situation, etc. that causes

something to start.

Researchers distinguish intrinsic and extrinsic motivation. With intrinsic motivation, we do things because we find them inherently satisfying and interesting. With extrinsic motivation, we do things to achieve rewards, and that is the problem with rewards. They massively undermine intrinsic motivation. Numerous studies showing that extrinsic rewardare not sufficient to sustain engagement, and sometimes have the opposite effect. Extrinsic rewards can deliver a short-term boost — just as a jolt of caffeine can keep you cranking a few more hours. But the effect wears off — and worse, can reduce a person’s long-term motivation. (2)

characters exist and the real-world is not affected. Games offer an opportunity to alter elements in this world in a playful way. Games can also help to simplify complex situations.

Gamification is about engaging people on an emotional level and motivating them to achieve their goals. “Pain Squad” is an iPhone app designed to collect daily information about children’s pain levels. The app includes a progression structure, so that when the children complete their pain report three days in a row, they progress from rookie to sergeant and through ranks until they become a chief. (2)

Gamification does not work.

When gamification appears to work, it is because the underlying experience is already a game. (2) So why design a product or service like a game?

Games are extraordinarily successful at creating engagement, triggering states of flow and a decision to design experiences based on a narrative, much like we would design a game.

The magic of games creates a different world where other rules, goals, stories, mechanics, and

36 felix schwamkrug masterthesis extended interaction space 2021 2022 37 theory and research
Educational presentation: Gamification Europe - Gamification & UX: The Big Picture, 2019 2 B. Burke. Gamify: How Gamification Motivates People to Do Extra-ordinary Things, 2014

Elements of Gamification and Game Design.

Game design principles have been translated into other application areas, such as products and services. These gamification elements have a lot in common with game design, but they’ve been adjusted and tailored to these environments.

Robust controls and consistent rules.

Players do not have to think about what buttons to press or how to carry out a particular action when good game design mechanics are in place.

Make the next action obvious.

Knowing what to do next keeps you in the present moment and in a state of Flow.

Experience bars.

These measure our incremental progress and help us visualize how far we’ve come and how far we have to go.

Multiple long- and short-term goals.

These are our main quests and side quests, which are broken down into smaller tasks.

Rewards for effort.

Every bit of effort exerted in a game is rewarded through credit, such as gold, loot and experience.

Immediate, frequent and clear feedback.

This clearly links actions to consequences, which makes learning easier and faster.

The element of uncertainty.

This is the neurological gold mine: we keep going because we want to find out what comes next.

Windows of enhanced attention.

Gameplay helps boost memory and confidence. Game players are more willing to take risks and pursue difficult tasks.

Other people.

Gameplay gives us a chance to connect with multiple players online.

38 felix schwamkrug masterthesis extended interaction space 2021 2022 39 theory and research
(1) Melinda Jacobs. Presentation at Gamification Europe Gamification & UX: The Big Picture, 2019 Working hard in the kitchen made fun with good game design seen in “Overcooked” Image source: Ghost Town Games

Total concentration and almost obsessive dedication to one subject. It is even possible that one does not notice changes in the environment or other sensual stimuli while in that state. This describes being “in the zone”; also known as “Flow state”.

Because this is such an unusual and counterintuitive state, it is a very fragile mental and physical state. The less exciting or dangerous the subject, the more important factors such as the appropriate amount of background noise, a comfortable temperature, adequate lighting, fulfilled basic needs, stress, and comfort. Humans have not evolved to be engrossed in a task without noticing a saber-toothed tiger approaching from behind. One exception would be if their lives depended on them concentrating on the task at hand, which would not be running away from the tiger.

To many people the experience of extreme focus is not a familiar feeling. For some this manifests as tingling sensations, or pleasant light headedness, or a feeling of being dissociated with your environment. (1)

Sounds can grab our attention, even when we do not want them to. If we can understand and predict this effect (‘auditory salience’), we can make

better Focus music by ensuring that distracting moments do not appear. (2)

After testing the effectiveness of Brain.fm, it reminds me of the Shepard tone. Instead of creating the auditory illusion of a tone that seems to constantly ascend or descend in pitch but never gets any higher or lower, it appears to be playing an ever-evolving music track that never changes and (almost never) ends. It worked remarkably well for me.

Pomodoro Technique.

The Pomodoro Technique is a widely known time management method that encourages people to work with rather than against the clock. You divide your workday into 25-minute pieces interrupted by five-minute intervals using this strategy. Pomodoros are the name for these intervals. You take a longer rest of about 15 to 20 minutes after approximately four Pomodoros. (3) The strategy works by instilling a sense of urgency through the use of a timer. Instead of feeling like you have endless time in the workday to get things done and then waste those crucial work hours on distractions, you know you only have 25 minutes to make as much progress on a task as possible. (4)

to be in the zone (phrase)
Being completely unaware of what’s going on around you as you are so extremely into what’s going on right in front of your face.
(Urban
40 felix schwamkrug masterthesis extended interaction space 2021 2022 41 theory and research
Flow. Getting work done
Dictionary) Brain.fm, How to use Brain.fm for Focus 2 Brain.fm, Theory & Process (whitepaper) 3 francescocirillo.com, The Pomodoro Technique, received 24.03.22 4 Kat Boogaard. themuse.com, Take It From Someone Who Hates Productivity Hacks - the Pomodoro Technique Actually Works, received 06.04.22 flowchannel high challenge low low skill high anxiety boredom ai Flow is when you are fully immersed and everything feels e ff ortless. A flow state is the sweet spot between being challenged and being competent.

Characteristics of a flow state:

¬ Intense and focuses concentration on the present

¬ Merging of action and awareness

¬ Loss of reflective self-consciousness

¬ Sense of personal control

¬ Distortion f temporal experience

Conditions for flow:

¬ Knowing what to do next

¬ Knowing how to do it

¬ Freedom from distractions

¬ Clear and immediate feedback

¬ High perceived challenges and high perceived skills

Many of the previously discussed items must be fulfilled in order to achieve this “flow start” and to be “in the zone”. Elements or conditions for a good flow state coincide with elements from the field of gaming design defining how to make games entertaining and immersive. The balance between skill and challenge is essential so that neither boredom nor overwhelming demands arise. The ultimate killer of the flow state is distraction. As a result, these must be avoided at all costs. Noises, distractions, as well as other parallel responsibilities obstruct the flow.

A selection of apps and functions that can help you work in a flow state.

Awork combines time tracking and invoice creation with a calendar and

42 felix schwamkrug masterthesis extended interaction space 2021 2022 43 theory and research
Blocking distracting apps and websites with Freedom, Freedom’s Blocked Page Image source: Own screenshot Planting virtual trees when not using the phone, Forest App Image source: Abby Jewett, The app that helped me write a book, 2019 Apple’s “Do Not Disturb”-mode syncs across devices Image source: Apple task manager for remote workers. Image source: Awork
“Tasty food, sex, exercising and successful social connections can release dopamine.”
(Krach, Paulus, Bodden, & Kircher 2001)

Psychology.

Excessive use of technology

We are becoming true junkies thanks to cellphones and the social media platforms that they promote says Trevor Haynes in an Science in the news article. While it is easy to dismiss this assertion as hyperbole, social media networks like as Facebook, Snapchat, and Instagram exploit the same neurological circuits that slot machines and cocaine use to keep people using their services as much as possible.

While cellphones themselves are not inherently addictive, the hyper-social surroundings they create are the main drivers of our addictions to them. Smartphones allow us to carry vast social environments in our pockets through every waking moment of our life, thanks to apps like Facebook, Snapchat, Instagram, and others. Dopamine is a neurotransmitter produced by our brains that plays a key function in behavior motivation. It is released when we take a bite of tasty food, have sex, exercise, and, most significantly, have successful social connections. It rewards us for favorable behavior and pushes us to repeat them in an evolutionary environment. (1)

When a reaction to a stimulus results in a reward, a process known as long-term potentiation strengthens these linkages. By raising the intensity with which neurons respond to specific

stimuli, this process builds commonly used connections between brain cells called neurons.

Smartphones have given us access to an almost limitless source of positive and negative social stimuli. Whether it is a text message, an Instagram “like,” or a Facebook notification, every notification has the potential to be a positive social stimulus and dopamine flood.

If you’ve been a Facebook user for a few years, you’ve definitely noticed that the site’s notification criteria has been expanding. When you first join Facebook, your notification center is centered on the first set of connections you create, establishing the critical link between notification and social reward. That notification center will grow more active as you use Facebook more and interact with various groups, events, and artists. After a while, you will be able to launch the app whenever you want and expect to get rewarded. When you combine that with the low cost of looking at your phone, you’ve got a pretty good reason to check in as often as you can. (2)

Smartphones and social media apps are here to stay, therefore it is up to us, the users, to decide how much of our time we want to devote to them. (3) Companies like Facebook will continue to do

Denying access to beloved apps will irritate your limbic system but satisfy your prefrontal cortex.

everything they can to keep your eyes riveted to the screen as much as possible unless the adbased profit model changes. They stack the deck — and our brains — against us by utilizing algorithms to exploit our dopamine-driven reward system.

Trevor Haynes continues end the interview by saying that there are a variety of techniques if you want to spend less time on your phone.”

Disabling social networking app notifications and keeping your phone’s display in black and white will diminish your phone’s capacity to capture and maintain your attention. Above all, the best tool you have is to use technology with care. So think about it the next time you pick up your phone to check Facebook: “Is this really worth my time?”

44 felix schwamkrug masterthesis extended interaction space 2021 2022 45 theory and research
S. Krach, F.M. Paulus, M. Bodden & T. Kircher. Frontiers, The Rewarding Nature Of Social Interactions, 2001 2 Trevor Haynes, Science in the news, 2018 3 linkedin.com, Dopamine and Social Media - Unrealised Truth in this Era, 2021, received 06.04.22
rephrased)
(freedom.to,
“If you’re not paying for the product, you are the product.”
(Krach, Paulus, Bodden, & Kircher 2001)

This widely known statement becoming popular in Netflix documentary “The Social Dilemma” from 2020, is not new thinking. Richard Serra and Carlota Fay Schoolman, artists, broadcasted a short video titled “Television Delivers People” as early as 1973, wherein they say, “It is the consumer who is consumed.” and “You are a product of television.” (1) In today’s social media environments, advertisements have evolved from a moving billboard on which you can learn about a product to something more aggressive and inconspicuous.

This is more true today than ever before. “It’s the graual imperceptible change in your behaviour that is the product.” (2), according to Jaron Lanier, a pioneering VR computer scientist, artist, and author. As a result, advertisements became more embedded in entertainment content as well as the products themselves. Millions of dollars have been spent on research into how to get users’ attention and change their behavior, which is then sold to advertisers. Today, they’ve gotten people to enjoy watching ads by making them extremely targeted, entertaining, or bizarre, to the point where they do not even realize they are doing so. The companies that collect personal data about you — social networks and entertainment

platforms — are only as valuable as their ability to predict the user’s next move and their ability of presenting advertiser’s content in an elegant manner.

We were always able to adapt to these new things in our lives that craved our attention before, with print, television, the internet, and smartphones — what is different now is that we cannot keep up with technology that transcends our intelligence and ability to differentiate the content.

There are now apps that block other apps in order to increase productivity. One of the most popular, while also being cross-platform, is Freedom. It is an app that limits Internet access for up to eight hours at a time. The way to get back online is inconvenient enough to act as a deterrent. Fred Stutzman, a Graduate student in information and library science, created the application after his own failsafe self-binding approach was to write in a café without an Internet connection until that location set up free Wi-Fi.

46 felix schwamkrug masterthesis extended interaction space 2021 2022 47 theory and research
Richard Serra “Television Delivers People” (1973), Authors of video: Richard Serra and Carlota Fay Schoolman, uploaded Feb. 2011
Lanier. Netflix, The Social Dilemma, Director: Je ff Orlowski, 2020
2 Jaron
“An average person taps their phone 2617 times a day, unlocks it 150 times per day, and spends up to 3.7 hours on it.”
(Source INC., Research by Dscout 2016)

Smartphones.

The bicycle of our minds

(4)

As already mentioned in the introduction, the modern smartphone is much more than just a phone. In just 10 years, the usage of smartphones and mobile devices has grown to an incredible magnitude. Every month, new and better devices are released, making it difficult to keep up as a consumer as well as a digital designer. Screens can be found almost everywhere, and they help us live more productive lives with more opportunities. Screens, on the other hand, are not the final chapter on this technology-fueled story. Humans have evolved to crave efficiency and convenience more than ever before. We expect things to be delivered immediately, and our patience for communication, commerce, and entertainment has dwindled to almost nothing.

Smart.

So what is so smart about smartphones?

“Smart” is used as a prefix or as a standalone term to claim a functional, classificatory technical property. It is also used as a buzzword, or as a supposedly upvaluing feature. The conclusion of extensive research done by an engineering team in 2018 states that “A smart device is a context-aware electronic device capable of performing autonomous computing and connecting to other devices wire or wirelessly for data exchange”. (1)

According to this definition, a smart device has three main characteristics: context awareness, autonomous computing, and connectivity. The Team’s Silverio explains an article. This definition conformes to the IoT’s main concept, which is that any “thing” can be a part of it. If we add a sensor, a small amount of computing power, and network connectivity to a chair, it can become a smart chair. What about the user’s participation? A smart device can interact with humans directly or indirectly. A weather probe, for example, could collect weather data and send it to the Internet of Things. Of course, humans will make use of the data, but the weather probe did not require any direct human interaction. (2)

smart device (term)

Is a context-aware electronic device capable of performing autonomous computing and connecting to other devices […] for data exchange.

Smart Devices.

The requirement to say a wake-up command that cannot be changed before every command makes using voice assistants unintuitive. Interpersonal communication is almost never structured in this way. Surprisingly, it reminds me of a relatively new way of interacting with people who spend their days wearing Bluetooth headphones in their ears. If you want to talk to them, you will have to do some sort of wake-up command or gesture to get them to put their headphones out. We could say that instead of changing voice assistants to be more natural companions, we changed our behavior to be more like robots. It is a fine line between how long voice assistants wait after you stop speaking — too short, and the rest of the question is not relayed; too long, and you have to wait unnecessarily for the answer.

4

2

48 felix schwamkrug masterthesis extended interaction space 2021 2022 49 theory and research
M. Silverio-Fernández, S. Renukappa & S. Suresh. What Is a Smart Device? - a Conceptualisation Within the Paradigm Of the Internet Of Things - Visualization In Engineering, 2018
M. Silverio-Fernández. towardsdatascience.com, What Is a Smart Device? — The Key Concept Of the Internet Of Things, 2021, received 12.04.22
M. Silverio-Fernández, S. Renukappa & S. Suresh. What Is a Smart Device? - a Conceptualisation Within the Paradigm Of the Internet Of Things - Visualization In Engineering, 2018
3
based
Steve Jobs. interview with WGBH, 1990
on:
(What Is a Smart Device?
(3))
“If people don’t understand the context for the work they’re doing, the work loses some meaning.”
(Lawrence Coburn, Co-founder and CEO of Twine)

Screens.

From new to nuisance

The screens of our mobile devices eventually have become another roadblock in our never-ending quest for efficiency. (1) When you are always in a hurry, taking the extra step of tapping an icon on your phone and then continuing with multiple processes to get to the end result feels daunting. The days of having to wait for letters to come in the mail, for your crush to call you on your corded home phone, and for dial-up Internet to manifest are gone. We want it right now, and new technology can make it happen. When it comes to creating digital experiences, context is crucial. We investigate and investigate the role of technology in the user’s daily life, as well as the impact of the user’s environment on our digital designs. We must now think beyond the digital. A broad environmental context is required for modern strategies. (2)

However, screens have a place today. Laptops, tablets, monitors, TVs, and specialized devices such as digital whiteboards are becoming more popular — as well as larger in size. The screen quality is the most important aspect of the hardware setup if you want to display a lot of information and media such as video, photography, graphics, and print. It makes little difference whether it is a touchscreen or not for this purpose. Apple is blurring the lines between its iPad and

MacBook product lines at the moment. Some configurations are so similar that the main interaction methods for manipulating content within the screens are the most distinguishing features. iPads have touchscreens, whereas Macbooks have keyboards and trackpads.

Zero UI.

The term Zero UI refers to a paradigm in which our movements, voices, glances, and even thoughts can all cause systems to respond to us through our surroundings. At its most extreme, Zero UI refers to a screen-free, invisible user interface in which natural gestures initiate interactions, as if the user were conversing with another person. (4), (5)

There are several trends in the technological world moving away from screens as the main form of interaction all together. Zero UI removes the traditional graphic user interface (GUI) from the equation and uses nature interactions of haptic feedback, context aware, ambient, gestures and voice recognition.

What is displayed on the screen is still important, but designing for the stages leading up to

or outside of a screen is just as important. A task can be started with a facial expression, a wave of the hand, or a bodily gesture. Saying “Alexa”, “Hey Siri” or “OK Google” activates a fresh set of instructions. Telemetry sensors are used in smart homes to turn on lights when you come into a room and turn them off when you depart. New automobiles will open the door and start the car while your keys are out of sight, and then drive down the road without your hands on the steering wheel. (2)

The idea of controlling electronic devices by pure thought might sound more futuristic than it is actually feasible today. Since we live in a time where enough data is collected about us in the form of preferences, likes, behavioral patterns, retention times, and histories while consuming content, the back-end systems responsible for serving content are already capable of knowing us well enough, even if not our exact thoughts.

Further, why press a button at all when you can simply call out the name of the channel you want to watch? Or, instead of having to speak the channel you want, maybe your TV automatically flips to the Bears game because you watch them play every Sunday.

pattern (noun)

the regular way in which something happens or is done.

50 felix schwamkrug masterthesis extended interaction space 2021 2022 51 theory and research
Creative Honey. Post-Screen Design: The Future of Designers in Tech 2 Jim Combs. How UX-Designers can prepare for a world with no screens, 2017 4 Andy Goodman’s definition, ordnet.com, 2015 5 t-3.com, Thinking how UX can prepare for a world with no screens, received 02.04.22
Dictionary)
(Oxford

Zero UI (term)

Refers

to a paradigm where our movements, voice, glances, and even thoughts

can all cause systems to respond to us through our environment.
(Andy Goodman from Fjordnet) (4)

Interacting with tech

and tech interacting with us.

Many experts and scientists in the fields of interaction design and science differentiate between different kinds of interactions when it comes to controlling and receiving information by technology apart from screens.

By Voice.

Since the early 1970s, speech recognition has been a cornerstone of science fiction, and it has now made its way into the mainstream with the inclusion of voice recognition and search in both iOS and Android phones. Users will emphasize syllables, words, and phrases in the hopes that the system would recognize their questions and requests. Additional user research is required when designing the conversational UI to determine how users will phrase and construct their inquiries or remarks. The necessity to adapt the algorithm to regional dialects and slang is one of the more difficult aspects of voice recognition. When all the many ordering options are examined, something as easy as ordering a pizza can become a difficult design problem. Understanding the user’s desires and motivation, or motive, aids in phrasing. (3)

Experiments and studies of interacting with screens in public spaces has been done and came to the conclusion that despite the fact that users had trouble understanding how the gestures worked, the results show a strong preference for gestures over voice commands. (1) Another study co-pared. (2)

Contextual Awareness.

Context-aware devices and apps that eliminate the need for further interactions improve digital and physical experiences by anticipating user desires. Personalization enables Zero UI devices and apps to be proactive and anticipatory. (3)

Domino’s Zero Click App is a working example of such an interaction in the consumer market. It works on the premise that if you launch the app, you want pizza delivered. It requires some setup and learning about the user’s preferences and dislikes, but by doing so, it is very quick and efficient and without any extra processes. (4)

Another example involving smart hardware are

“Stop trying to make air gestures happen.”

Apples AirPods. Remove one AirPod, and music playback switches to mono. Remove both, and playback stops and resumes when the user puts them back in.

Nest Thermostats, rely on collecting data on usage and interactions to adapt to the user’s anticipated needs. We can build contextual experiences that become implicit rather than explicit interactions by exploiting sensors within a device or positional data. What is the user’s current circumstance or context? What is the device’s application context? It is necessary to develop and construct backdrop and temporal interactions.

Gestures.

Gestures must be simple to learn and repeat. One of the challenges with gestures is getting systems to detect the physicality of the motions in three dimensions (wave, hover, and swipe). The system’s responsiveness to gesture inputs can cause issues if users use various gesture sets in an attempt to get the system to respond. There must be a balance between teaching the gestures and delivering feedback to the user when actions and tasks are completed correctly. In the article on medium “stop trying to make air

52 felix schwamkrug masterthesis extended interaction space 2021 2022 53 theory and research
J.
ettinen. Gesture and speech-based public display for cultural event exploration, in Proceedings of the Tilburg Gesture Research Meeting, 2013
A. E. Sarabadani Tafreshi, A. Soro, & G. Tröster. frontiersin. org, Automatic, Gestural, Voice, Positional, or Cross-Device Interaction? Comparing Interaction Methods To Indicate Topics Of Interest To Public Displays, 2018
R. Dharmik. intuz.com, Zero UI: Get Ready For The Upcoming Screen-Less Future, 2021, received 05.03.22 4 fastcompany.com, What is zero UI and why is it crucial to the future of design?, received 02.04.22
Hakulinen, T.
Heimonen,
M. Turunen, T.
Keskinen
& T. Mi-
2
3
Alpaca, published on Medium, 2019)
(UX

gestures happen” “UX Alpaca” talks about those unintuitive hand gestures you see in movies and automobile media control systems. However, air gestures are not completely useless. She sees a market for it, and as with any technology, they can be useful beyond playing with it once when you set up your new phone. Just like it happened with augmented reality (AR), there are apps where it adds value to the users (like fitting IKEA furniture in your home, or showing the constellations when you point your camera at the sky). Simply because it is technically possible, new technology does not have to be applied to everything.

Google’s Project Soli makes “your hands the only interface you need” by using radar to detect fine movements. (It is a more refined approach to conventional ‘rough’ air gestures because it senses movement very precisely. UI layers and interactions should not overextend the user’s motion away from the body as this can be fatiguing. Make gestures small and natural. Currently, it finds its place in Google Nest products like the Thermostat and Hub. Built in, it senses a human approach and hand gestures. I will concentrate more on Google Soli later on.

When it comes to the tools we use every day, people will always choose the path of least resistance, prioritizing utility and pragmatism over all else — this is something to keep in mind when creating gestures and other novel forms of interaction.

Glanceability & Ambiance.

With no need to open apps or read notifications, ambient devices operate on the principle of glanceability. Like a wall clock or a single-day calendar, a single glance should provide the user with all of the necessary information or context.

The “Nabaztag Rabbit” was a companion style device that combined colored lights and changing positions of the rabbit’s ears to provide glanceable experiences. (3)

The “Chumby”, a smart screen from 2006, used widgets to provide a snapshot of data. Ambient devices are designed to create a seamless transition between physical and digital spaces. These are interconnected, browserless interactions. (3) Both of these devices were short-lived due to their inability to adapt to changing technologies and user information requirements though. (3) New iterations would be the Google Hub or Amazon Echo Show.

Haptics.

Haptics, or kinesthetic communication, is a form of motion or vibration-based feedback the user can feel. In our current technological environment, vibrations are the most widely used haptic feedback. From game controllers to phone notifications and haptic feedback on touch interfaces, there are a variety of ways to experience haptic feedback. The value of that feedback for users is

“The biggest constraint of the next 100 years of computing is the idea of metaphors.”
(Evan Spiegel, CEO of Snapchat)

increased by keeping haptics to a minimum and avoiding overuse.

The Moto-Cross video from Sega included haptic feedback via the controller to enhance the experience of colliding with other players on screen. With the “Rumble Pak” accessory, Nintendo brought haptics to home consoles. When we interact with small touch screens today, we frequently experience haptics. Fitness trackers and smartwatches currently use haptics to provide notifications to the wearer. Haptic feedback is being applied to clothing, and virtual reality (VR) and Microsoft Kinect-style gaming (6) will benefit from ultrasound or ultrahaptics.

The touch of a touchscreen can also represent a kind of haptic input method. If the touch itself, which the touch surface can register, is recognized as an action or reaction to the system, it can even transmit information in this way.

Object manipulation.

Sensors enable technology to detect the movement of non-technological objects. Toniebox, for example, is a user-friendly, wireless, portable audio player that can import audio stories, music, and other content from Tonie audio characters with NFC capabilities. Choose an audio character, or Tonie, and place it on the Toniebox to begin your audio adventure. (4) It takes a fresh approach to storytelling and allows users to interact with the content. (5)

In an interview with The Verge, CEO and founder

of Snapchat, Evan Spiegel, said that it is hard for us to elevate technology and how we interact with it because we are thinking too much about how we have started to technologize it. He had been thinking about digitally replicating the ways we talk in real life, and materialized many of those thoughts in the app.

Besides of the well-known self-destructing messages — he changed the way users call their friends. Instead of allowing you to ring friends for a video chat, as with FaceTime or Skype, Snapchat forces both users to be present inside a chat window before video can begin. The “Hey, want to chat?” text replaces the ring entirely. (7)

54 felix schwamkrug masterthesis extended interaction space 2021 2022 55 theory and research
2 Project Soli. google.com, received 06.04.22 3 R. Dharmik. intuz.com, Zero UI: Get Ready For The Upcoming Screen-Less Future, 2021, received 05.03.22
whichplm.com, Press release tonies® (Boxine), FlexPLM, 2021, received 30.03.22 5 red-dot.org, Toniebox, 2016, received 30.03.22 6 3D camera or LiDar assisted movement detection setups 7 Evan Spiegel. theverge.com, Real talk: the new Snapchat brilliantly mixes video and texting, 2014, received 03.04.22
4

Sensors.

What they are and why they exist

Touchscreen.

The main screen naturally comes first when it comes to glanceability, or the delivery of information that can be grasped quickly. There are also a few models with secondary screens for quick access to information. Some models also have a status LED that flashes and changes colors to indicate the presence of notifications.

A phone’s screen not only detects when it is touched, but it also knows where it has been touched — and, in recent years, it has been able to tell how many fingers are touching and how strongly.

Vibration.

The smartphone has a vibration mechanism, which can be either a motor or an electromagnet, in order to provide haptic feedback on interactions. Simulating different buttons with different amplitudes and frequencies of vibrations, as well as reacting to pressing a virtual two-element on the screen with different strengths, has made progress in the last years.

Wireless networks.

A cellular network or mobile network is a type of wireless connection facilitated by cellular towers.

Bluetooth is a wireless technology that allows devices to exchange information over short distances. (4)

Wi-Fi is similar to Bluetooth in that it also uses radio waves for high-speed data transfer over short distances without the need for a wire connection. Bluetooth and Wi-Fi differ in terms of their purpose, capabilities, and other factors. Bluetooth allows for short-range data transfer between devices. As an example, it is commonly employed in headsets for mobile phones, enabling hands-free phone use. Wi-Fi, on the other hand, allows devices to connect to the Internet. (5)

Bluetooth Low Energy.

Bluetooth Low Energy (BLE) beacons are mounted on objects, walls, ceilings, and other surfaces transmit radio signals at certain intervals, allowing for indoor navigation. After determining that two objects are close to each other, Bluetooth location services can estimate the distance between them using the Received Signal Strength Indicator (RSSI). In particular, the new direction-finding capability in Bluetooth 5.1, enhances precision to the centimeter level. The antennas detect the angle at which the signal is received to determine the direction when locating the BLE-Tag. (1)

RFID & NFC.

RFID (radio frequency identification) technology has been around for decades, and NFC (Near Field Communication) is an evolution of it. NFC is now found on nearly all smartphones and wearables. The main difference between RFID and NFC is their transmission ranges; the former is frequently used over greater distances. Some regions, for example, use RFID to collect road tolls automatically. Toll tags are usually attached to the windshields of vehicles, and all you have to do is drive through the toll booth.

The convenience of NFC extends to devices without a screen. It is used by a lot of wireless speakers and headphones to exchange pairing information with your phone. NFC technology is currently used in data transfer, mobile payments, quick pairing, public transport access, gaming, home automation and other more niche areas.

UWB.

Similar to BLE, UWB (ultra wideband) technology has the ability to transmit info at low energy usage — but across a wide radio bandwidth, from 500 MHz (megahertz) to several gigahertz. That gives it a short range, compared to Wi-Fi. It even tried to compete with Wi-Fi back in the days, but it main purpose today is locating and measuring things. (7) It has security extensions built in, allowing it to be used for sensitive data transmission and other high-security applications.

56 felix schwamkrug masterthesis extended interaction space 2021 2022 57 theory and research

A real-world possibility for UWB includes using it in smart locks for passive, key-less entry, which means the tech can tell not only if you are approaching, but also if you are inside or outside the building or car.

While RFID, NFC, BLE, and UWB may appear to be similar, there are significant differences in technology and capabilities, and each has its own set of applications in the real-world.

Prositive thing is that UWB can deliver a much more precice location because it uses more frequency bands, the negative is that you must broadcast a signal, so UWB tags need energy compared to passive UHF RFID tags.

GPS.

Global Positioning System (GPS) is locating on a different scale than the technologies mentioned above. It is a sensor with antennas to help navigation. It receives continuous signals from satellites that help calculate the distance traveled and the location of your phone.

When a signal is received, the GPS sensor records a location. Depending on the time difference between any two signals, the distance is calculated. Navigation apps use both GPS and magnetometer to identify location and direction. (6)

As the name suggests, GPS is used globally and focuses on the outdoors, while Bluetooth is more useful indoors because of its directional capabilities and higher accuracy.

Cameras.

Cameras consist of a lens and an image sensor. An image sensor is a device that captures light and converts it into an image; when combined with an AI-enabled processor, the camera can classify objects without the need for human intervention. Certain Objects, signs and shapes could deliver information or trigger

predetermined events and settings. Hand and face gestures can also be identified in this way to control interactive elements and initiate processes.

LiDar.

Starting with the iPhone 12 Pro, Apple introduced a LiDAR scanner to its list of sensors. LiDAR, or Light Detection And Ranging, is similar to the ToF (time-of-flight) sensor, found in other devices, mainly to help with getting in-focus pictures. Instead of firing one laser, it fires multiple lasers for a more accurate 3D map of the environment. (3)

Radar.

Radar is a detection system that uses radio waves to determine object distance, angle, and velocity. (8) The 60 GHz (gigahertz) radar chip from Infineon is one of the most advanced and capable radar chips available. This sensor was first shown at the Google Developer Conference (Google I/O) in 2016, and it is now included in Google’s smartphones as “Soli”.

Soli, like other radar sensors, detects objects that reflect some of the energy back to the radar antenna by emitting electromagnetic waves in a broad beam. The data collected and analyzed includes, among other things, information about the object’s size, shape, and movement. This radar sensor can track sub-millimeter motion with extreme speed and precision.

Google Soli project uses radar expertise to enable new types of touchless human-machine interactions, such as those in which the human hand serves as a natural, intuitive interface for new devices and services. (3)

58 felix schwamkrug masterthesis extended interaction space 2021 2022 59 theory and research
Apples’ UWB tag compared to a coin, photograph and x-ray. Image source: iFixit

Proximity sensor.

Proximity sensors detect how close a certain object is to your phone. A quick example of this is your phone’s display turning off when you pick up and answer a call.

Light.

Ambient light sensors measure the intensity of light around the device. These sensors detect the changes in brightness of the surroundings and record its intensity.

Magnetometer.

Measures Earth’s magnetic field, typically in three directions, often as a compass. Combined with GPS, and some smart interpretation algorithms, measuring the magnetic field can help you automatically know where you are heading West on the freeway and not East. (2)

Accelerometer.

An accelerometer is designed to measure non-gravitational acceleration. The sensor could detect movement in different rooms. If you are wearing a wearable, your movements around the house could trigger lights or other devices, especially if they are connected via Bluetooth.

Gyroscope.

A gyroscope measures three dimensional rotation. Used in todays smartphones as a very sophisticated camera-shake correction: A gyroscope can detect your shaky hand and remove that exact motion to create a better picture. In certain situations, the gyroscope can also be a better indicator for direction than a magnetic compass. (2)

Shock, Step sensor.

A sensor, similar to the accelerator, specific to detect step movements.

Microphone.

Microphones detect the sound waves created by the expansion and rarefication of the air and turn this into a sound we can hear through a speaker. (2)

Ports.

The smartphone can be controlled via cable and control other devices or expand its functionalities via USB, jack, or special connectors such as Apple’s smart connector. Larger screens and additional sensors and output devices are common examples.

…and others.

In some phones there are — or were even more sensors. The remaining sensors are either obsolete and have been removed from current devices’ feature sets, or they are unrelated to the research topic and conceptualization though.

(hygrometer (humidity sensor), barometer (air pressure sensor), thermometer, RGB light sensor, AM/FM radio, heart rate monitor)

Amos Kingatua. medium.com, Bluetooth Indoor Positioning and Tracking Solutions, 2020, received 16.03.22

2 Golden Krishna. The Best Interface Is No Interface/Designers Toolkit, 2021

3 smartphonedomain.com, Smartphone Sensors Explained, 2021, received 19.03.22

4 signalboosters.com, The Di fferences Ethernet, WiFi and Cellular, received 18.03.22

5 britannica.com, What’s the Di fference Between Bluetooth and Wi-Fi?, received 19.03.22

6 techmende.com, What Is A Smart There So Many Of Them?, received 19.03.22

7 Eric Gri ffi th. pcmag.com, Ultra-Wideband Explained, 2021, received 19.03.22

8 Nature Magazine, Radio detection and ranging, 1943

9 infineon.com, received 20.03.22

Enough to work with.

Following this list, it should be clear that modern smartphones, as well as the ecosystem in which they exist, are extremely capable devices that provide an ideal platform for experimenting with, testing, and establishing novel interaction methods. Of course, these sensors are interconnected by a processor that is capable of connecting data from multiple sensors and interpreting it in a way that is appropriate for the current application. If technologies like artificial intelligence and especially machine learning are used, the smartphone can even work with contradictory and changing values to arrive at the same conclusion.

60 felix schwamkrug masterthesis extended interaction space 2021 2022 61 theory and research

Wearables.

and other cyborg modules

“Wearables” refers to computer systems that are worn on the body and are designed to assist the wearer in their daily activities. They can be connected to the internet or wirelessly to a smartphone or tablet. (1)

Unlike other mobile computing systems, wearables’ primary functions include tracking via sensors, applications, hardware, and software, as well as mobile data processing. The data gathered by tracking is gathered from the user’s environment, behavior, and physiological condition, and is usually only displayed in broad strokes to the user. For a more complete presentation and review of the data, you will need a smartphone, tablet, or PC.

Wearables have one thing in common: despite being worn consistently or frequently, they are usually hardly apparent and should provide the wearer with useful added value.

Smartwatches, data glasses, headsets, smart earbuds, technology tattoos, fitness trackers, running watches, smart jewelry, and other devices are classified based on their use and how they

are worn. Wearables, for example, can be woven into textiles and used as smart apparel. Google Glass, a wearable worn as visual data addition on the face was announced in 2012 with a lot of media attention and interesting features. (3)

In last years the concept has spread to other sectors of application, which has been made possible by the devices’ increasing miniaturization and communication capabilities. (4)

Besides being able to collect data and acting as haptic feedback, microphone can help with communicating actions of users because of their nature being placed at a specific location on the body and therefore acting as specific interface for input or output of communication. For example can the pickup of the Bluetooth In-Ear headphones act as a trigger for music to start playing. These triggered actions just have to be programmed and do not necessarily need additional hardware because the action of turning something on or getting into some wireless range itself carries the information of the intent and purpose.

62 felix schwamkrug masterthesis extended interaction space 2021 2022 63 theory and research
1 Prof. Dr. Oliver Bendel. wirtschaftslexikon.gabler.de, Gabler Wirtschaftslexikon, Wearables, received 10.04.22 3 foerderland.de, Förderland: Business Magazin für Entscheider, Was sind Wearables?, received 10.04.22 4 Manfred Klein. healthcare-computing.de, Healthcare Computing, Was sind Wearables?, 2019, 10.04.22
steps
opeening a smart doorlock of a screen based flow. The steps with a sensor based flow – either using the smartphone or wearables.
The
of

Computing and intelligence.

Artificial Intelligence (AI)

AI Is the theory and development of systems that are able to perform tasks requiring intelligence. (5) Artificial intelligence, then, refers to the output of a computer. (3)

Machine Learning

Is a form of AI that mimics how humans learn. (1), (2) The system learns to solve problems in ways that are not explicitly programmed. In machine learning, the learning process is supervised, and the programmer has to be specific when telling the computer what types of things it should be looking for. (3)

Deep Learning

Is a brain-like logic structure for machine learning algorithms. (1), (2), (3) Each algorithm in the hierarchy performs a nonlinear transformation on its input before generating a statistical model as an output. Iterations continue until the output is accurate enough to be useful. The label “deep” was inspired by the number of processing layers that data must pass through. To achieve an acceptable level of accuracy, deep learning programs require access to immense amounts of training data and processing power, neither of which were easily available to programmers until the era of big data and cloud computing. (3)

4

5 oxfordreference.com,

Design and AI.

We are forgetful, fragile, and busy. Computers should do the things we do not want to do, that we do not know we should do, and that we are not able to do.

When

to

use AI in a service design context?

¬ People do not know how to do something

¬ People cannot do something

¬ The task is boring, repetitive or dangerous

When not to use automatic AI?

¬ People enjoy or feel responsible for the task

¬ High stakes situation

¬ Complicated personal preferences

(1)

At the moment, many machine learning services require you to send your data to the cloud in order to perform the actual intelligent analysis. Advanced AI necessitates computers performing a large number of small calculations quickly. Regular CPUs (central processing unit in a computer) are unsuitable for the demands of machine learning. As a result, GPUs (graphics processing units) are used — they are designed to perform small simultaneous calculations. (4) Other architectural changes must be made to increase the number of simultaneous tasks your chip can perform, so manufacturers have already begun to install dedicated AI chips, essentially specialized GPUs, by default on new devices. As a result, the above concerns are a thing of the past, and data-heavy and data-sensitive AI applications will be available on any new smartphone in the coming years.

64 felix schwamkrug masterthesis extended interaction space 2021 2022 65 theory and research
levity.ai 2 ibm.com, AI vs. Machine Learning vs. Deep Learning vs. Neural Networks 3 techtarget.com, deep learning (definition), 2021, received 21.03.22
James Vincent. theverge.com, A brief guide to mobile AI chips, 2017, received 21.03.22
artificial intelligence

IoT (abbreviation)

Internet of Things Objects

with

computing devices

in them that are able to connect to each other and exchange data using the internet.

APIs.

You probably use them everyday

An API (Application Programming Interface) is an “interface that is defined in terms of a set of functions and procedures, and enables a program to gain access to facilities within an application.” (3) Simply put, an API is an interface that allows software to interact with other software. This is part of its name and is core to its functionality. APIs are held allowing an existing or new business capability available from across a network as an on-demand service for usage by other applications and software. (4)

APIs helped transforming “the web from a page-centric collection of documents” to an “architecture of participation” in which users create and share data with the help of a network of Web applications and communication with each other through APIs.” Therefore, they deem APIs as the true building blocks of digital transformation. (4)

In today’s economy, they play a bigger role than ever before. They enable us to share important information and expose valuable functionality among devices, applications, and individuals. APIs are everywhere, powering our lives from behind the scenes, even if we do not notice them.

Some of the most relatable examples of API usage in our everyday lives.

Weather Report snippets.

For example, if you search for your local weather conditions on Google, you will see a dedicated box at the top of the search results with a forecast. Google is not in the business of providing weather data, so they source this information from a third party.

Log–In using Facebook, Google or others.

Instead of actually logging-in to users’ social media accounts, applications with this functionality leverage these platforms’ APIs to authenticate the user with each login.

Paying via PayPal.

When the user clicks the “Pay with PayPal” button, the application sends an “order” request to the PayPal API, specifying the amount owed and other important details. Then, a pop-up authenticates the user and confirms their purchase. Finally, if everything goes to plan, the API sends confirmation of payment back to the application.

Twine for Zoom.

Twine is an example of an API that facilitates communication in the area of remote work and can be integrated into the existing infrastructures of the Zoom video communication service.

Twine for Zoom is a current and relevant example of implementing functions through the use of APIs.

It is one of a few new breakout room-focused apps that will be added to the Zoom App Marketplace in the coming weeks. Zoom recently granted API access to a select few companies, allowing them to build platforms on top of its breakout room technology. Twine for Zoom will give businesses the tools they need to create social breakouts strategically at all-hands meetings, company socials, onboarding events, and other virtual gatherings. Virtual tables with specific conversation topics can be created by company leaders. The tables are visually represented as a sidebar within Zoom, and employees can see who is seated at each table and what the topic is before selecting one to join and changing at any time. (6)

Online travel agencies to compare flight prices.

APIs are great for travel services since they make it easy for machines to quickly and autonomously exchange both data and requests — in this case, trip availabilities and reservation requests. (5)

66 felix schwamkrug masterthesis extended interaction space 2021 2022 67 theory and research
3 Dictionary of Computer Science – Oxford Quick Reference, 2016 4 Jukka Huhtamäki, Rahul C. Basole, Kaisa Still, Martha G. Russell & Marko Seppänen. Visualizing the geography of platform boundary resources: the case of the global API ecosystem, Proceedings of the 50th HI International Conference on System Sciences, 2017 5 nordica-pis.com, Five examples of apis we use in our everyday lives, 2019, received 02.04.22 6 inc.com, Zoom’s Breakout Room Feature Is About to Get a Lot Smarter Article, received 22.02.22
Joining the “Breakout Room” in Twine for Zoom Image source: Twine for Zoom

The development in recent years toward equipping smartphones with ultra wide band (UWB) sensors to precisely locate devices inside, magnetic accessories, and other technologies that could support using the smartphone when working from home is inspiring to me and my concept.

68 felix schwamkrug masterthesis extended interaction space 2021 2022 69 theory and research Timeline of Smartphone Milestones.
Camera Sharp J-Phone Accelerometer Samsung SCH-S310 GPS Benefon ESC NFC Nokia 6131 First “Smartphone” Ericsson GS88 Mobile Internet (3G) Motorola A820 Fingerprint Sensor Pantech GI100 Apps Apples first iPhone Multi-Touch interface Apples first iPhone Proximity & ambient light Sensor Apples first iPhone 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 1995 1994 1993 3D Face detection sensor Apple iPhone X Siri Apple iPhone 4 Gyroscope Apple iPhone 4 Qi Wireless Charing Nokia Lumia 920 PowerSharing Samsung S10 Soli Google Pixel 4 Folding screens Samsung Galaxy Fold UWB Apple iPhone 11 MagSafe Apple iPhone 12 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2009
Enableing new ideas and possibilities for developers and designers.

Connecting the dots.

Introduction.

Remote work will soon be the norm. Organizations will have to change structures and we have to see what it is that is important to be happy and efficient in this situation.

Prelimanary Questions

Keeping up boundaries, meeting new people, collaborative teamwork suffers when working remotely. Besides offering a more flexible and calm environment, it requires self-discipline and strong communication skills to be effective. According to many, the office represents a concatenation of interruptions. This is especially true when comparing the experience to working remotely.

Work.

Additionally, remote work also cuts commute time and living expenses. So all in all, remote working is here to stay but needs to be optimized for the human factor. Working from home parttime will play a significant role in the future of office jobs, as will services that optimize a life lived in this way, because work brings essential purpose to everyone’s life.

COVID–19.

The pandemic eased changes made in digitalization and normalization of working from home and is offering now opportunities for new thoughts, workflows and products.

Trends for basic needs.

This decline in real social interaction has already created and brought up new products and services that try to bring people together. Podcasts

are used as a replacement for those social needs nowadays. It is obvious that this is a big issue in this generation and it is on a rising trend. Physically working together in a room has also proven to be essential for effective and creative results.

Communication.

That is partly because communication is so much more than words and partly because energy and motivation are transferred better in person. Solutions that can transfer these human qualities will always prevail.

Motivation.

This kind of motivation can also come from within if the worker is capable of the task, can learn to add their own ideas and work in an individual approach to a purposeful goal.

Gamification.

A way to trigger people’s emotions and get them engaged in their work is through the means of game design, which is implemented in products, services, or tedious tasks with a meaningful purpose. This modification is called “gamification”.

A mix of feedback loops, alternating rewards and efforts packaged with entertaining elements can strongly trigger those emotions. Implementing those elements leads to a greater acceptance of annoyances and repetitive tasks, as well as ideas for how to break down complex processes in a playful way.

Flow.

Keeping in mind those elements and techniques can help with work to keep motivated. Analyzing

and creating recaps of the workday can show progress and break down large projects. Awarding oneself with a pause after a block of work and a supportive environment helps stay in focus and in a state of flow. Important for this to work are thinking about precautions and a good level of self-skill and challenge in work. The separation of actual work and its reflection also keeps you in the present.

Psychology.

Today’s devices and platforms often keep us from being in the present as they are increasingly made to be more engaging and distracting. The huge amounts of money being made in advertisements and user data has changed many digital products to play with people’s emotions so much that they have even become addictive.

Smartphones.

The smartphone has become the main transporter of almost all digital content. That made it indispensable for both users and companies. Since the device handles so much in our lives already and adds a lot of smart to areas of our lives, we should start using this hugely capable device for work.

Screens.

The screen acts as the main interface of smartphones, being a deliverer of entertainment, information, and advertisement content. People are unlocking their screens 150 times a day to use their phone. Since this is far from being perfect and intuitive in many situations, there are trends toward concentration of automation of processes and sensor utilization.

Interacting with tech.

Computers like smartphones are equipped with an unprecedented number of sensors, unknown and underestimated by many, and underused by services and apps. They can interpret several human communication methods. Users can interact by voice, gestures, and by manipulating the surroundings that engage with the phone. The phone, on the other hand, can interchange information through its various outputs and by inferring.

Wearables.

Wearables are the most elegant solution when it comes to intuitively exchanging information with the user. Their proximity to the body helps feed gestures and data directly to the smartphone or a corresponding service for it to react or collect.

Computing and intelligence.

The inferences made by artificial intelligence in this case are to be used a limited amount. Because the user and remote worker want to stay in control and because of privacy matters, the automation is to stay on the primary task of action.

APIs.

One way services work together and collect data from each other is via APIs. These digital interfaces are working mostly in the background, raising the value of all the connected components. An interface could therefore solely work via APIs by interacting with colleagues on a collaboration platform like Zoom, Slack, or Teams by gesticulating near the smartphone. Of course, smartphones play a huge role in APIs too.

70 felix schwamkrug masterthesis extended interaction space 2021 2022 71 theory and research
We interact based on sensory impressions and communicate through our physical virtuosities.

As outlined in the beginning, my focus has been on remote work environments, sensor-based interaction and utilization, and collaboration and communication. My research on these topics has brought out a lot of new ideas and issues to work on for my concept. Considering remote work is the whole model of work outside of a dedicated office, I will focus on working at home. As for the aspect of collaboration, I will focus on the low-threshold, in-parts recreational conversations and efficient communication happening throughout the day in a good work setting. The sensor implementation is thought of as being used as the control and connection between those two areas. For my concept, I considered radar technologies the most useful for determining the location and setting of the user as well as inferring their reactions.

72 felix schwamkrug masterthesis extended interaction space 2021 2022 73 theory and research

Turn static files into dynamic content formats.

Create a flipbook
Issuu converts static files into: digital portfolios, online yearbooks, online catalogs, digital photo albums and more. Sign up and create your flipbook.