Designing for the User Experience - DG310 -
Kevin Lu Lucas Haude Charlotte van der Sommen
2nd semester 2015
Table of Content Introduction Week 1 Assignment Result Group Reflection Week 2 Assignment Result Group Reflection Week 3 Assignment Result Group Reflection Week 4 Assignment Result Group Reflection Week 5 Assignment Result Group Reflection Week 6 Assignment Result Group Reflection Individual Reflections References Appendix A - Cultural Probes Pilot Appendix B - Cultural Probes Final Appendix C - User-tests Appendix D - Results User-tests
Page 1 Page 3 Page 4 Page 5 Page 8 Page 11 Page 13-14-15 Page 13-14-15 Page 15 Page 17 Page 17 Page 17-18 Page 19 Page 21 Page 21-22 Page 21-22 Page 23 Page 25 Page 25 Page 25 Page 25 Page 26 Page 26 Page 26 Page 27 Page 28 Page 33 Page 34 Page 36 Page 38 Page 42
Report: “Designing for the User Experience” - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Introduction In this report the assignment ‘Designing for the User Experience’ will be detailed by the workgroup of Kevin Lu, Charlotte van der Sommen and Lucas Haude. For each lecture and its corresponding homework assignment there will be a section explaining our progress, results, conclusion and a group reflection on the feedback we received from fellow students and lecturers. The assignment started out with a definition of different paradigms or economies in which a product can be categorised, starting from the industrial economy that arose during the 1950’s. When growth in this paradigm became stagnant, companies tried to increase their marketshare by beginning to sell more of an experience or lifestyle connected to their products, providing them with some new business ventures. Eventually, the growth potential of this paradigm was reached and companies had to innovate again. The new paradigm that emerged was the knowledge paradigm, characterised by products or services that provided a platform for users to share knowledge or other aspects of life, supporting personal development. Looking into the future, the paper about these paradigms predicts the next type of economy that can provide growth opportunities: the transformation economy that focuses more on intrapersonal development. The products and services of this economy are geared towards creating revenue, but doing so while trying to make real, tangible improvements to peoples lives - not just in the direct environment of the designer, but (preferably?) so in developing countries. The environment itself can be influenced in a positive way as well. This provided some background for the following lectures in this assignment that were centred around the user experience (UX) itself. Because experiences are fleeting and difficult to quantify, different ways of analysing the UX were explained. Both qualitative and quantitative research methods were outlined and discussed, as well as effective ways to analyse the resulting data. We were immediately challenged with a practical approach and conducted these researches ourselves on a small scale. The final lectures concerned the statistical analysis of the quantitative user tests or questionnaires. With this knowledge we were able to discern between statistically significant results and results that could have been obtained or influenced by chance. Each group member concludes the report with an individual reflection on the assignment as a whole.
Report: “Designing for the User Experience” - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 1
Page 2
Week 1 During the first week of the assignment we got introduced in designing for the user experience. We learned that (the value of interactive systems lies in their ability to support and mediate user experiences rather than in what we can do with them and what they look like. It’s about how they enable us to carry out activities that are meaningful to us “Jacque Terken”). When you design for the user experience you need to understand all the ins and outs of the interactive system. To make it more accessible you can look at the product aspects and the people aspects: It the product functional? How is the interaction with the product? And what does the product look and feel like? Also, what values are important to the user? And what goals, needs and pleasures has the user? Maslow and Jordan, created hierarchies to categorize the needs of the clients towards the product or system. In Jordan’s frameworks he looks at different levels and different types of pleasures that could affect the user.
The different types of pleasures are: ‘Physio’ to do with the body and the senses, ‘Psycho’ to do with the mind and the emotions, ‘Socio’ to do with relationships and status and ‘Ideo’ to do with beliefs and values. For the usability and usefulness you look at: ‘effectiveness’, ‘efficiency’ and ‘satisfaction’. Then we got introduced in the different economy paradigms: The Industrial economy In this economy it’s all about product ownership and it played a primary role in modernizing societies and advancing standards of living of many people around the fifties. The Experience economy Report: “Designing for the User Experience” - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 3
This economy is focused on experiences. It introduced lifestyle brands as a business response to the loss of identity and cultural references that many people experienced around the eighties. The Knowledge economy This economy is about self-actualization. In this economy people no longer need brands to signify their identity, social status or lifestyle. By the help of social media platforms people can be their own brand. This economy unfolded in the last ten years. The Transformation economy This economy is focused on meaningful living. The imperative of this economy will be to provide meaningful context-specific propositions built with long-lasting profitable, ethical, and fair business based on multiple stakeholders collaboration and value sharing. This economy is upcoming at this moment. Another way to rate a user experience is to ask yourself the: What and How about the product and the Why for the deeper layer behind the product. This way you can also find out the Utility, Relevance, Usefulness, specificity, usability and feasibility of the product. People have a few basic needs they search for in a product. Once a product meets most or all of these needs, it will provide a positive attitude towards the product. These basic needs are: Autonomy, Competency, Relatedness, Stimulation, Popularity and Security.
Assignment 1:
The first assignment we got for this assignment was to select three different products. First a communication device, than a transportation device and then a music device. For every device we had to find the right economy paradigm and we had to match it with the basic needs they provide for.
Page 4
Result1:
Report: “Designing for the User Experience� - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 5
Assignment 2:
Choose examples of interesting applications within the applications: Communication, Physical exercise and Listening to music. Analyze examples related to experience paradigms and scale an existing product up to a new scale.
Result 2: Music performance
opening up the experience Nightspots and festivals were placed under the experience paradigm with the following human needs met: - Relatedness - Stimulation - Popularity - Autonomy To take these type of venues to the knowledge paradigm we want to transform them to a type of platform where creativity can be shared. This accomplish this we want to: - Add competency as one of the fulfilled needs - Change the environment into a network of people and technology - Incite creativity and collaboration The project - Group Music Improvisation System - OoC A project where: - Students create a space that enables an interactive experience - Musicians and the audience play instruments - The instruments, musicians, audience and space all influence each other Goal: to create an interactive network of technology and people. This concept can be further expanded with people outside of the venue participating through their cellphones or computers. Visitors become more proficient at following rhythm and melodies (competency). Higher level of stimulation & relatedness
Page 6
Bike
starting from an industry economy The bicycle was placed under the industry paradigm with the following human needs met: - Autonomy - Competency - Relatedness - Stimulation - Popularity - Security To take this product to the experience paradigm we want to transform it to add an experience and a brand identity to the product. The bike into the Experience economy- The Adjustibike - There will be a global concept - Well promoted website where you can order a bike adjusted to your needs andwants. - Colour, materials, size & shapes are customised. - Delivered to your house - Assembled by the customer Improved or added: - Competency - Stimulation - Popularity
Report: “Designing for the User Experience� - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 7
Group reflection:
While presenting the first assignment we figured out that we didn’t totally understood the difference in paradigms. After the group discussion this became much more clear. In the beginning we placed the bike in the experience economy because we thought about biking and that it would be an experience. After the discussion we figured out that a bike belongs in the industry economy since the focus is about the product itself and a bike is a mass production product that every person can have without any experiential influences due to brands. Another example was the smart phone. This was a very complicated example since a smart phone has many different features that makes every smart phone different from the other. In this case it is very important that you frame the product well. We decided to go for only the device itself without all the special features and we had a hard time to agree in which economy it would fit. Eventually we concluded that it would fit it in two different economies. The industrial economy because it is a person owned device that’s made in mass production created to modernize life. It also fits in the experience economy because in many cases people buy a phone because of the brand or just the way it looks. In this way the experience around the product is very important. Look at the IPhone of apple. Also the Dance Club was a challenging example. We put it in the experience economy since it really is an experience and it could be a way to break social taboos. On the other hand it not necessarily has to be a brand experience, it is most of the time and experience on itself. And it definitely isn’t very global. We decided to place it in the Experience economy because even after the group discussion we still thought The Dance Club would fit best in that economy. We can conclude that this method of categorizing products among economy paradigms works best for touchable product and makes it more difficult to categorize experience like products. We concluded that many products clearly fit in just one economy, but that some products can fit to different economies. We also concluded that framing the product is very important to do further research on the product. An observation we had was that the personalized products that we chose of which almost everyone one also matches all the basic needs, people search for in a product. The product we chose that was more something you could use/do so now and then, didn’t match all the basic needs. Here we can conclude how important it is for a product that it matches the basic needs.
Page 8
Report: “Designing for the User Experience� - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 9
Page 10
Week 2 Week two was about designing for the User Experience. While designing for the User Experience it is very important that you understand how interactive systems can help people to satisfy their needs and aspirations. For creating this understanding there are many different Methods to do research in the early phases of the Design process. Within the Reflective Transformative Design Process you could see this User Experience Evaluation within the “Exploring, Validating in Context” part of the process. The reason to involve users in the early stages of the design process is to create empathy with the user, to understand the user in his/her context, to understand the user’s goals, to uncover the user’s needs and to form a vision on what exactly you wan to design for. There are different kinds of methods for doing this kind of early design phase research. To start with, there are qualitative and quantitative methods to gain information. Next to that there are different ways to test. You can ask you uses for instance by the help of a questionnaire, you can direct observe your users, you can indirect observe your users and you can participatory design methods.
An example of an indirect observation study is the “Cultural Probes Method”:
In this method the researcher hands out a toolkit with tools like camera’s post-cards, maps etc., to gain information from the users about certain circumstances. Strong points: As researcher you have no personal influence on the data. The data is qualitative as well as quantitative. You give the user total creative freedom This method doesn’t provoke inspirational responses from the users. The tools can create personal spontaneous conversations that make the connection with the target group stronger. The results of the tests create an awareness in the detailed texture of sites. The probes can also educate the target group to create conversations about the subjects. Weak points: Because you don’t have any influence on how the users use the tools, it is hard to direct your users to the data you want. The postcards where sent in return, if the post doesn’t work sufficient, you have the chance to loose data. It is very important that you use the right tools and design them well for the situation in order to gain the data you want. By just handing over packages to people, there exist a chance that people forget them and never return them. The whole period of testing takes quite some time. It is not very clear what the direct influence of the results of the test on the design is. Report: “Designing for the User Experience” - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 11
Example: When you are designing a communication service for Chinese and Dutch students. In this situation the “Cultural Probes” Method would work very good. With the tools you would be able to get insight in the differences and similarities within the cultures that can help in designing a fitting solution.
An example of an indirect observation study is the “Diary Studies Method”: This is a method of understanding participant behavior and intent in situ that minimizes the effects of observers on participants. The participants are in charge according to time management and means of capture.
Strong points: With this method, you get the participants very focused and detailed about the subject due to the uses of photos. When using the participant-driven photo elicitation, participants hold their attention because it becomes more personal and it makes it easy to structure the interviews. Weak points: Because the participants decide what is going to be captured and how long it is going to take, it is hard for the researcher to plan the test and to direct the data in a certain direction. It is a big challenge to concretize the content of the data. Example: When you get the challenge to design an experience for a theme park. The Diary stud method would be very fitting to test the experience. You give people a camera and let them experience your design. Afterwards you can talk about the experience with the participants by the help of the pictures. This way it becomes very clear what part of your designed experience communicates what you want it to do, and what part doesn’t.
An example of a direct observation study is the “Contextual Inquiry Method”:
Contextual Inquiry is a technique that helps you understand the ‘real’ environment people live in and work in, and it reveals their needs within that environment. By visiting people and observing them as they go about their work and in watching them carefully and studying the tools they use, it’s possible to understand what problems people face and how you product can fit into their lives. Strong points: This method makes you understand the whole environment of the users, which makes the research very complete. According to the time you have for your research, you can decide for yourself how many people you observe. It is also possible to use only five people. You can enlarge this method as much as you want in order to make the result more accurate for your research. You can easily combine this method with other methods. For instance the “sportscaster method”. The method is very intuitive for the user. After using this method, you have very elaborate research results. It is qualitative data.
Page 12
Weak points: It is only possible to practice this method in the workspaces of the clients, you are not able to do it from a distance. This also results in the fact that you have to be thoroughly prepared because you can’t just go back to get something you forgot. For this method you should use people that you think will want to use your product and that will fit to the target group. This might make it very hard to find a target audience. It takes a lot of time to do the sessions. For the researcher but also for the target audience. You might arrange a lot of thing since you invade in a working office of someone. Like nondisclosure agreements with colleagues. You have to do a lot of research on forehand, according terminology, the tools and the techniques that they are likely to be using in the environment. This method asks for a lot of preparation and very good communication with the user, otherwise it won’t give you the results you need. This method won’t work if there is no comfortable relationship between the interviewer and the user. Since the data is qualitative data, it is not easy to process it. Example: When designing a tool for primary school children to help them understand how to make calculations. In this situation this method would be great to get qualitative data about the struggles the children have with calculations, but also the things that are easy. It gives also insight in the methods the kids use now and all the activities around making the calculations.
Assignment 1:
We had to select a case study and one of the three methods above to do User research on the case. There were three subjects, enhancing communication between parent and outliving child, more exercise in the workspaces and listening music on your way to school.
Result:
Challenge: Enhancing communication between parents and children living away from each other. Setting goal: Designing a tool to improve the contact and make the contact more convenient between parents and their children that live apart from each other. User research question: To become aware of the behavior in communication between children and their parents. Method: We will use the “Cultural Probes Method”. We think this method will give us the most input in the whole experience of the communication between the students and their parents, compared to other research methods. The “Contextual Inquiry method” can only be done during the contact between the student and the parents, we don’t want to force the situation. The “Diary Study method” restricts the participants because of set times and dates of answering questions. With the “Cultural Probes Method” the participant has all the freedom to express themselves within the assignments. That will provide more valuable feedback in our eyes. Report: “Designing for the User Experience” - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 13
Implementation What we want to research: Who starts communicating? Do you want that to be different? In which environment do you communicate? Do you want that to be different? How often do you communicate? Do you want that to be different? Which medium do you use to communicate? Do you want that to be different? What do you share during your communication? Do you want that to be different? What emotional impact did your communication have? Do you want that to be different? We will make about four or five different probes Use the camera of your smart phone to make a picture of your environment, every time you have contact with your parents. If you have a direct reason for contact make a picture of this. Sent this picture to us. (environment – how often – (reason for contact)) Create a visual that shows the amount of communication you have with people around you compared to the communication you have with your parents. Create a list of questions and assignments, and sent these questions and assignments by Whatsapp, and sent these thinks ever view hours. Identify participants: Children – students What do we expect? We expect to receive information about how communication takes place between more or less independent students and their parents. This information comes from different probes that are designed to reveal the answers to the questions mentioned under Implementation. Using the probes we can find out more about the ways students communicate with their parents and come up with tools to improve their contact. Informed consent form This survey is voluntary – the information you send us is on your own discretion. It’s not mandatory to mention or record conversations with private topics you don’t feel comfortable sharing. The information you send us will only be used for the assignment that we are executing. Triangulation The probes we designed approach the subject in different ways to give us multiple angles on the subject. We ask the students to draw out what they talked about and include their emotions to find out what is discussed. We also ask them to record their medium and surroundings to find out about differences in informal conversation and more serious interactions.
Assignment 2:
Do a pilot study with the first probes (Appendix A). Use for this study at least three participants. In the our first attempt, we’ve made an envelope with 5 probes. Each probe had a description and
Page 14
some paper to express themselves. Draw a scenario of each interaction, what happens during your conservation and how do you feel. Draw it in a strip-scenario. Postcard, there is a postcard provided in the envelope. We would like to ask you to write a postcard to your parents. Communication icons, we like go know what mediums are used during your conversations with your parents and how often. We supply some commonly used icons (telephone, Whatsapp, etc.) and ask you to relate them to icons of yourself and your parents. Photo, in what environment are you if you have a conversation with your parents. Is it something related to the occasion when you are in touch with them. Make a photo or a drawing of something which you think fit in the occasion. Expression, we would like to know how your conversation starts.Please make an overview how many times you are in contact with your parents. Also specify who made contact first.
Result 2:
We expected that all answers were very similar and that there was just a slight difference in then. However, everyone was very creative and made something different. We also received feedback from the participants who told us that some probes were a bit vague and time consuming. We would use their advice to create the second probes.
Assignment 3:
Finalize the Cultural Probes (Appendix B) and do the final User research by asking at least nine people to participate in the User research test. In the second attempt we’ve used the feedback we received, we changed the descriptions of the probes a little bit and let the participants be more free.
Result 3:
The final probes we received was less successful, we didn’t received all results back from the participants, for some reasons they eventually didn’t wanted to fill them in, so we had only results from 7 people. The results were like we wanted more diverse, it was interesting that most of the participants had called their parents in such a short timeframe (maybe they forced themselves to call them to be able to participate in this probe), and with these results we’ve made an affinity diagram.
Group reflection
There was some hesitation regarding the method we used for communication between child and parent. Sometimes there was discussion if our probes were too much in the direction of diary studies. We concluded that we also could use the diary studies to research this subject. We should have made an overview of methods for qualitative data gathering in the early phase of the process for designing for User Experience to avoid mixing up different methods. During the test we noticed some participants were not willing to fill in the probes as we asked them to do. So we were told that it should be more direct and clear, and some descriptions were too open. In our probes, we didn’t gave the participants an overall description, so some misunderstanding between us and the participants. Next time the probes need a more elaborate description in general. The people really liked to be creative so adding different colors pencils and giving them blank papers was a really good idea. The cultural probes study went pretty well, we received lots of (positive and negative) feedback. For the next time a bit more time to make, spread and fill-in such method is very appreciated. The participants only had 2 or 3 days to fill in these probes. Report: “Designing for the User Experience” - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 15
Page 16
Week 3 Qualitative analysis To help process the data we gathered from the cultural probes, the week 3 lecture provided information about affinity diagrams. This type of diagram allows researchers to order and categorise large amounts of (qualitative) data into a format that can be used for further steps in the design process. An affinity diagram is created by a small team; each member takes turns to write down keywords on pieces of paper. These keywords must take inspiration from the data pool, but there is no ‘right’ or ‘wrong’ possibility with each of these notes. Team members can build upon the notes of others, creating groups or categories of notes. When there are no more notes being added, the team can then discuss each item and try to group the individual notes. Each group is then given a one-word or short sentence description. The affinity diagram we made looked like this:
Report: “Designing for the User Experience” - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 17
The notes written in block letters indicate a group; the groups were defined by us as follows: • Number of participants - either a conversation with a single person (2 total), or a group of 3 or more people. This distinction rules out some mediums of communication. For instance, WhatsApp can be used for a conversation with a single person, but the data showed that nearly everyone who uses it is also in multiple groups with family, friends and/or students. Phone calls on the other hand are nearly always between two people. • Relation of participants - this probably has some influence on what communication mediums are used. Grandparents might be more likely to call by phone instead of Skype, while friends and students more often use WhatsApp and Facebook to connect. • Medium - we made a distinction between direct and indirect communication. Phone calls and face-to-face conversations are direct, WhatsApp and e-mail are indirect. Text messages, Skype and FaceTime are also used. • Reason for contact - the initial motive for people to contact another might be a ‘need’ or ‘want’, urgent or just because. • Subject of conversation - although most conversations are about more than one subject, there is usually a main theme, especially if there is a set ‘goal’ for a conversation e.g. a student asking their parents to borrow their car. This is often strongly linked to the reason for contact. Conclusions Nowadays there are many options available for students and parents to contact each other; there is no need to reinvent the wheel and create a viable competitor of WhatsApp or Skype. To improve the contact between students and their parents in a meaningful way, a different approach has to be taken. One of the probes that we handed to the participants was a post card that the student would send to their parents. Because this medium is rarely used by most people, the written text is also a little different from usual conversation. A possible contribution to improve contact could be a service where students (or others) digitally write a post card with an image they choose or upload with a hand written text (by photo), which is then stamped and mailed to their parents. This combines the habit and ease of digital mediums students are used with the charm of a post card that their parents appreciate.
Page 18
Group reflection
Making the affinity diagram turned out to be a creative and informal yet effective way to organise the data we received from our participants. It allowed for a overview of the results, which is quite helpful because the cultural probes method yields a large amount of information - it’s easy to lose track of the overall picture or getting lost in details. A downside of this method is that there is a loss of data in the sense that the received results are not directly represented in the affinity diagram; the team members use the data to make a summary or association of (parts of) the data. Therefore it is important to keep the data at hand while brainstorming about the notes/ideas so that everything gets represented. We could improve upon this step in the research process by allocating two different moments to the creation of the post-it notes, perhaps doing the sessions in another setting. This could lead to more diverse results which seems beneficial to the end result. This method relies on the ideation and creativity of the participants to be really effective. After creating our affinity diagram, we concluded that we had probably made a good choice by going for the cultural probes model to conduct our research. Our goal was to create a product or service to improve contact between parents and their studying kids, and the cultural probe does seem to give the most opportunity for the creation of concepts. The other two study methods (contextual inquiry & diary studies) seem to be favourable when a researcher wants to know specific points of improvement for an existing product.
Report: “Designing for the User Experience� - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 19
Page 20
Week 4 Storyply Workshop The Storyply method is a way of creating concepts for a design. It requires some knowledge of the situation or design challenge, and provides a guide by which the user envisions and draws a scenario of the current situation that needs improvement. The scenario is complete with a main character (user of the new product or service) and a supporting cast (in our case the students’ parents) with props and surroundings. The aim is to first identify the conflict, or the problem that calls for a solution by drawing out the circumstances in which it arises. After identifying the problem the method aids in coming up with a solution; the user is then asked to re-write the scenario with the new design/solution and see how this affects the scenario and the people playing in it. As a final step, the method then calls for the creation of three concepts based on the initial design proposal. Storyply - people, places, objects The people in our scenario are: - Daniel, main character. 23 year old student. - His parents, Marie & Patrick - aged 57 and 61. The characters play out the scenario in their respective homes; Daniel lives apart from his parents in his Amsterdam room where he studies political sciences. His parents live in their house in Zierikzee.
To communicate they often use their mobile phones. Marie and Patrick usually call their son, while he initiates contact mostly using WhatsApp. Patrick’s phone does not support WhatsApp. Report: “Designing for the User Experience” - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 21
Identifying the conflict Daniel usually goes to his parents’ house during the weekend, but they always have contact about when he will arrive. The contact over the phone or text messaging is therefore mostly restricted to ‘business’ conversations. During Daniel’s stay they talk about lots of things, but the electronic contact remains superficial and does not have much inherent value. Daniel’s parents are quite interested in his everyday activities and would like to hear more often about what is keeping him occupied. Solving the conflict The conflict can be eased a bit by introducing a service that doesn’t compete with either phone calls or text messaging services, but rather adds a different, unusual communication medium. Another idea would be to create a service or product that makes the student inclined to share his daily activities more often with his/her parents. Three potential concepts • A service that enables users to create their own post card by selecting a picture for the front (from a library or their own upload) and adding their handwritten text (by photo) or printed text. The card is then sent to the specified address, combining some electronic ease of use with the charm of a paper post card. • A service that compiles a journal-type file from the user’s photos that he/she took on their smartphone, together with data from the user’s calendar and messaging services. The journal can be easily edited and sent to the recipients electronically.
•
Page 22
An app for smartphones that (at sensible times of the day) tries to arrange a phone call between two persons. These persons can decide the desired frequency of contact and the timeframe during which the app is active. Using this data the app randomly picks a time to try to connect the two people by initiating a phone call with notification - with the end goal to increase the amount of contact between two people.
Group reflection
The Storyply method is a very creative way for designers to try and come up with design concepts. It is mainly useful if there are not so many obvious solutions: by taking on a different perspective and way of thinking, this method can open up some new ideas. The downside is that relies on the imagination of the designer, and thus might not give the same results for everyone or a single individual every time. Next to that, it can be time consuming if scenarios are drawn with a lot of detail - the scenarios should adequately and effectively convey the message and incite new ideas, but should probably not become drawn-out comics with detailed backgrounds. Although this method was definitely fun to work with, it did not provide us with any new concepts, the more useful part is drawing out the scenario with the new product because it entices to think a little bit further about your product and its implications and workings.
Report: “Designing for the User Experience� - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 23
Page 24
Week 5 The fifth week of this assignment was about evaluating the design process. Did the results match your expectations, or is there something which could be improved. There are two major types of evaluations; Usability evaluation, it focusses on metrics which are related to effectiveness, efficiency and satisfactions. UX Evaluation is more related to usability, usefulness and pleasure. In the lecture more focus was put on UX Evaluation. Reasoning behind this method was explained and the method’s strong and weak points. This will be used to set up the surveys which are needed to get the results. This week’s assignment was to evaluate the products of the topic we’ve chosen. We would like to know which service is better to share moments with parent and child, via WhatsApp, In this case we had to make a survey which would help to draw a conclusion if WhatsApping with your parents is emotionally ‘better’ than making a phone call with your parents, or vice versa. To get the best impression we’ve chosen to use an AttrakDif questionnaire with a seven point scale. Each question in the survey would have two opposite characteristics and the user could choose on which scale the characteristic would fit best by their social event. We made two similar questionnaires, with exactly the same questions, but with different service descriptions and asked people to fill in these questionnaires. These filled in questionnaires would be analyzed the next week to conclude if our hypothesis was significant or not. The questionnaire we’ve made and spread out was a questionnaire with 11 questions with different characteristics of how the conversation went, for example fun-boring, formal-informal and friendly-unfriendly. To be able to analyze and get a result, at least 9 people had to fill in each questionnaire. Our questionnaire was made in Google Docs and we could easily distribute them to the users. In the end 18 people had filled in the questionnaire about making a phone call with your parents and 9 people had filled in the questionnaire about having a conservation over WhatsApp with your parents. In week 6 we’ve analyzed the data we’ve obtained and made a conclusion. The questionnaires can be found in Appendix C and the results are in the appendix D.
Group reflection
After conducting the first surveys we were a bit unclear if this was the right method for our research. The reason we’ve chosen the AttrakDif method was because it fits very well in conducting surveys in which involves feelings and emotions at a certain moment. But in the end, it didn’t fit quite well in our research, because both groups reflect on content of the conservation, and less likely on the experiences of it. We should have asked more questions about the user interface in combination with the feelings, and that’s what we’ve left out of it. Since we used only a few of the AttrakDif questions it may be wrong on our side, but not every question was relevant in our research, it was more related to hedonic and pragmatic questions. It was pretty difficult to choose which method would give the best results. In our opinion the papers we had to read in advance were a bit vague. In the end you should test all methods first before you know which method fits the best in your research. Report: “Designing for the User Experience” - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 25
Week 6 The sixth week of this assignment was all about analyzing data. The data gathered, which are qualitative or quantitative, and objective or subjective can all be analyzed by a method. There are several methods which has each their own characteristics and results. With the results a conclusion about the significance of the survey can be made. By choosing the right approach a significant conclusion can be drawn. Results can be calculated manually and with use of a program, such as SPSS Statistics, which is provided by the TU/e. With these statistics the significance of a survey can be determined. Our homework for this week was to learn how to use these analyzing methods by applying them with the program SPSS Statistics and Samplepower. First there were some example exercises about the different methods and Power tests. In the last exercise we had to analyze the results of our own questionnaire. The questionnaire we had made and spread out is an AttrakDif questionnaire with 11 questions about the emotional feeling when WhatsApping with your parents or making a phone call with your parents. Each question has a seven point scale which let the user choose between two characteristics of the service. Our goal was to come with a conclusion if one service is significant better than the other one. The results of the analyses were all pretty negative. None of the questions were significant and could be used to conclude if one service is better than the other one. The graphics with results are in the appendix. The get an overview of how many people had to fill in the questionnaire to get a conclusion where one service is more liked with a power of 0.80, we conducted a power test for each question and the results are below in the table.
N
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 59 44 205 37 182 137 1862 112 27 69 215
N is how many people are needed to get a result with a power of 0.80. In general the power test went smooth, there are some high exceptions, for example question 7, with a remarkably 1862 participants, and question 11 and 205, which both require more than 200 participants. We compared the results of those 3 questions and the means and noises were very low and evenly spread, so it takes more people to get a power of 0.80.
Page 26
Group reflection
The results of the analyses were all insignificant. We couldn’t conclude for any of the questions we’ve made if we could use it for any other case. We’ve asked in both surveys exactly the same questions, but since both actually reflect on the content of the conservation, we went wrong. The questions weren’t really about the service, but about (emotional) feelings of the users, and the service, which we were reviewing, was only a medium. Since a positive conversation with Whatsapp, and over the phone can both be given a 5 or 6 in happiness, it doesn’t mean one service is significant better than the other one. We should have asked more different questions regarding the services or conducted the survey a bit differently. For example, less options, in where we used a 7 point scale, only using a 3 or 5 point scale will give us more concrete answers. However for the power test the results were very positive. There were some exceptions but we were able to explain them. Since both means and noise were quite low and evenly spead, it takes a lot of people to get a power of 0.80.
Report: “Designing for the User Experience” - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 27
Individual Reflections Lucas Haude In the admission letter I received from the FTC, the UFP competency was one of three competencies that needed some attention in order for me to be eligible for the Master’s program. There was only one possible combination of assignments during this semester that covered all three of these competencies, so that was an easy choice. As this was my first semester at TU/e and ID, I was not entirely sure what to expect from the assignments in general. The first lecture was about the different types of economies or paradigms in which products are developed. Starting with the industrial paradigm that began in the 50’s, leading manufacturers were forced to innovate their products as the market for products from the industrial economy became saturated - this process repeated itself a few times and is currently happening as we are going from a knowledge economy to a transformation economy. There is however still a place for products from previous paradigms; consumers still buy vacuum cleaners and washing machines, but the growth of these markets is limited. This gave me a good idea about how to categorise products according to these paradigms, and illustrated what direction the designs at ID should be going in - either the knowledge- or transformation paradigm. I think it is important not to forget the previous paradigms either: it’s quite likely that a new product that fits the transformation economy still makes use of mass production techniques and should also provide a satisfying user experience. The second lecture provided information on how to conduct user studies in three different ways. Our workgroup selected the cultural probes method to try to find out how students and their parents communicate with the end goal to improve this contact. This method seemed to require the most creativity from both the researchers and the participants, and it yields some interesting results. For the homework assignment I made a comparison of the three methods and their strong and weak points. The results we received from our pilot and ‘real’ cultural probes was interesting. Our workgroup tried to give the participants as little limitations as possible, but it turned out we had left our instructions too open, causing the participants to be confused about what exactly was expected from them. The third lecture was cancelled; it was about the affinity diagram that can give some overview and insight into the large amount of data that can be generated by user research. This method relies on associations by the team members; after reviewing the data each member takes turns to write down words or short sentences that they feel are related to the results. These notes are grouped and given a category name. I think this method is a good way to deal with qualitative data because of the overview it gives. It is important to sometimes go through the data while creating the notes in order not to miss anything; there is an inherent risk of leaving out some important clues. Creating the affinity diagram was useful to give us an overview of the results that we received, but it did not directly lead to a design concept.
Page 28
I missed the fourth lecture due to illness. In this lecture a method called Storyply was introduced that can help designers with creating concepts for a design challenge by envisioning a conflict and a solution. By adopting a different perspective (the end user’s perspective), this method tries to give the designer a new angle on the problem. In my opinion this largely depends on the creativity and imagination of the designer. It can be a useful method if there is a ‘block’ in the design process, where the designer is stuck on finding out what the actual problem (conflict) is or has difficulties with creating concepts. The biggest merit to this method was creating the scenario in which the user interacts with the new product, as it forces you to see the interaction from their view. This can lead to some interesting insights on your product. During the fifth lecture we were instructed on how to conduct a quantitative user study, as opposed to the qualitative user research that was taught in the 2nd lecture. The quantitative user study returns data that can be evaluated with statistical methods, for example to compare two different versions or prototypes of a product. We were instructed to conduct a user study, again by making a choice between three different questionnaire models. Each of them has specific strengths and weaknesses - our group decided to go with the AttrakDiff questionnaire as a base because of the assessment of both pragmatic and hedonistic qualities of the analysed products. The initial results of this questionnaire showed us that phone calls between parents and children are often more fun and interesting than conversations using WhatsApp. The final two lectures dealt with the analysis of the questionnaires. In order for a questionnaire result to be meaningful, it needs to be statistically significant. For every type of dataset, an appropriate test is available. A researcher can upload his results to an analysis program to find out if the results satisfy certain standards. These standards more or less guarantee that a questionnaire result is not influenced by chance and therefore unreliable. Although I had some previous knowledge of statistics, these tests were new to me. The questionnaire we made yielded no statistically significant results; one of the questions was answered so uniformly that we would have needed nearly 1900 survey participants to obtain a significant result. The main take-away here for me was that questions with similar answers require more answers than polarising questions with a large difference between ratings. Looking back on this assignment I feel like I have learned a lot of things that can be applied directly to my other study activities. The first lecture provided a good illustration of different product characteristics and enables me to assess how ‘advanced’ certain products are. It also gave me a better idea of the types of concepts that are appreciated at TU/e - concepts that are innovative and support openness, collaboration and social improvements (amongst others). I also know now how I can conduct user tests, both for quantitative and qualitative results. This will be useful soon in the project I’m participating in for which I want to find out what movements are intuitively used to create percussive music. In general, what I will be doing differently during my next assignments is mainly keeping better track of the feedback that I and our group received during the lectures. It’s now scattered across my digital notes of each lecture and my notebook - in hindsight, it would be more useful to keep a central document for feedback on my assignment work and type out all the received feedback per lecture and subject. Report: “Designing for the User Experience” - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 29
Kevin Lu I’ve chosen this assignment with in mind that I’ve to improve my User Focus Perspective competency. I had no real pre-knowledge about his assignment besides the description written on OASE. I thought this assignment would fit very well with my project about making a game, because it requires a lot of user testing. I already took the opportunity to apply methods which I’ve learnt in this assignment in our project Each week design cases were explained and lots of examples were given during the lectures. The first week the different paradigms were explained and I had quite some difficulty to place them in the right economy. My problem was I took too much in consideration. I was told you should just look at the product when it comes right out of the factory. For example, a smartphone, ignore the apps, additional functions, it’s just a phone and SMS each other. In the following weeks evaluation methods for user experience was covered. We as group had chosen a certain subject and method was chosen and we had to apply each weeks lecture to our subject. I noticed it’s not as easy as it looks like to get valuable information from your users. We tried to be open and let them fill in whatever they wanted as long as it had the right subject, but we lacked depth to conduct better research. In the last weeks, it was more about analyzing the data which we had gathered. Our research appeared to be insignificant because we didn’t apply the right method. All the different methods that were explained were a bit vague to me, we’ve chosen the wrong method . I think you should first apply all kind of methods and understand what each strong and weak points are before you try to apply it in your own research. All in all, I gain more knowledge and awareness about this competency and assignment. I plied various methods in practice and can analyze the data. This elective was fun to follow, but it required a lot of collaboration with your group, also some user tests require time to setup, spread, retrieve etcetera, sometimes a one week delivery time is too short, especially if you have to retrieve the information from multiple users.
Page 30
Charlotte van der Sommen I choose this assignment because I wanted to develop the competency area “User Focus and Perspective”. Since I’m in my B3.2 semester I need to be at a sufficient level for all competencies and I felt that “User Focus and Perspective” needed some extra attention. During the first lecture of this assignment I got insight in a big part of my vision on design. I already use the example “Rural Spark” of Marcel van Heijst within my vision for a few semesters now. In this lecture I figured out about the paradigm transformative economy. This comes very close to what I want to create as designer. For me it is important that my designs support communities that by being a community and working together create behavioral change within people to have a societal impact on the world. With this theory I’m able to create more base for my thoughts. During the assignment if found out about myself that I can be chaotic in what I communicate. When handing out the Cultural Probes it was not very clear for the people what they had to do, due to my poor explanation. I’m too easy going and I think they will just understand it, but that is not the right attitude for doing user tests and also not in other circumstances in life. I learned that I need to take more effort and care in how I present and communicate things. The group work went very fluently in my eyes. Everyone was very motivated and flexible which made the group work very pleasant. Before this assignment I already did the assignment UFP basics, and there I learned how to make and do user tests and I learned about the importance of user testing. UFP basics mainly focused on testing to research the working of you product. During this assignment it became much more clear to me that testing can already be done in the begin phases of the design process and I learned many methods that can help me to do such tests. I also learned how to use the gained knowledge from the test, within concept development part of the design process in order to design for the User Experience. Now I understand how to test the User Experience by researching the important needs of the user and I understand the differences between a qualitative and quantitative user test. Eventually I learned a software program (SPSS) that can help me to verify whether my test is sufficiently. It also gives insight in when I need to use another research method or when I need to survey more people. At the beginning of the semester my level within the competency area “User Focus & Perspective”, was at the point that I had followed one assignment and that have done some small user-tests within my project. Right now I’m able to do and elaborate user test within different parts of the design process. I know how to research for methods to do my test and I know how to research whether my test went sufficiently. I think I just got on the level of awareness within the competency “User Focus and Perspective”. For the last part of the semester I will use my new gained knowledge within my project to get more practice.
Report: “Designing for the User Experience” - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 31
Page 32
References - Cultural Probes method: Gaver, B., Dunne, T., and Pacenti, E. Design: Cultural probes. interactions 6, 1 (1999), 21–29. - Diary studies method: Carter, S. and Mankoff, J. When Participants Do the Capturing: The Role of Media in Diary Studies. Proceedings of CHI 2005 , 899–908. - Contextual inquiry method: Kuniavsky, M. Contextual Inquiry, Task Analysis, Card Sorting. Chapter 8 of Observing the User Experience: A Practitioner’s Guide to User Research (2003), 159 - 174. - Reon Brand & Simonoa Rocchi (2011). Rethinking value in a changing landscape - A model for strategic reflection and business transformation, Philips Design - Creswell, J.W. (2014) Research design. Qualitative, quantitative and mixed methods approaches. Sage, Los Angeles (USA). - Flick, U. Designing qualitative research. Sage, London (UK). - Kjedskov et al, (2004) Using cultural probes to explore mediated intimacy, AJIS special Issue. - A. Vermeeren et al (2010) User Experience Evaluation Methods: Current State and Development Needs, NordiCHI 2010, 521-530 - allaboutux.org - Arhippainen L., Pakanen M., Hickey S., “Mixed UX Methods Can Help to Achieve Triumphs”. Made for Sharing: HCI Stories of Transfer, Triumph & C C 2013 C f & Tragedy, Workshop at the ACM CHI 2013 Conference (Checklist for UX studies) - Vermeeren A., Cremers A., Kort J., Fokker J., (2008) Comparing UX Measurements, a Case Study. Proceedings of the International Workshop on Meaningful Measures: Valid Useful User Experience Measurement VUUM2008 - Zimmermann P.G. (2008) Beyond Usability – Measuring Aspects of User Experience. PhD thesis ETH Zurich, Ch.2, Existing approaches to User Experience evaluation, pp. 17-42 - http://eval.attrakdiff.de/
Report: “Designing for the User Experience” - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 33
Appendix A - Cultural Probes Pilot Cultural Probes Probe 1
Cultural Probes Probe 2
Cultural Probes Probe 3
Cultural Probes Probe 4
Cultural Probes Probe 5
Pictures
Postcard
Icons
Scenario
Express
by: Lucas Haude Kevin Lu Charlotte van der Sommen
by: Lucas Haude Kevin Lu Charlotte van der Sommen
by: Lucas Haude Kevin Lu Charlotte van der Sommen
by: Lucas Haude Kevin Lu Charlotte van der Sommen
by: Lucas Haude Kevin Lu Charlotte van der Sommen
These are the different Cultural Probes
This postcart is probe 2
Page 34
These icons are for probe 3 Cultural Probe Scenario of the conversation
This scenario is probe 4 Report: “Designing for the User Experience� - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 35
Appendix B - Cultural Probes Final
These are the adjustments that are made to probe 2
Page 36
These are the adjustments that are made to probe 3
Report: “Designing for the User Experience� - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 37
Appendix C - User-tests
Page 38
Report: “Designing for the User Experience� - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 39
Page 40
Report: “Designing for the User Experience� - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 41
Appendix D - Results User-tests Question 1 results
 Question 2 results
Â
Page 42
Question 7 results
Question 8 results
Report: “Designing for the User Experience� - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 43
Question 9 results
Question 10 results
Question 11 results
Page 44
Average all questions
The results of both tests are insignificant. We can’t conclude one is significantly better than the other.
Report: “Designing for the User Experience” - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 45
Power Analyze
Question 1 Two different methods to do the power test.
Question 2
Page 46
Question 3
Question 4
Question 5
Report: “Designing for the User Experience” - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 47
Question 6
Question 7
Â
Question 8
Page 48
Question 9
Question 11
Report: “Designing for the User Experience” - Kevin Lu - Lucas Haude - Charlotte van der Sommen
Page 49