Pictocom

Pictocom Graphical Interface for Pictogram Comunication Devices In collaboration with the Swedish Institute for Special Needs Education (SIT)

Benjamín Dueñas Figueroa - bedu0002@student.umu.se Umeå Institute of Design

Index Summary ..................... 1 Introduction ................. 2 Research ..................... 3 Design Process ........... 6 Design Proposal........... 11 References .................. 14

Summary In this project were studied individuals with cognitive disabilities and their relation with an alternative way of communication based on pictograms.

A graphical user interface for communication is developed as an application for a mail system that uses the pictograms as a main language.

The challenge was to understand how the individuals with cognitive disabilities and caregivers understand and use pictograms and other systems for communication in their daily lifes. caregivers, linguistics and experts were interviewed as part of the ethnographic research to obtain information.

The findings indicates that there is a big group of people with cognitive disabilities that includes: down syndrome, autism, and arspberger syndrome, and that every person has different and specific skills that need to be treated individually, for that reason customization becomes a very important concept for this project.

Paper and digital prototypes were done to be tested by the users, and the comments on them seems to be promising.

The final proposal is focused on the grammatical structure of a language and how can be applied to the mail system and is an approach on how can be solved the problem of sending digital mails by structuring coherent sentences in an easy way for the users.

1

Pictocom

Introduction This project is about the development of a graphic user interface for cognitive disabled people and an exploration on how could be developed a method for easily expressing coherently and grammatically structured messages by the users, using pictograms.

People with cognitive disabilities have many difficulties on reading, speaking and writing and with the help of the new technologies as the internet and new electronic devices, can be helped to be more independent enhancing their communication capabilities.

Pictograms are used as an alternative way of comunication, they replace written language with images, currently only used to reference objects, persons and situations of the real world, but not to form sentences.

As a method to obtain information interviews with caregivers, teachers and liguistics were done, literature survey was researched, and paper and digital prototypes as well as sketches of the ideas were used as techniques to develope the proposal.

The project has been carried in collaboration with the Swedish Institute for Special Needs Education (SIT).

2

Pictocom

Pictograms

Pictograms are abstract representations of the daily life concepts, objects, actions and situations that are shown as white symbols over a black background.

They are used as alternative method for communication that has to be learned and practiced in order to obtain knowledge since it is the main tool for communication for cognitive disable people who has great difficulties on reading, writing and speaking.

Pictogram Problematic The pictogram database comprise 1400 pictograms, however this we found that this database do not satisfy at all the communication needs of the user due to the small variety of them against the great and increasing amount of the real world concepts, objects, activities and situations that every single user can manage.

Abstract Vs Real The findings on the research let us know also that the meaning of an abstract pictogram is not always clear for the user, in fact, for many of them are confusing or just memorized but not really understood. “Some of them see the separated head of the human representations as snow balls�.

3

Research

Personal or individual booklets are used by each user, they contain their personal vocabulary and interests depicted on pictograms, mostly the caregivers put on them whatever works, a pictogram, an image of other existing systems or a photography that can be usefull to express what the user wants to say.

This object is for many of them an avatar or personal object that is very important for them. “Dan is very proud of his notebook, and he wants to say to you that he can borrow it to you for a while�

- explained a caregiver from one of the specialized schools for cognitive disabled kids.

Every user has his/her own vocabulary based on images from different systems, because every one is totally different from each other, they have different skills and interests so every notebook is customized for every user by the caregivers.

Grammatical Structure of Pictograms Pictograms are used currently to relate single concepts as: objects, actions or situations to refer them to ones of the real world. They communicate only the idea of a pictogram, but they don’t use any grammatical structure to form sentences as in any spoken language.

4

Pictocom Pictocom

Graphical Interface for Mailing Pictograms The current graphical interface available on the web is shown in the image below. We found some functionality problems on it but I have decided to focus on three of them, the pictogram organization, the grammatical structure to form sentences that is abscent on this system, and the understanding of abstract pictograms.

Research

Navigabile - Graphical Interface for sending messages based on the PCS system 5

Pictocom

Design Process

Brainstorming sessions During the work in group, brainstorming sessions were done to obtain ideas about a specific problems of the project.

Three problems were chosen: Sending messages gramatically well structured Easy understanding of the Pictograms Organization of the Pictogram Database.

6

Pictocom

Design Process

Analisys of helpful details used on the user’s booklets helped to visualize possible solutions for the grammatical expressions.

The combination of a verb + an action is shown as a powerfull and very easy way for the user to express an idea that is grammatically correct.

Sketches and Mind Maps To visualize first stages of the ideas, this were sketched on paper.

7

Pictocom

Design Process

Second Stage of sketches On this stage, I put more effort on detailing the process of sending a grammatically well structure message with pictograms, on a graphical interface.

Some options are explored as the bubbles for the message visualization, and the organization of the pictogram library.

Possible pictogram combinations to form the concept of “single button expressions� are explored as well.

8

Pictocom

Design Process

The Double Sided Card The double sided card is a proposal to enhance the pictogram application and understanding.

It consists on relating the abstract pictogram with a real and specific image of the concept on the back side of the pictogram.

This proposal can be explored on a series of games like the memory game, in order that the user easily relates the abstract image of one concept to the object.

On the backside the user can select the shoes he/she wants to use for a specific situation.

Prototype Testing Digital and paper prototipes were shown to caregivers and teachers.

We obtained information from them, about the user tasks, the different systems used, and the problems with them, as a bridge for communication with cognitive disabled people.

Comments on the proposals seemed to be very possitive, although a second stage of prototype testing with the user it is required.

9

Pictocom

Inspiration

10

Pictocom

Design Proposal

The Design Proposal The final proposal of the graphical interface, it’s focused on the message composing.

It is divided on two main parts: the writing section, and the pictogram library or personal notebook as it is shown in the image.

The writing section is represented by a normal sheet of paper that can be customized for every user.

The Pictogram library is represented by the same notebook concept that the users have to express their ideas, an article that is important for them and it is easily recognizable.

This concepts were applied with the intention of being empathic, using real references from the world.

11

Pictocom

Design Proposal

It is shown in the main interface also, a picture of the sender and the picture of the person who receives the pictogram e-mail, the photography of the sender appears on a bigger scale that the person who receives the message to indicate the sensation of proximity and gives importance to the sender.

A single pictogram of happiness is located also in the left - bottom corner as an option to express emotions and the most used sentences or phrases.

When the user clicks on a pictogram it shows a sub-menu, that is the main proposal of the interface. The menu acts as a palette of possible options that are customized for each single pictogram, depending on wich pictogram you select.

12

Pictocom

Design Proposal On the sub-menu are represented the possible options a pictogram can have as: I want, I don’t want, yes, no, Do you want?, and the possibility to use only the pictogram selected.

By clicking on the pictogram of pizza located over the blue section, it is shown the backside of the card, where are displayed real images of the same concept.

13

References Dawe, M. 2006. Desperately seeking simplicity: how young adults with cognitive disabilities and their families adopt assistive technologies. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Montreal, Quebec, Canada, April 22 - 27, 2006). R. Grinter, T. Rodden, P. Aoki, E. Cutrell, R. Jeffries, and G. Olson, Eds. CHI ‘06. ACM, New York, NY, 1143-1152.

DOI= http://doi.acm.org/10.1145/1124772.1124943.

Lewis, C. 2005. Bridges for the mind: opportunities for research on cognitive disabilities. In Proceedings of the 7th international ACM SIGACCESS Conference on Computers and Accessibility (Baltimore, MD, USA, October 09 - 12, 2005). Assets ‘05. ACM, New York, NY, 1-1. DOI= http://doi.acm.org/10.1145/1090785.1090786.

Lewis, C. 2005. HCI for people with cognitive disabilities. SIGACCESS Access. Comput. , 83 (Sep. 2005), 12-17. DOI= http://doi. acm.org/10.1145/1102187.1102190 Lewis, C. 2006. HCI and cognitive disabilities. interactions 13, 3 (May. 2006), 14-15. DOI= http://doi.acm. org/10.1145/1125864.1125880.

Sevilla, J., Herrera, G., Martinez, B., and Alcantud, F. 2007. Web accessibility for individuals with cognitive deficits: A comparative study between an existing commercial Web and its cognitively accessible equivalent. ACM Trans. Comput.-Hum. Interact. 14, 3 (Sep. 2007), 12. DOI= http://doi.acm.org/10.1145/1279700.1279702.

Danielsson, H. 2001. The use of digital pictures for people with cognitive disabilities. In CHI ‘01 Extended Abstracts on Human Factors in Computing Systems (Seattle, Washington, March 31 - April 05, 2001). CHI ‘01. ACM, New York, NY, 253-254. DOI= http:// doi.acm.org/10.1145/634067.634217.

Lepisto, A. and Ovaska, S. 2004. Usability evaluation involving participants with cognitive disabilities. In Proceedings of the Third Nordic Conference on Human-Computer interaction (Tampere, Finland, October 23 - 27, 2004). NordiCHI ‘04, vol. 82. ACM, New York, NY, 305-308. DOI= http://doi.acm.org/10.1145/1028014.1028061 Small, J., Schallau, P., Brown, K., and Appleyard, R. 2005. Web accessibility for people with cognitive disabilities. In CHI ‘05 Extended Abstracts on Human Factors in Computing Systems (Portland, OR, USA, April 02 - 07, 2005). CHI ‘05. ACM, New York, NY, 1793-1796. DOI= http://doi.acm.org/10.1145/1056808.1057024 Dawe, M. 2007. Understanding mobile phone requirements for young adults with cognitive disabilities. In Proceedings of the 9th international ACM SIGACCESS Conference on Computers and Accessibility (Tempe, Arizona, USA, October 15 - 17, 2007). Assets ‘07. ACM, New York, NY, 179-186. DOI= http://doi.acm.org/10.1145/1296843.1296874.

Belinchon, M.; Gortazar, P.; Martinez-Palmer, M.; Flores, V. y García-Alonso, 5th Congress Autism-Europe: La Comprension verbal en el autismo: Alteraciones morfosintacticas, semanticas o conceptuales?. http://www.autismo.com/e107_plugins/content/content. php?content.23 Ravi Nigam; a, Ralf W. Schlosser; b, Lyle L. Lloyd; c: Concomitant use of the matrix strategy and the mand-model procedure in teaching graphic symbol combinations. A) Duquesne University, Pittsburgh, PA, b) Northeastern University, Boston, MA, c) Purdue University, West Lafayette, IN, USA. Web: http://epost.pictogram.se/flash.asp http://www.studioaka.co.uk/html/index.html http://www.navigabile.eu/ http://www.cricksoft.com/us/products/clicker/default.aspx http://www.autismo.com/e107_plugins/content/content.php?content.23 http://www.ablelinktech.com/_handhelds/pocketdiscovery.asp http://www.pictogram.se/english/

14