Postboard: Free-form Tangible
Messaging for People with Aphasia
(and other people)
Abstract
In this paper we report the design of a communication system for people with aphasia. It consists of pairs of physical whiteboards that are connected over the internet. This allows users to combine any personal language skills with visuals and already existing materials at hand. The design has been based on feedback and evaluation sessions with aphasics and their therapists. The user test results show that the design provides a low entry barrier and enables relaxed conversations.
Keywords
Aphasia; communication; tangible interaction; inclusive design; assistive technology
ACM Classification Keywords
H.5.2 [Information Interfaces and Presentation]: User Interfaces - Interaction styles
General Terms
DesignCopyright is held by the author/owner(s). CHI 2012, May 5–10, 2012, Austin, TX, USA. ACM 978-1-4503-1016-1/12/05.
Abdullah Al Mahmud
Faculty of Industrial Design Engineering
Delft University of Technology Landbergstraat 15
2628 CE Delft The Netherlands a.almahmud@tudelft.nl
Liza Blummel
Department of Industrial Design Eindhoven University of Technology Den Dolech 2 5600 MB, Eindhoven The Netherlands Sander Dijkhuis
Department of Industrial Design Eindhoven University of Technology Den Dolech 2 5600MB, Eindhoven The Netherlands S.Q.Dijkhuis@student.tue.nl Iris Elberse
Department of Industrial Design Eindhoven University of Technology
Den Dolech 2 5600MB, Eindhoven The Netherlands
Introduction
Aphasia is a communication disorder that is caused by brain injury or trauma. Persons with aphasia often suffer from additional disorders like paralysis, apraxia, hemianopsia, agnosia, dysphagia, dysarthria,
dyscalculia, epilepsy and problems in expressing emotions [2]. Several solutions were created to support the user group such as non-technological and
technological solution [1, 3, 5, 7]. An example of non-technological solution is Gespreksboek (‘conversation book’) by ‘Afasie Vereniging Nederland’ (afasie.nl), which is a book of categorized words illustrated with icons that help to express. In the PCAD project (Portable Communication Assistant for People with Acquired Dysphasia), the application TouchSpeak was developed for PDA devices [6]. It helps with face-to-face communication. The interface-to-face is highly
configurable and allows a therapist or partner to set up custom pages for different situations. A simpler variant for the iPhone and iPod is Sizzit, which shows
configurable lists of icons. Tapping them shows the word and reads it out loud. Another interesting program is Talksbac, also described in [6]. It helps during conversations to formulate sentences by what will be relevant to discuss in a certain situation with a specific person.
Motivation
An advice from the World Health Organization as mentioned in [6]: ‘designers should focus on the levels of activity and participation of patients instead of on their impairments’. Most of the aphasia products are not very attractive. Some of are designed not to look too ‘medical’ like and thus aphasics can use the same system as non-aphasics. We found that most existing solutions lack depth, because they are entirely
icon-based. This makes the device or application often very limited and hard to apply to real-world situations and therefore these devices often fail to serve their purpose [5].
Our goal is to create a product that helps them to share experiences and communicate with help from pictures, tickets, flyers, notes and drawings, because according to [5] “images do overcome the shortcomings of icons in breadth and depth and therefore may have the potential to support the vocabulary needs of our target population.” This shifts the focus from what they can’t do anymore to what they are still able to do and returns a part of the control over communication. We find it important to design a product that enhances the daily life of aphasics by helping them to keep in touch with relatives and friends on a daily base, because “daily communication not only enriches peoples’ lives but also expands their vocabulary and enhances their language skills” [5]. Research shows that aphasics find their life to be successful when they are able to maintain meaningful relationships [2].
In the following section we describe postboard.
Postboard can be seen as a shared whiteboard. Sending messages is done through interaction with traditional tools like pen and paper. Because it is so flexible, it can be used by people with various types of aphasia. It is designed to appeal and be useful to a larger group of elderly people who might prefer traditional means of communication above computer GUIs.
Design process
Throughout our design process, we had a strong focus on exploration and validation activities.
User requirements. Early in the process, we wrote down
the following requirements. They were based on our literature research, the video interviews of people with aphasia that we had watched and on our initial ideas. a) to use the product you should not have to be able to talk or read, b) the device/service can’t have too many buttons, in order to avoid putting on too much labels which would require reading, c) we would like the user group to be able to use the product to share
experiences, because aphasia patients do not have this skill anymore due to their loss of use of language.
Getting empathy and understanding of the user. we
found it important to find ways to get more
understanding and empathy for the user group. One way to do that was to try communicating without words for a while: this way we felt how hard it was and where the problems where.
Idea generation. We have used several methods to
generate many ideas during brainstorming. In our ‘classical brainstorms’ we sat around a large paper sheet on which we wrote down our ideas. We used this several times to communicate ideas we had ourselves, combine them and improve aspects of them. A common theme in our first ideas was replacing language with something else. For example, we thought about using music to communicate emotions with a jukebox. Something that we mostly used to test and improve ideas was acting them out. An idea that we had was about a camera that allows for emailing photos. By acting out hypothetical situations, we thought about how an aphasic might use this device and how other people would interpret photos. Two more focused methods that we didn’t expect would work appeared to help us get good ideas that we used further on. First we
tried to come up with the most absurd ideas possible, to adapt it to a more realistic one. Here we got the idea to create a labyrinth for aphasics, where a non-aphasic partner would help finding the way out by a hints-like game of non-verbal communication.
Idea selection & evaluation. Four things influenced what
ideas we would continue with: our requirements and design brief, feedback from a user interface expert and other design students, results from trying them out by acting them out and creating paper prototypes and our personal preferences. We chose to elaborate on several ideas individually. They were chosen after discussing them and voting on them. The ideas we didn’t continue with, we wrote down the good and bad aspects. The idea that eventually continued was Postboard. This is a bulletin board, shared with someone else over the internet. The board is a computer screen with a camera directed to it. Everything the user puts on the board (for example a photo or sticky note with text on it) is photographed and sent to the contact’s board, and vice versa. Because the user can decide what to put on the board, he chooses the level of language that is used and creates his own way of communicating. Also when used as a bulletin board, there is not much pressure to respond quickly so aphasics can take the time to do that.
During our first two visits to local aphasia centres we presented our key concepts with drawings and
discussed them. There were three people with aphasia: two men aged around 40 and a woman aged 50+. We presented the concepts using big drawings. The Postboard concept was selected for further development because it had the most potential. Therapists and design experts found it promising figure 1.The initial setting of
Postboard, that was used to gather feedback from other students and exparts (top); the final prototype with its visible hardware, exhibited in a domestic context (bottom).
because it allowed for rich communication with a verbal level fit to the users’ abilities. The other two key concepts appeared to have practical issues that were hard to overcome. One was based on a new kind of emotion language that yet to be invented, and the other was designed to be used at public. Interestingly, during the first meeting, the aphasics liked one of the other concepts better. During the discussion,
misunderstandings arose about what the concepts entailed. For example, since Postboard was only illustrated using a drawing with a newspaper article as example content, the aphasics got the impression that the system would always use lot of text.
Prototype description: Postboard
We chose to continue with the Postboard concept. We thought this was a strong concept because it allows for rich communication, is more intuitive and feasible. A Postboard device is an always-on computer with an integrated matte screen, a high-resolution camera directed towards it and a proximity sensor. It has a frame made of dark grey metal. On the bottom are ten ‘fingers’, made of wire with silicone on the outside. Those can be bended to hold tools like a marker and sticky notes. The device should be hung on to a wall at home.
The display is 24”or larger. It uses ‘e-paper’ technology [4], using color electrophoretic display technology (which is currently being developed by various companies). This can reflect from external light
sources, so no backlight is needed and it’s easy to read from. E-paper doesn’t require much energy when kept turned on, and its low refresh rate is no problem for this concept. When the proximity sensor measures that a user is close to the device, possibly changing what is
put on to the screen, a picture is taken of the screen every five seconds. At those moments, nothing is displayed on the screen for a fraction of a second, so that only the things written and pasted onto it are visible. Each device is connected to the internet using a Wi-Fi connection with the home router. It
communicates with a central Postboard server. This is used to send photos made by the camera and receive those of other Postboard devices. The internet server does automatic image editing to improve quality and sends the photos to the right devices. Devices can be connected to each other to form ‘clusters’. Generally these will contain two devices that communicate with each other. When someone’s device has sent in a new picture, it is shown immediately on his contact’s devices. Some coloured glow is added along the borders of the screen, at the location where new messages are found.
Scenario of use
This scenario should make clear how we envision Postboard being used. “John has aphasia. He has problems formulating sentences and talking is difficult for him. He can read and write simple words with some effort. Today he has been to the movies and wants to let Jane know. She is a friend of his who doesn’t have aphasia but has got a Postboard device. John walks to the Postboard device on the wall of his living room. From one of the silicone fingers he grabs a piece of tape, and tapes the ticket to an empty part of the screen. He walks away to do something else. When he returns after 20 minutes, he sees a coloured glow on the screen and part of a yellow sticky note, behind the ticket. To read the text on it, he takes the ticket off the screen. With some effort he can read Jane’s message: ‘How was the movie?’ He grabs a sticky note that is figure 2. A prototype put in the
working space of students in order to elicit reactions and learn about the ways ofcommunicating through Postboard(top); a stylized prototype exhibited at ID'11 during the 2011 ‘Dutch Design Week’
held between two of the silicone fingers, a pen from another one and writes down: ‘Funny’. He sticks it to the board next to Jane’s note and walks away again.”
Versions of the postboard prototypes
We created several prototype setups for various purposes. The first one is with two laptops and external cameras. This allowed for a quick setup and testing at the aphasia center. To keep our laptop screens safe, we attached transparent paper to it. The second one was publicly fixed next the IDcafé at our building. We used old university computers with 17” CRT screens and hided them behind cardboard walls. The webcams were fixed so that we would only have to configure the perspective correction once. For the final prototype we have created two MDF walls with frames that resemble our final design. Behind those, our laptops are placed connected to 24”LCD screens. The webcams are fixed to the top of the wall using foam holders.
Evaluation
The user evaluation was performed during a
conversation group session at the local aphasia centre and took 1-2 hours. It was started by testing the prototype and concluded with a discussion. It was set up in an informal way, because this fit well in the character of the conversation group session of persons with aphasia. The prototype system consisted of two laptops in the same room with a piece of cardboard to separate them. Sticky notes, markers, newspapers and tape were given to the users in order to put things on their Postboard computers. Two people with aphasia used one computer, and one person with aphasia used the other. The team walked around with a therapist in order to assist the users in composing or interpreting messages and to take notes and pictures.
The goal of the final evaluation was to validate three assumptions about Postboard: namely that (1) it is easy to use, (2) it is enjoyable to use, and (3) it is a rich enough medium to share stories and react on each other. This was measured by observing the users as they interacted with the prototype and discussing their experiences with the users and the therapist. Overall the impression was positive. After an initial slow start where the users were not yet sure what to do, they soon took initiative to send new messages and showed fun doing so. During the discussion the therapist and the aphasic individuals agreed that Postboard would be a useful medium for communication. The aphasics indicated that they would want to use it to
communicate with other aphasics, friends and family. It was interesting to see how a conversation developed, involving personal stories and pieces from the newspaper. Users started sending jokes. Using the Post-it notes they used the same kind of symbols as during a regular conversation group session. This reaffirmed the assumptions that were made in the design process and suggest that the design goals are reached.
A point of improvement is how users learn how to interact with the device and what kind of messages to send. Improvements could be made to the device to provide clearer affordances or 'first use' guidance. The device may also make suggestions about what kind of messages can be sent. The speech therapist suggested that pre-printed notes might be used, like from the ‘conversation book’. However this could also be too impersonal. Those pre-printed notes could also be made per aphasic together with a therapist. figure 3. The setup that was
used during the final user test session (top); snapshot from the user test session where a user prepares a sticky note to put on his Postboard device.
Discussion
The concept of the Postboard had the most potential and it was clear that we would like to develop this concept more. Though we were still in a conceptual phase we were very focused on the practical problems of the concept. It would have been better to make notes of the practical problems, but postpone solving them to a more appropriate point later in time. During our first visit to the aphasia centre we had noticed that a larger part of the people with aphasia have a solid anxiety against technology, especially computers. Therefore we decide that we would try to make Postboard look as less like a technologic device as possible. We first tried this form with the technological prototype we made for the IDcafé at the university. We hide all parts of the computer expect the front view of the screen and we soon noticed that passers-by knew how it worked without any explanation. Before we decide on the final shape we tested the prototype on our laptops with aphasics in the aphasia centre in a local aphasia center. This turned out to be a really successful.
Conclusion and future work
Postboard was designed to allow for richcommunication over the internet using little digital user interface. The user evaluation and expert feedback strongly suggest that Postboard provides a viable way of communicating for people with aphasia. In future we would like to explore how use of pre-printed cards and other ways of helping with communication through Postboard. Currently Postboard is completely free-form, but there seem to be opportunities for pre-defined content or other assistance with composing messages. The device needs to be developed in a way that it is
looks less like a computer screen with an obtrusive camera.
Acknowledgements
We thank Flip Ziedses Des Plantes for his guidance and feedback during the project, and René de Torbal for his help with getting the required materials for the
prototype, and Robin Brouns and Chris Hoedemakers for their advice during the design process.
References
[1] Al Mahmud, A., and Martens, J.-B. 2010. Re-connect: designing accessible email communication support for persons with aphasia. In CHI EA '10. ACM, New York, NY, USA, 3505-3510.
[2] Brown, K., Worrall, L., Davidson, B. and Howe, T. 2010. Snapshots of success: An insider perspective on living successfully with aphasia. Aphasiology, 24-10. Psychology Press, London, England, 1267-1295. [3] Churchill, T. 2010. What are Lightwriters? Retrieved January 1, 2011, from http://www.toby-churchill.com/en/content/lightwriters-0
[4] Electronic paper. 2011. Retrieved on January 2, 2011, from ttp://en.wikipedia.org/wiki/Electronic_paper [5] Ma, X., Boyd-Graber, J., Nikolova, S. and Cook, P.R. 2009. Speaking through pictures: images vs. icons. In Proceedings of Assets ‘09. ACM, New York, NY, USA, 163- 170.
[6] Sandt-Koenderman, M van de. 2004. High-tech AAC and aphasia: Widening horizons? In Aphasiology, 18. Psychology Press, London, England, 245-263. [7] Wolf, T. de, Gooren, D.W.H., and Martens, J.B. 2008. Context aware documenting for aphasics. In Proceedings of Assets ‘08. ACM, New York, NY, USA, 287-288.
