Use of Tangible Interfaces to Support a Literacy System in Children with Intellectual Disabilities Janio Jadán-Guerrero1,3, Gustavo López2, and Luis A. Guerrero1,2 1

Doctoral Program in Computer Science, Universidad de Costa Rica, San José, Costa Rica 2 Centro de Investigaciones en Tecnologías de la Información y Comunicación, CITIC Universidad de Costa Rica, San José, Costa Rica 3 Universidad Tecnológica Indoamérica Av. Machala y Sabanilla, Quito, Ecuador {janio.jadan,gustavo.lopez_h}@ucr.ac.cr, [email protected]

Abstract. People with intellectual disabilities should develop reading and writing skills since children. Otherwise, they will have additional difficulties in their daily life activities, as buying in a store, taking medicaments or getting a job. In this paper we present a prototype that uses tangible interfaces and augmented objects to support a literacy method for children with intellectual disabilities. The prototype was evaluated and the preliminary results showed that the system can not only assists teachers in the classroom, but also helps to better manage the interaction with children and non-traditional interfaces. Keywords: Non-traditional interfaces, Literacy, Tangible interfaces, Augmented objects.

1

Introduction

Literacy development in people with intellectual disabilities is a key factor to ensure their integration into society. This paper is focused on Down Syndrome. People with mild or moderate level of the syndrome often have additional difficulties to read and develop daily life activities, such as buy in a store, take medications, take a bus or get a job [1]. Reading and writing are very important skills that people with intellectual disabilities should development since children. Children with intellectual disabilities like others grow up and live in their community surrounded by technology e.g., laptops, tablets, smart phones, smart TVs, videogame consoles and other non-traditional devices [2]. In this scenery, caregivers or teachers try to take advantage of some mobile applications. However, most of the programs and technology aids are designed with language and cultural considerations that vary considerably from the needs and requirements of native Spanish speaking users [3]. Designing technologies for children with disabilities is highly complex due to the diversity of needs involved in their daily lives. However, educational technologies have emerged rapidly in recent years due to the growth of Internet access and mobile devices [4].

R. Hervás et al. (Eds.): UCAmI 2014, LNCS 8867, pp. 108–115, 2014. © Springer International Publishing Switzerland 2014

Tangible Interfaces to Support a Literacy System in Children with Intellectual Disabilities 109

As we know, the purpose of education is to prepare children for life in the real world, and technology helps to strengthen the teaching and learning process, particularly in special education where it plays a significant role in a child´s life [4] [5]. Our goal in this research project is to help children with special needs to recognize and associate real world objects with pictograms or photographs. There are educational methods specially designed to consider the characteristics for teaching children with intellectual disabilities. Troncoso and Del Cerro´s method is one of them. We attempt to apply this method jointly with tangible objects to enhance the literacy process for children with intellectual disabilities [5]. The rest of this paper is organized as follows: Background and related work of tangible interfaces used for teaching reading is presented in Section 2, as well as the method to teach children with Down syndrome how to read. Section 3 describes the system architecture of the implemented prototype. The information obtained from a preliminary evaluation is detailed in Section 4. Finally, the conclusions and future work are described in Section 5.

2

Background and Related Work

Literacy is one of the crucial elements of early childhood education. It starts with parents and other caregivers that begin preparing the child to read with some activities at home [6]. However, due to the diminished intellectual capabilities of children with intellectual disabilities, different ways of learning basic reading and writing skills are required in comparison with other children of the same age [5] [7]. In the next section we present the most important methods used with those children and some computer systems based on tangible objects for support literacy. 2.1

Literacy Methods

Literacy methods can be classified into two categories: synthetic or ascending, that begins with sublexical units such as letters, syllables ending in the word; and global or down, that starts with the word or phrase ending in syllables and letters. The two most widely used teaching methods for reading in Spanish are the syllabic and phonics methods [8]. In the case of children with intellectual disabilities, who have difficulties with syllable recognition there is not a unique method. However, therapists suggest a global reading method. One of these methods was published in the book “Down Syndrome: Reading and writing” by María Victoria Troncoso and Mercedes Del Cerro [9]. The pedagogical approach that the method applies is the discriminative-perceptual learning. This approach focuses on teaching a kid to perceive sounds and relate them to actions or objects. The method emphases in the development of five abilities: memory, attention, association, discrimination and denomination. For each one of them the method is taught with cards and pictures. The purpose of these cards is that the child can relate a word to an image and associate the both concepts. Figure 1 shows some samples of those cards.

110

J. Jadán-Guerrero, G. G López, and L.A. Guerrero

Fig. 1. Troncosso's Method cards for a key and a ball (in Spanish)

2.2

Computer-Supportted Literacy Systems

In the literature review, wee found that in the past five years, some applications w were developed for teaching read ding skills using tangible interfaces. Some of these projeects are mentioned below. E-du box is an educatio onal multimedia platform including a tangible compannion that helps children in readin ng activities. The system has a tangible interface that uuses familiar and everyday livess objects such as wood and tissues. Using a Bluetooth cconnection children interact with w a pen-shaped mouse that vibrates according to soome situations defined by the ed ducator. Feedback is provided by a tangible, interactive and animated e-du agent, who iss able to move and speak to children [10]. Reading Glove is a weaarable RFID (Radio Frequency IDentification) reader tthat engages readers in an audio o-based interactive narrative by allowing them to “extrract memories” from tagged phy ysical objects [11]. T-maze is a tangible prog gramming tool for children. Authors use a camera to caatch the sequence of wooden blocks in real time, which will be used to analyze the sequence and enable the child dren to receive feedbacks immediately [12]. Table Top is a Troncoso o’s method application using inversed projection with a videobeam. Authors use a tan ngible interface that uses objects with external tags reccognized by a camera. The tan ngible interfaces are physical objects that represent the images or words presented to the t children with Down Syndrome [5]. Co-StiCap (Stimulating Collaborative Cognitive Capabilities) is a multi-devvice based on distributed user in nterfaces and games systems. It aims to provide cognitivvely stimulating activities for children c with Attention Deficit Hyperactivity Disorderr. It shows the use of a laptop,, a smartphone and tangible object coded by NFC labbels [13]. RoyoBlocks is a toy set that t provides children an opportunity to develop fundam mental literacy skills. The study y concludes that when a child is playing, they bring to the interaction an array of existting cognitive and emotional associations [7]. Lit Kit is a portable, cyber-physical c artifact supporting children's picture-boook reading. It creates room-scaaled, audio-visual and spatial effects to both contextuaalize language and provide feedb back during dialogical interactions between a child andd an adult reader [14]. Figure 2 shows s the picture of the systems described above.

Tangible Interfaces to Support a Literacy System in Children with Intellectual Disabilities 111

Fig. 2. Tangible computer supported literacy systems

All the systems mentioned offer an environment to practice the educational concepts of learning by doing, which can enhance the efficiency of children’s learning [12] [15]. Children's literacy can be advanced in a tangible, co-creative environment that is both physical and digital. Our solution differs with the presented above in the use of Wi-Fi technology enabling long distance communications and portability. It also differs because we avoid using external tags or signs that can modify the external appearance of the objects. We accomplish this by applying a digitalization and rapid manufacturing process i.e., 3D scanning and printing to redesign the objects and embed the required tags.

3

System Architecture

The system architecture is shown in Figure 3. The system prototype consists of three main components: a tangible user interface that uses RFID recognition, a processing unit, and a mobile user interface that displays the object abstractions (an image and a word). The whole process is based on the Troncoso & Del Cerro’s literacy method. In this section we will provide the details of implementation of the three main components showed in Figure 3. 3.1

System Components

The prototype is composed by three main components: • Object recognition tool. It is a little box that contains an RFID reader and is connected to the computer in order to send the cards to the tablets. • Recognizable object set. It is a group of objects that were redesigned in order to contain an RFID tag that the system can recognize. • Tablets. Tablets or smartphones are used to display the abstract representation of the physical objects. These cards are shown to the children.

112

J. Jadán-Guerrero, G. López, and L.A. Guerrero

Fig. 3. Software Architecture Diagram

The system involves two different users, which are described below. • Teacher/Parent/Caregiver. The main user of the tangible user interfaces and the recognition system. They have to provide the session number to access the processing unit session. They pass objects near the recognition device and the system creates the cards and shows them to the children. • Child. The secondary user of mobile application and physical objects. However they do not interact directly with the processing unit. 3.2

System Workflow

The system's main flow consists of the teacher/parent/caregiver (henceforth teacher) which start the Processing Unit (PU). The PU could be a laptop, a PC or even a server depending on the scale required. The PU hosts the recognition software that determines the object to be presented to the children. It also sends the data to the tablets. The tablets display a card with an image of the object and the word associated with that object. Figure 4 shows an image of a real object presented to the children and a screenshot of the tablet showing the picture and the word. In this case, the label “la bola” refers to the ball in Spanish because the system is being implemented for Spanish speakers. The tablet application is very simple. It consists of two main screens: login and cards. The login screen requires a session number from the teacher. This session number is given to him/her when the recognition system is executed. This number enables many tablets to be connected to the same work session if that is required, for instance, in a class with several children (several tablets).

Tangible Interfaces to Support a Literacy System in Children with Intellectual Disabilities 113

Fig. 4. Left: Physical object showed to the children. Right: Tablet displaying the card with the information associated to the object.

The objects (Tangible User Interfaces) were created using a digitalization and additive manufacturing process, i.e., 3D digitalization and 3D printing of the object, which allowed us to create several augmented objects. This process is required in order to embed the RFID tags into the objects maintaining their physical appearance to avoid any cognitive load in the children. We carry out a performance testing process in order to determine how long it takes from the time when the object is recognized to the time it is displayed on the tablet. For this purpose we used a Wi-Fi network of 36 mbps. With 12 changes between objects we found that the mean time between the object recognition and the display in the tablet is one second and forty-six milliseconds. Considering that the time frame of an object displayed can vary from one to four minutes, a 2 second delay for image change is not significant.

4

Prototype Evaluation

In order to get a first insight of the prototype and with a final user we carried out an interview with four experts, i.e., special education teachers. The evaluation was conducted in two steps. The first step consisted on executing the prototype functionality using all the components, i.e., Laptop, RFID reader, recognizable objects set and two tablets. We explained the functionality of the system and we asked them about the general impression. The first part of the evaluation allowed us to understand the issues of the interface (found by the experts) and helped us to improve the user interface. The second part of the evaluation was a direct interaction of the expert and the system followed by an interview about the user experience. For this purpose we conducted a thinking-aloud test [16]. The interview was recorded and notes were taken during and after the conduction of the test. All the participants agreed that the prototype is useful to enhance the first stages of the literacy process. However, they stressed that the process should be pleasurable and not tedious in order to work. Figure 5 shows the system being evaluated: when a key (left picture) and a ball (picture on the right) are positioned close to the Recognition Tool the corresponding card is displayed on the tablet.

114

J. Jadán-Guerrero, G. López, and L.A. Guerrero

Fig. 5. Functional System: a key and a ball are recognized by the system and the respective cards are created and sent to the tablets

One of the participants in the evaluation mentioned that the objects could help to create new types of interaction for each child, for instance, they can create stories using the objects. Teachers also thought that tangible interfaces emphasize the connection between the body and cognition, facilitating thinking through physical actions. However, they found some problems related with the type of letter and the size of the image. For an introductory phase of reading it is better the font not to be in script. Teachers also state the size of the images and the words should be bigger. The results of the tests served as feedback for an improved version of the prototype. This preliminary evaluation indicates that the interaction design of our prototype needs to be enhanced and further evaluation in the field is required. Structured experiments are required in order to assess the perceived usefulness and usability of the prototype in real scenarios. Is important to mention that most of the participants in the evaluation process pointed the fact that the system could be used to help the literacy process in any person, e.g., elders and children without disabilities.

5

Conclusion and Future Work

The results of the prototype evaluation showed that the system can not only assists to teachers but also helps to improve the interaction with children and non-traditional interfaces. It is important to mention that educational inclusion of children with intellectual disabilities works better when the whole community of classmates is involved in same class activities. A technological system for supporting this process is just a tool people can use, i.e., a technological tool can never replace the teacher guide. We state that, a system with a Tangible User Interface like the one we developed can support teacher and/or parents working with children with intellectual disabilities, especially if the system supports a well-known method –like the Troncoso and Del Cerro literacy method. We want to highlight that our final users are the teacher, i.e., our system supports people who are experts in teaching literacy rather than the children learning it. Depending on the cognitive deficits, more features can be easily implemented. For instance, the card on the tablet can include a sound or video. In the same way, using their finger children can paint over or follow the written word in the tablet to introduce writing. These new features are scheduled in a future version of the prototype and more evaluations and experiments will be conducted.

Tangible Interfaces to Support a Literacy System in Children with Intellectual Disabilities 115

References 1. Ferreras, A., Belda, J.-M., Barberà, R., Poveda, R., Urra, M., García, N., Tito, M., Valero, M.: PDA Software Aimed at Improving Workplace Adaptation for People with Cognitive Disabilities. In: Miesenberger, K., Klaus, J., Zagler, W., Karshmer, A. (eds.) ICCHP 2010, Part II. LNCS, vol. 6180, pp. 13–20. Springer, Heidelberg (2010) 2. Philip, K.: HCI beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces. Morgan Kaufmann Publishers, San Francisco (2008) 3. Al-Wabil, A., Al-Shabanat, H., Al-Sarrani, R., Al-Khonin, M.: Developing a Multimedia Environment to Aid in Vocalization for People on the Autism Spectrum: A User-Centered Design Approach. In: Miesenberger, K., Klaus, J., Zagler, W., Karshmer, A. (eds.) ICCHP 2010, Part II. LNCS, vol. 6180, pp. 33–36. Springer, Heidelberg (2010) 4. Drigas, A.S., Ioannidou, R.-E.: ICTs in Special Education: A Review. In: Lytras, M.D., Ruan, D., Tennyson, R.D., Ordonez De Pablos, P., García Peñalvo, F.J., Rusu, L. (eds.) WSKS 2011. CCIS, vol. 278, pp. 357–364. Springer, Heidelberg (2013) 5. Muró, B., Santana, P., Magaña, M.: Developing reading skills in Children with Down Syndrome through tangible interfaces. In: Proceedings of the 4th Mexican Conference on Human-Computer Interaction, México DF, (2012) 6. Kleiman, J., Pope, M., Blikstein, P.: RoyoBlocks: An exploration in tangible literacy learning. In: Proceedings of the 12th International Conference on Interaction Design and Children, IDC 2013, pp. 543–546 (2013) 7. Kleiman, J., Pope, M., Blikstein, P.: RoyoBlocks: An exploration in tangible literacy learning. In: Proceedings of the 12th International Conference on Interaction Design and Children, IDC 2013, pp. 543–546 (2013) 8. Bentolila, A., Bruno, G.: Learning to read: choosing languages and methods. In: Education for All Global Monitoring Report 2006. UNESCO (2006) 9. Troncoso, M.V., Del Cerro, M.: Síndrome de Down: lectura y escritura. Fundación Iberoamericana Down 21, Santander, Spain (1998) 10. Brotto, A., Pontual, T., Gomes, A., Monteiro, C., Sonnino, R.: e-du box: Educational multimedia with tangible-enhanced interaction. In: Proceedings of the 7th ACM Conference on Designing Interactive Systems, DIS 2008, pp. 139–146 (2008) 11. Tanenbaum, K., Tanenbaum, J., Antle, A., Bizzocchi, J., Seif, M., Hatala, M.: Experiencing the Reading Glove. In: Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction, TEI 2011, pp. 137–144 (2011) 12. Wang, D., Zhang, C., Wang, H.: T-Maze: A tangible programming tool for children. In: IDC 2011: Proceedings of the 10th International Conference on Interaction Design and Children, pp. 127–135 (2011) 13. De La Guía, E., Lozano, M., Penichet, V.: Co-StiCap: Sistema Basado en Interfaces de Usuario Distribuidas y Tangibles Para Mejorar las Capacidades Cognitivas en Niños con TDAH. XIV INTERACCIÓN 2013, pp. 61-68 (2013) 14. Schafer, G., Evan, K., Walker, I., Lewis, E., King, S., Soleimani, A., Norris, M., Fumagali, K., Zhao, J., Allport, R., Zheng, X., Gift, R., Padmakumar, A.: Designing the LIT KIT, an interactive, environmental, cyber-physical artifact enhancing children’s picture-book reading. In: 12th International Conference on Interaction Design and Children, pp. 281–284 (2013) 15. Stull, A.T., Mayer, R.E.: Learning by doing versus learning by viewing: Three experimental comparisons of learner-generated versus author-provided graphic organizers. Journal of Educational Psychology 99(4), 808–820 (2007) 16. MacKenzie, S.: Human-Computer Interaction: An Empirical Research Perspective. Morgan Kaufmann Publishers Inc., San Francisco (2013)