Students Creating VR Teaching Tool For The Deaf

Purdue University has announced that some of its computer graphics technology students are working on a new project that will hopefully help teach deaf children math through the use of virtual reality. The technology, according to the project's supervising professor, Nicoletta Adamo-Villani, aims to offer a means to help break down some of the barriers deaf children experience. Students on the project are working in Purdue's Envision Center for Data Perceptualization with high-tech cameras and "cybergloves" that can translate body and hand motions into digital images. Using this technology, a user can interact with virtual characters such as a cartoon rabbit, robot and pig in order to learn to use sign language. According to the university, users wear lightweight stereoscopic glasses so the virtual reality images of the characters appear three-dimensional. A device monitors the student's head position so the environment is consistently redrawn to match the user's perspective. A wrist tracker and telemetric "pinch gloves" monitor the student's hand and finger movements allowing interaction with the virtual environment and prompting responses from the characters. The university offered up this example of how this could be used: “In a virtual candy store environment the student communicates to the storekeeper in sign language, some of which is specific to mathematics. Pinch gloves allow students to count candies and to add and subtract by putting candy on or off the counter. The task can be repeated over and over at the student's own pace while providing consistent and understandable feedback.” The virtual reality program is designed to provide early elementary school age students with disabilities with a number of active, individualized learning conditions, such as the ability to control their environment, engage in learning activities at their own pace, repeat activities as needed, see or feel items or processes in concrete terms, and practice daily living tasks in a safe and barrier-free environment, as well as offer the students motivation to succeed. The program is being designed to overcome the barriers deaf children experience in learning math skills. Those barriers include delay in reading comprehension, parents' inability to convey math concepts in sign language, as well as a general difficulty in taking advantage of supplemental learning opportunities, such as television shows and dinner-table conversation. Adamo-Villani hopes by reducing the impact of these barriers, the program can help increase the number of deaf students who go on to study science, technology, engineering and mathematics. "As a leading educator in these fields, Purdue is dedicated to increasing access for deaf students," Adamo-Villani said. "Learning is the development of one experience into a new experience," added Ronnie Wilbur, professor and chair of Purdue's audiology, speech sciences and linguistics department who has served as a consultant to the College of Technology during the project. "Immersive learning environments such as this are more effective than traditional computer software." According to Adamo-Villani, research shows that humans process visual information 60,000 times faster than textual information and that an eight-week virtual reality program can improve student math skill scores by 16 percent. She said that learning enthusiasm remains even after the novelty of virtual reality fades. "Our mission is to be a worldwide leader in virtual learning environment development," said Laura Arns, the associate director of the Envision Center who is helping develop this virtual reality application. "We want to advance industry-leading concepts, software and services while fostering an environment of empowerment, creativity and commitment." Because research shows that children prefer bright colors, student programmers are working to ensure that the program is as colorful as possible. Also, certain colors increase alpha waves directly linked to awareness and improve students' attention span and elicit emotional responses. Specific examples in the Purdue program include shelves lined with jarfuls of hard candies and Chef Pig's icing- smeared apron. "Fluid, non-mechanical motion is fundamental to learning sign language effectively," said Edward Carpenter, the graduate research assistant working directly with the five undergraduate programmers who have dubbed themselves "Dented Can, LLC." "That's why we have invested our efforts in developing natural gestures that are appealing to children." Undergraduate David D. Jones never expected to be mastering complex skills such as environment modeling, character modeling and rigging, motion-capture data application and programming for interaction in 3- D space so early in his academic career. He said he is excited that his work will be utilized for years to come to help others learn - a prospect that he said far outshines any class project grade he will receive. "I truly hope that this program plants some of the seeds necessary to develop good math skills in those children," said Jones, who continues his computer graphics training as a graduate student at Purdue fall semester. "Who knows, maybe someday they'll be as fortunate as I was and get to apply those skills to something interesting and worthwhile." Gifted children attending academic camps at Purdue came to the Envision Center this summer to test the system and provide feedback. In the fall, a portable version of the virtual reality application will be taken to Indianapolis and introduced to classrooms at the Indiana School for the Deaf. "Three years of inventive, collaborative work have created a tool I am eager to see used to open new horizons for deaf students," Wilbur said.