Magical Musical Mat

Aim: Designing a domain-general communicative platform that surfaces the embodied interactions of students in Special Education.

Team Members: Rachel S.Y. Chen (Graduate School of Education, UC Berkeley; Special Education, SFSU); Arianna Ninh (Cognitive Science and Computer Science, UC Berkeley); Rebecca Abraham (Electrical Engineering and Computer Science, UC Berkeley); Betty Yu (Speech, Language and Hearing Sciences, SFSU); Dor Abrahamson (Graduate School of Education, UC Berkeley)

Duration: Spring 2019 – Present

Funding: Barbara White Bequest; Jacobs Innovation Catalyst Grant (winner of Jacobs Design Showcase 2019); Berkeley Center for New Media fellowship

Background: The MMM came to fruition during Fall 2018 in Professor Kimiko Ryokai’s course on Tangible User Interface (TUI) design, where Arianna, Justine, Fang, and Rachel worked towards an interface design that would facilitate joint action and collaboration between people. Inspired by projects that brought people together through touch, we created a musical mat that amplified a variety of touch-based interactions through sound. We saw the potential for this project within the context of Special Education and decided to continue developing the MMM.

Motivation: Although social interaction in learning contexts is available to many people, it is not readily accessible to individuals who may communicate differently, such as non-speaking students on the autism spectrum. The dominant modality for communication in the classroom through representational verbal language may not be the best modality for these students. There is therefore a need to make expressive activity more readily accessible to this population by using the body as a conduit through which interaction can happen. The MMM returns to one of the basic modalities of communication–touch– as a means for people to connect. The MMM removes interactional asymmetry between diverse people and surfaces the basic human capacity to connect with one another in the learning context and beyond.

Publications:

Chen, R. S. Y., Ninh, A., Yu, B., & Abrahamson, D. (2020). Being in touch with the core of social interaction: Embodied design for the nonverbal. In The Interdisciplinarity of the Learning Sciences, Proceedings of the 14th meeting of the International Society of the Learning Sciences (ICLS 2020) (Vol. 3, pp. 1681–1684). International Society of the Learning Sciences.