Accessible, Expressive Speech Synthesis
IntroductionSensorium AI is a research initiative dedicated to expanding the creative possibilities of voice expression for minimally-verbal users, developed in partnership with the Sensorium Ex opera. It explores real-time speech synthesis and voice modulation, empowering users to personalize and express themselves beyond the constraints of contemporary Augmentative and Alternative Communication (AAC) devices.
…coming soon…
…coming soon…
Project Director / Software Designer
R. Luke DuBois, NYU Tandon School of Engineering
Sensorium AI Research Team
Mark Cartwright, New Jersey Institute of Technology
Danzel Serrano, Michael Clemens, NJIT
Cameron Churchwell, University of Illinois Urbana-Champaign
Max Morrison, Nathan Pruyne, Caedon Hsieh, Bryan Pardo, Northwestern University
Sensor Hardware
Eric Singer, Researcher-in-residence, NYU Tandon
Andrea Lauer, Designer
Moira Zhang, Alex Parent, Jason Wallach, NYU student researchers
Accessibility Research and Interaction Design
Amy Hurst, Director, NYU Ability Project
Lauren Race, Accessibility Design Researcher, NYU Ability Project
Gideon D’Arcangelo, Raj Patel, ARUP
Michael Coney, Spandita Sarmah, Apoorva Avadhana, Isabella Rodriguez, NYU student researchers
VisionIntoArt (VIA) Impact Lab Fellow for Sensorium AI
Daniella Omuero
Exemplars
Jakob Jordan, Kader Ziouehe, Jessica Frew, Owen Atkins, Thomas Ellenson, Jacob Yoon Egeskov Nossell, Martin Merrild
Vocal Capture for Emotion
Wayne Bennett, Amanda Morrow, Kendal Cafaro, Tony Torn, Saidu Tehan-Thomas, Remsen Welsh, Orlando Grant, Noah Sanders, Michael Tran, Spencer Evett, Bianca Rogoff, NYU Tisch Graduate Acting
Sensorium Ex Creative Team
Paola Prestini, composer and Sensorium Project Director
Ras Dia, creative producer
Brenda Shaughnessy, libretto
Jerron Herman, Jay Scheib, directors
Sensorium AI was commissioned by VisionIntoArt in collaboration with the Sensorium Ex opera.
Sensorium Ex is developed and presented with support from the Ford Foundation’s Creativity and Free Expression Arts and Culture (CFE A&C) program, the Mellon Foundation, the Doris Duke Foundation’s Performing Arts Technologies Lab, Jill and Bill Steinberg, National Sawdust, the Achelis & Bodman Foundation, and New Music USA.
The commissioning of Paola Prestini for Sensorium Ex received funding from OPERA America’s Opera Grants for Women Composers program supported by the Virginia B. Toulmin Foundation. Additional commissioning support was provided by the Allen R and Judy Brick Freedman Venture Fund for New Music. Additional production support was provided by the National Endowment for the Arts and Elizabeth Madigan Jost, with development support from the American Academy in Rome, Artscape Theatre Centre - Cape Town, The Shed, Performance Space New York, the REACH at The John F. Kennedy Center for the Performing Arts, Washington National Opera, and the New York State Council on the Arts. Sensorium Ex is also presented with support from Creative Capital Foundation for Paola Prestini.
Institutional support for Sensorium AI at New York University has been provided by the NYU Ability Project; Integrated Design & Media (IDM) and NYU Tandon @ The Yard, both at the Tandon School of Engineering; the Music & Audio Research Lab (MARL) at the Steinhardt School of Culture, Education, and Human Development; and the Grad Acting program at Tisch School of the Arts. Programming related to Sensorium Ex has been presented with the support of the Center for Disability Studies and the Center for Ballet and the Arts.