Sound synthesis is the generation of sounds using algorithms, whether implemented in analogue or digital forms. It is an important application for cinema, multimedia, games and sound installations. It fits within the wider context of sound design, which is the discipline of acquiring, creating and manipulating sounds to achieve a desired effect or mood. Sound synthesis research within the Centre for Digital Music crosses several themes, including Audio Engineering and Augmented Instruments. We seek to uncover new synthesis techniques, as well as enhance existing approaches and adapt them to new applications. With a strong emphasis on performance, expression and evaluation, much of our research is focused on real world applications, empowering users and bringing sound synthesis to the forefront of sound design in the creative industries.
Some of our current and recent research projects include;
- RTSFX (PI, Dr. Reiss, Innovate UK, 2015-16) - This project is concerned with developing and assessing a cloud-based real-time sound effects service, providing a streamlined sound synthesis-based workflow for sound designers.
- Digital Foley Artistry (PI, Dr. McPherson, Queen Mary Innovation, 2015) - The Digital Foley project creates a rapid prototyping environment for game audio, deploying procedural audio models to embedded hardware where they can be performed using physical sensors and capturing expressive parameter trajectories from these performances.
- Physically informed procedural audio (Researcher: Rod Selfridge, 2014 - ) – This research is concerned with developing realistic, controllable, real-time procedural audio techniques for synthesizing sound textures (e.g. wind and rain) using physical models.
- Improved sound synthesis through perceptual evaluation (Researcher: Dave Moffat, 2014 - ) – This research seeks to advance the state of the art in sound synthesis evaluation, in order to identify the performance of synthesis techniques, both using objective and subjective measures, thus leading to deeper insights into the advantages and disadvantages of the different approaches, and guiding work towards improved techniques.
Some relevant, recent publications describing our work include;
- D. Moffat and J. D. Reiss. Perceptual Evaluation of Synthesized Sound Effects. ACM Transactions on Applied Perception (TAP) 15, 2, Article 13 (April 2018).
- R. Selfridge, D. Moffat and J. D. Reiss. Sound Synthesis of Objects Swinging through Air Using Physical Models. Applied Sciences 7 (11), 2017.
- D. Moffat, D. Ronan and J. D. Reiss, Unsupervised Taxonomy of Sound Effects, 20th International Conference on Digital Audio Effects (DAFx-17), Edinburgh UK, September 2017.
- R. Selfridge, D. Moffat and J. D. Reiss. Physically Derived Sound Synthesis Model of a Propeller. ACM Audio Mostly Conference 2017, London UK, July 2017.
- R. Selfridge, D. Moffat and J. D. Reiss. Real-time Physical Model for Synthesis of Sword Swing Sound, 14th International Conference on Sound and Music Computing, Helsinki, Finland, July 2017. (Won Best Paper Award)
- R. Selfridge, D. Moffat, J. D. Reiss and E. J. Avital. Real-time Physical Model of an Aeolian Harp, 24th International Congress on Sound and Vibration, London, UK, July 2017.
- L. Mengual, D. Moffat and J. D. Reiss, Modal Synthesis of Weapon Sounds, AES 61st International Conference, London, UK, February 10-12, 2016.
- G. Durr, L. Peixoto, M. Souza, R. Tanoue and J. D. Reiss, Implementation and evaluation of dynamic level of audio detail, AES 56th International Conference, London, UK, February 11–13, 2015.
- C. Heinrichs, A. McPherson and A. Farnell. Human performance of computational sound models for immersive environments. The New Soundtrack Journal, 2014.
- C. Heinrichs and A. McPherson. Mapping and interaction strategies for performing environmental sound. IEEE VR Workshop on Sonic Interactions for Virtual Environments, Minneapolis, USA, 2014.
- C. Heinrichs and A. McPherson. A hybrid keyboard-guitar interface using capacitive touch sensing and physical modelling. Proc. Sound and Music Computing, Copenhagen, Denmark, 2012. Video Demo
- A. Primavera, F. Piazza and J. D. Reiss, Audio Morphing for Percussive Hybrid Sound Generation, AES 45th Conference on Applications of Time-Frequency Processing in Audio, Helsinki, March 2012.
- A. Zacharakis and J. D. Reiss, An additive synthesis technique for the independent modification of the auditory perceptions of brightness and warmth, AES 130th Convention, May 2011.
- S. Hendry and J. D. Reiss, Physical Modeling and Synthesis of Motor Noise for Replication of a Sound Effects Library, 129th AES Convention, San Francisco, Nov. 4-7, 2010.
PhD Study - interested in joining the team? We are currently accepting PhD applications.
Members
| Name | Project/interests/keywords |
|---|---|
| Berker Banar | Towards Composing Contemporary Classical Music using Generative Deep Learning |
| Marco ComunitĂ | Machine learning applied to sound synthesis models |
| Max Graf | PERFORM-AI (Provide Extended Realities for Musical Performance using AI) |
| Benjamin Hayes | Perceptually motivated deep learning approaches to creative sound synthesis |
| Yin-Jyun Luo | Industry-scale Machine Listening for Music and Audio Data |
| Prof Andrew McPherson Professor of Musical Interaction |
new interfaces for musical expression, augmented instruments, performance study, human-computer interaction, embedded hardware |
| Brendan O'Connor | Singing Voice Attribute Transformation |
| Prof. Joshua D Reiss Professor of Audio Engineering |
sound engineering, intelligent audio production, sound synthesis, audio effects, automatic mixing |
| Eleanor Row | Automatic micro-composition for professional/novice composers using generative models as creativity support tools |
| Shubhr Singh | Audio Applications of Novel Mathematical Methods in Deep Learning |
| David SĂĽdholt | Machine Learning of Physical Models for Voice Synthesis |
| Ningzhi Wang | Generative Models For Music Audio Representation And Understanding |
| Chin-Yun Yu | Neural Audio Synthesis with Expressiveness Control |
- Dr. Andy Farnell (collaborator) – guidance on physically inspired procedural audio approaches and research directions