PhD " Advanced neural audio coding for mono and stereo signals " F/M

Twoja rola

Your role is to carry out a PhD work on the subject: “Advanced neural coding for mono and stereo audio signals”. 

Overall context and problem statement
Audio compression (or audio coding) is a field that originated in source coding, with a long history marked by the development of numerous codecs, some of which are well known to the general public, such as MP3 or AAC for music transmission or storage.

In recent years, the field of audio coding has been shaken up by deep learning technologies. Artificial neural networks make it possible to achieve very low compression rates.
As a result, a new generation of multimedia signal compression methods has emerged, based on deep learning. Auto-encoder architectures based on Generative Adversarial Network (GAN) learning give very good results, with codecs such as SoundStream, EnCodec, or Descript Audio Codec (DAC). Other approaches, such as diffusion models, are also being investigated.

Current neural audio codecs are essentially mono. Compared with “traditional” codecs, they are generally much more complex (in terms of computational resources), requiring very significant storage (on the order of 10 to 80M parameters, for example).

Scientific objective – expected outcome and challenges to be addressed
In this context, the aim of the thesis is to design and develop innovative audio coding methods based on deep learning, for mono and stereo signals.

In particular, the thesis will aim to address the following challenges:

• Obtain an audio representation by (artificial) neural networks that is capable of covering both mono and stereo
• Reduce the complexity of representation models in neural audio coding
• Obtain and interpretable latent space (giving a frequency-wise or content-wise separation)

Recent approaches such as transformers or diffusion models will be studied, and new neural network architectures will be tested and explored.

Indicative list of references
1.     Minje Kim and Jan Skoglund, “Neural Speech and Audio Coding,” arXiv:2408.06954v1, 2024

2.     Thomas Muller, Stephane Ragot, Laetitia Gros, Pierrick Philippe, Pascal Scalart, Speech quality evaluation of neural audio codecs, Interspeech, 2024

3.     N. Zeghidour et al., “SoundStream: An End-to-End Neural Audio Codec,” IEEE/ACM Trans. TASLP, 2021, arXiv:2107.03312

4.     R. Kumar et al., “High-Fidelity Audio Compression with Improved RVQGAN,” in Advances in Neural Information Processing Systems, vol. 36, 2023.

5.     J.D Parker et al., Scaling Transformers for Low-Bitrate High-Quality Speech Coding, arXiv:2411.19842, Nov. 2024

6.     Yaoxun Xu, et al., “ MuCodec: Ultra Low-Bitrate Music Codec,” arXiv:2409.13216, Sep. 2024