A Benchmark of Dynamical Variational Autoencoders applied to Speech Spectrogram Modeling

Xiaoyu Bie (1) , Laurent Girin (2) , Simon Leglaive (3, 4) , Thomas Hueber (2) , Xavier Alameda-Pineda (1)

1 ROBOTLEARN - Vers des robots à l’intelligence sociale au travers de l’apprentissage, de la perception et de la commande
2 GIPSA-CRISSP - GIPSA - Cognitive Robotics, Interactive Systems, & Speech Processing
3 IETR - Institut d'Électronique et des Technologies du numéRique
4 CentraleSupélec [campus de Rennes]

The Variational Autoencoder (VAE) is a powerful deep generative model that is now extensively used to represent high-dimensional complex data via a low-dimensional latent space learned in an unsupervised manner. In the original VAE model, input data vectors are processed independently. In recent years, a series of papers have presented different extensions of the VAE to process sequential data, that not only model the latent space, but also model the temporal dependencies within a sequence of data vectors and corresponding latent vectors, relying on recurrent neural networks. We recently performed a comprehensive review of those models and unified them into a general class called Dynamical Variational Autoencoders (DVAEs). In the present paper, we present the results of an experimental benchmark comparing six of those DVAE models on the speech analysis-resynthesis task, as an illustration of the high potential of DVAEs for speech modeling.

Keywords: Speech signals modeling; dynamical variational autoencoders; speech spectrograms; speech analysisresynthesis

Domains: Machine Learning [cs.LG]; Artificial Intelligence [cs.AI]; Sound [cs.SD]

Xiaoyu Bie, Laurent Girin, Simon Leglaive, Thomas Hueber, Xavier Alameda-Pineda. A Benchmark of Dynamical Variational Autoencoders applied to Speech Spectrogram Modeling. Interspeech 2021 - 22nd Annual Conference of the International Speech Communication Association, Aug 2021, Brno, Czech Republic. pp.46-50, ⟨10.21437/Interspeech.2021-256⟩. ⟨hal-03295657⟩