EE

PhD Thesis Defence of Aravind Illa
2021-09-29 16:30 - 17:30 (Asia/Kolkata)
Thesis Title: Acoustic-Articulatory Mapping: Analysis and Improvements with Neural Network Learning Paradigms
Abstract: Human speech is one of many acoustic signals we perceive, which carries linguistic and paralinguistic (e.g: speaker identity, emotional state) information. Speech acoustics are produced as a result of different temporally overlapping gestures of speech articulators (such as lips, tongue tip, tongue body, tongue dorsum, velum, and larynx) each of which regulates constriction in different parts of the vocal tract. Estimating speech acoustic representations from articulatory movements is known as articulatory-to-acoustic forward (AAF) mapping i.e., articulatory speech synthesis. While estimating articulatory movements back from the speech acoustics is known as acoustic-to-articulatory inverse (AAI) mapping. These acoustic-articulatory mapping functions are known to be complex and nonlinear.
Complexity of this mapping depends on a number of factors. These include the kind of representations used in the acoustic and articulatory spaces. Typically these representations capture both linguistic and paralinguistic aspects in speech. How each of these aspects contributes to the complexity of the mapping is unknown. These representations and, in turn, the acoustic-articulatory mapping are affected by the speaking rate as well. The nature and quality of the mapping varies across speakers. Thus, complexity of mapping also depends on the amount of the data from a speaker as well as number of speakers used in learning the mapping function. Further, how the language variations impact the mapping requires detailed investigation. This thesis analyzes few of such factors in detail and develops neural network based models to learn mapping functions robust to many of these factors.
Electromagnetic articulography (EMA) sensor data has been used directly in the past as articulatory representations (ARs) for learning the acoustic-articulatory mapping function. In this thesis, we address the problem of optimal EMA sensor placement such that the air-tissue boundaries as seen in the mid-sagittal plane of the real-time magnetic resonance imaging (rtMRI) is reconstructed with minimum error. Following optimal sensor placement work, acoustic-articulatory data was collected using EMA from 41 subjects with speech stimuli in English and Indian native languages (Hindi, Kannada, Tamil and Telugu) which resulted in a total of ~23 hours of data, used in this thesis. Representations from raw waveform are also learnt for AAI task using convolutional and bidirectional long short term memory neural networks (CNN-BLSTM), where the learned filters of CNN are found to be similar to those used for computing Mel-frequency cepstral coefficients (MFCCs), typically used for AAI task. In order to examine the extent to which a representation having only the linguistic information can recover ARs, we replace MFCC vectors with one-hot encoded vectors representing phonemes, which were further modified to remove the time duration of each phoneme and keep only phoneme sequence. Experiments with phoneme sequence using attention network achieve an AAI performance that is identical to that using phoneme with timing information, while there is a drop in performance compared to that using MFCC.
Experiments to examine variation in speaking rate reveal that, the errors in estimating the vertical motion of tongue articulators from acoustics with fast speaking rate, is significantly higher than those with slow speaking rate. In order to reduce the demand for data from a speaker, low resource AAI is proposed using a transfer learning approach. Further, we show that AAI can be modeled to learn acoustic-articulatory mappings of multiple speakers through a single AAI model rather than building separate speaker-specific models. This is achieved by conditioning an AAI model with speaker embeddings, which benefits AAI in seen and unseen speaker evaluations. Finally, we show the benefit of estimated ARs in voice conversion application. Experiments revealed that ARs estimated from speaker independent AAI preserves linguistic information and suppress speaker-dependent factors. These ARs (from unseen speaker and language) are used to drive target speaker specific AAF to synthesis speech, which preserves linguistic information and target speaker’s voice characteristics.

PhD Thesis Colloquium of Praveen Kumar Pokala @ 4pm
2021-08-06 21:30 - 22:30
Event : Thesis Colloquium
Title : Robust Nonconvex Penalties for Solving Sparse Linear Inverse Problems and Applications to Computational Imaging
Speaker : Praveen Kumar Pokala
Degree Registered : PhD
Advisor : Prof. Chandra Sekhar Seelamantula
Date : 06/08/2021
Venue : Online
Abstract : Sparse linear inverse problems require the solution to the l-0-regularized least-squares cost, which is not computationally tractable. Approximate and computationally tractable solutions are obtained by employing convex/nonconvex relaxations of the l-0-pseudonorm. One such approximation is obtained by considering the l-1-norm, which is a convex relaxation of the l-0-pseudonorm. However, l-1 regularization is known to result in biased estimates due to over-relaxation of the l-0-pseudonorm but it comes with the advantage of convexity of the regularized least-squares cost. Several nonconvex approximations of the l-0 pseudonorm have been proposed to overcome the bias introduced by the l-1-norm and to ensure better sparsity. However, certain aspects of nonconvex sparse regularization have not been explored. Some of these are as follows:
Nonconvex sparse priors have been explored in the synthesis-sparse framework, but not in the analysis-sparse framework due to the unavailability of proximal operators in closed-form in the analysis setting. Existing nonconvex approaches attach the same regularization weights across all the components of a sparse vector and treat them as fixed hyperparameters. Considering different weights for the entries and adapting them iteratively is likely to result in a superior performance.
Prior learning networks based on deep-unfolded architectures for solving nonconvex penalties have not been explored. This thesis addresses the above aspects in three parts and considers applications to various computational imaging problems.
Part-1: Nonconvex Analysis-sparse Recovery
In this part, we solve the analysis-sparse recovery problem based on three regularization approaches:
Convexity-preserving nonconvex regularization: We propose the analysis variants of the generalized Moreau envelope and generalized minimax concave penalty (GMCP) over a complex domain. Since the cost is a real-valued function defined over a complex domain, it is nonholomorphic, i.e., it does not satisfy Cauchy-Riemann (CR) conditions. To circumvent this problem, we rely upon on Wirtinger calculus to derive the proximal operator for the analysis l-1 prior and develop an efficient optimization strategy employing projected proximal algorithms. The projection transform maps the analysis-sparse recovery problem into an equivalent constrained synthesis-sparse formulation.
Nonconvex sparse regularization: We consider the problem of nonconvex analysis sparse recovery in which the signal is assumed to be sparse in a redundant analysis operator. Standard nonconvex sparsity promoting priors do not have a proximal operator in closed-form under a redundant analysis operator and therefore, proximal approaches cannot be applied directly. This led us to develop two alternatives — Moreau envelope regularization and projected transformation.
Generalized weighted l-1 regularization: We develop a generalized weighted l-1 regularization strategy, which allows for efficient weight-update strategies for iteratively reweighted l-1-minimization under tight frames. Further, we impose sufficient conditions on the weight function that leads to a reweighting strategy, which follows the interpretation originally given by Candès et al., but is more efficient than theirs. Since the objective function is nonholomorphic, we resort to Wirtinger calculus for deriving the update equations. We develop an algorithm called generalized iteratively reweighted soft-thresholding algorithm (GIRSTA) and its fast variant, namely, generalized fast iteratively reweighted soft-thresholding algorithm (GFIRSTA). We provide convergence guarantees for GIRSTA and empirical convergence results for GFIRSTA.
We demonstrate the efficacy of the proposed regularization strategies in comparison with the benchmark techniques considering compressive-sensing magnetic resonance image (CS-MRI) reconstruction under a redundant analysis operator, more specifically, shift-invariant discrete wavelet transform (SIDWT).
Part-2: Weighted Minimax Concave p-pseudonorm Minimization
In this part, we develop techniques for accurate low-rank plus sparse matrix decomposition (LSD) and low-rank matrix recovery. We proposed weighted minimax-concave penalty (WMCP) as the nonconvex regularizer and show that it admits a certain equivalent representation that is more amenable to weight adaptation. Similarly, an equivalent representation to the weighted matrix gamma norm (WMGN) enables weight adaptation for the low-rank part. The optimization algorithms are based on the alternating direction method of multipliers. The optimization frameworks relying on the two penalties, WMCP and WMGN, coupled with a novel iterative weight-update strategy, result in accurate low-rank plus sparse matrix decomposition and low-rank matrix recovery techniques. Further, we derive an algorithm, namely, iteratively reweighted MGN (iReMaGaN) algorithm, which has a superior low-rank matrix recovery performance. The proposed algorithms are shown to satisfy descent properties and convergence guarantees. On the applications front, we consider the problems of foreground-background separation and image denoising. Simulations and validations on standard datasets show that the proposed techniques outperform the benchmark techniques. Next, we extended the idea to obtain a generalized l-p-penalty, namely, minimax concave p-pseudonorm (MCpN) based on a novel p-Huber function as the sparsity promoting function, and its weighted counterpart, weighted MCpN (WMCpN) as a regularizer for solving the sparse linear inverse problem. WMCpN is a generalization of which several penalties, namely, l-1-norm, minimax concave penalty (MCP), l-p penalty, weighted l-1-norm, and weighted l-p penalty become special cases. However, MCpN and WMCpN regularizers do not have closed-form proximal operators, which makes the optimization problem challenging. To overcome this hurdle, we develop an equivalent representation that is more amenable to optimization and allows for an analytical weight-update strategy. MCpN is a special case of WMCpN where all the weights are fixed and equal. The optimization algorithms are based on the alternating direction method of multipliers. Considering the application of interferometric phase estimation, we demonstrate that MCpN and WMCpN result in accurate interferometric phase estimation. Simulations and experimental validations on standard datasets show that the proposed techniques outperform the benchmark techniques.
Part-3: Nonconvex Sparse Regularization and Deep-Unfolding
In the final part, we transition from fixed analytical priors to data-driven priors. To begin with, we develop a deep-unfolded architecture, namely, FirmNet, for sparse recovery. FirmNet has two parameters — one that controls the noise variance, and the other that allows for explicit sparsity control. We show that FirmNet is better than Learned-ISTA (LISTA) by at least three-fold in terms of the probability of error in support (PES), and about 2 to 4 dB higher reconstruction SNR. Further, we solve the problem of reflectivity inversion, which deals with estimating the subsurface structure from seismic data through FirmNet. As an application, we consider the problem of seismic reflectivity inversion. We demonstrate the efficacy of FirmNet over the benchmark techniques for the reflectivity inversion problem by testing on synthetic 1-D seismic traces and 2-D wedge models. We also report validations on simulated 2-D Marmousi2 model and real data from the Penobscot 3D survey off the coast of Nova Scotia, Canada. Next, we propose convolutional FirmNet (ConFirmNet), which is an extension of the FirmNet approach to solve the problem of convolutional sparse coding. As an application, we build a ConFirmNet based sparse autoencoder (ConFirmNet-SAE) and demonstrate suitability for image denoising and inpainting. Further, we also show that training ConFirmNet-SAE with the Huber loss imparts robustness to outliers. ConFirmNet-SAE also proves to be robust to mismatch between training and test noise conditions than convolutional learned iterative soft-thresholding algorithm (CLISTA). Finally, we propose a sparse recovery formulation that employs a nonuniform, nonconvex synthesis sparse model comprising a combination of convex and nonconvex regularizers, which results in accurate approximations of the l-0 pseudo-norm. The resulting iterative optimization employs proximal averaging. When unfolded, the iterations give rise to a nonuniform sparse proximal average network (NuSPAN) that can be optimized in a data-driven fashion. We demonstrate the efficacy of NuSPAN also for solving the problem of seismic reflectivity inversion.
Speaker Biodata : Praveen Kumar Pokala received his B.Tech. degree in Electronics and Telecommunication Engineering from Jawaharlal Nehru Technological University, Hyderabad, India, in 2006 and M. Tech degree in Signal Processing from Indian Institute of Technology (IIT), Guwahati, India, in 2009. Subsequently, he worked as an Assistant Professor in LPU university, Jalandhar, India and GITAM university, Hyderabad, India. He is currently pursuing Ph.D. in the Department of Electrical Engineering, Indian Institute of Science, Bangalore. His current research interests are machine learning, deep learning, and nonconvex optimization algorithms, with applications to inverse problems in computational imaging.

Events earlier to Aug 2021
2021-08-01 05:30 - 2021-08-02 05:30
Click to see the Content
