A Proximal Algorithm for Estimating the Regularized Wavelet-based Density-Difference


Density-Difference (DD) estimation is an important unsupervised learning procedure that proceeds many regression methods. The present work details a novel method for estimating the Difference of Densities (DoD) between two distributions. This new method directly calculates the DD, in the form of a wavelet expansion, without the need for explicitly reconstructing individual distributions. Furthermore, the method applies a regularization technique that utilizes both l2 and l1 norm penalties to robustly estimate the coefficients of the wavelet expansion. Optimizing the regularized objective is accomplished via a Proximal Gradient Descent (PGD) approach. Thus, we term our method Regularized Wavelet-based Density-Difference (RWDD) with PGD. On extensive simulated datasets, from complex multimodal to skewed distributions, our method demonstrated superior performance in comparison to other contemporary techniques.

6th Annual Conference on Computational Science & Computational Intelligence
Georgios C. Anagnostopoulos
Georgios C. Anagnostopoulos
Associate Professor of Electrical & Computer Engineering

I lead the Machine Learning Research Group at FIT.