Abstract

Conventional statistical theory operates under the regime with a large signal-to-noise ratio, where the quality of estimating an unknown parameter can be measured by various convergence metrics. A different, recent line of statistical theory operates in the so-called mean-field regime with a moderate signal-to-noise ratio. In this regime, consistent parameter estimation becomes infeasible, but the theory provides precise understanding of the behavior of statistical estimators, mostly in the idealized Gaussian data setting. Despite this, numerical evidence strongly suggests its validity for non-Gaussian scenarios, a phenomenon known as “universality”.
This talk will present several new theoretical tools to validate this universality phenomenon. First, we will introduce a universality framework for global statistics associated with regularized regression estimators based on Gaussian process theory, and demonstrate its utility for both the classical Lasso and the more recent interpo-lating estimators. Second, we will introduce an algorithmic approach that provides refined, entrywise universality guarantees. In particular, we will discuss an entrywise distribution theory for the so-called Approximate Message Passing algorithm, and illustrate its utility in several regularized regression problems.