Georgia Institute of Technology College of Sciences

Iterative algorithms are the workhorses of modern statistical signal processing and machine learning. While the choice of an algorithm and its hyperparameters determines both the speed and fidelity of the learning pipeline, it is common for this choice to be made heuristically, either by expensive trial-and-error or by comparing upper bounds on convergence rates of various candidate algorithms. Motivated by these issues, I will present a toolbox for deriving “state evolutions” for a wide variety of algorithms with random data. These are non-asymptotic, near-exact predictions of the statistical behavior of the algorithm, which apply even when the underlying optimization problem is nonconvex or the algorithm is randomly initialized. We will showcase these predictions on deterministic and stochastic variants of complex algorithms employed in some canonical statistical models.

Spatio-temporal modeling of real-world data presents significant challenges due to high-dimensionality, noisy measurements, and limited data. In this talk, we introduce two frameworks that jointly solve the problems of sparse identification of governing equations and latent space reconstruction: the Bayesian SINDy autoencoder and SINDy-SHRED. The Bayesian SINDy autoencoder leverages a spike-and-slab prior to enable robust discovery of governing equations and latent coordinate systems, providing uncertainty estimates in low-data, high-noise settings. In our experiments, we applied the Bayesian SINDy autoencoder to real video data, marking the first example of learning governing equations directly from such data. This framework successfully identified underlying physical laws, such as accurately estimating constants like gravity from pendulum videos, even in the presence of noise and limited samples.

In parallel, SINDy-SHRED integrates Gated Recurrent Units (GRUs) with a shallow decoder network to model temporal sequences and reconstruct full spatio-temporal fields using only a few sensors. Our proposed algorithm introduces a SINDy-based regularization. Beginning with an arbitrary latent state space, the dynamics of the latent space progressively converges to a SINDy-class functional. We conduct a systematic experimental study including synthetic PDE data, real-world sensor measurements for sea surface temperature, and direct video data. With no explicit encoder, SINDy-SHRED allows for efficient training with minimal hyperparameter tuning and laptop-level computing. SINDy-SHRED demonstrates robust generalization in a variety of applications with minimal to no hyperparameter adjustments. Additionally, the interpretable SINDy model of latent state dynamics enables accurate long-term video predictions, achieving state-of-the-art performance and outperforming all baseline methods considered, including Convolutional LSTM, PredRNN, ResNet, and SimVP.

In this talk, we present recent results regarding the existence of invariant probability measures for delay equations with (stochastic) negative feedback. No prior knowledge on invariant measures is assumed. Applications include Nicholson's blowflies equation and the Mackey-Glass equations. Just like the dynamics of prime numbers, these systems exhibit "randomness" combined with deep structure. We will prove this both analytically and numerically and focus mainly on intuition. In general, additive noise typically destroys all dynamical properties of the underlying dynamical system. Therefore, we are motivated to study a class of stochastic perturbations that preserve some of the dynamical properties of the negative feedback systems we consider.

Expander graphs are highly connected sparse graphs. They have wide theoretical and practical applications in Computer Science and Engineering. Ramanujan Graphs are optimal expanders and as the name suggests they are constructed number theoretically. We review their construction as well more recent constructions that use statistical physics. We highlight some recent applications in the reverse direction where combinatorial ideas are combined with arithmetical ones to establish expansion of graphs arising in diophantine analysis.