Georgia Institute of Technology College of Sciences

We are going talk about three topics. First of all, Principal Components Analysis (PCA) as a dimension reduction technique. We investigate how useful it is for real life problems. The problem is that, often times the spectrum of the covariance matrix is wrongly estimated due to the ratio between sample space dimension over feature space dimension not being large enough. We show how to reconstruct the spectrum of the ground truth covariance matrix, given the spectrum of the estimated covariance for multivariate normal vectors. We then present an algorithm for reconstruction the spectrum in the case of sparse matrices related to text classification. 

In the second part, we concentrate on schemes of PCA estimators. Consider the problem of finding the least eigenvalue and eigenvector of ground truth covariance matrix, a famous classical estimator are due to Krasulina. We state the convergence proof of Krasulina for the least eigenvalue and corresponding eigenvector, and then find their convergence rate.

In the last part, we consider the application problem, text classification, in the supervised view with traditional Naive-Bayes method. We find out an updated Naive-Bayes method with a new loss function, which loses the unbiased property of traditional Naive-Bayes method, but obtains a smaller variance of the estimator. 

Committee:  Heinrich Matzinger (Advisor); Karim Lounici (Advisor); Ionel Popescu (school of math); Federico Bonetto (school of math); Xiaoming Huo (school of ISYE);

In standard (mathematical) billiards a point particle moves uniformly in a billiard table with elastic reflections off the boundary. We show that in transition from mathematical billiards to physical billiards, where a finite size hard sphere moves in the same billiard table, virtually anything may happen. Namely a non-chaotic billiard may become chaotic and vice versa. Moreover, both these transitions may occur softly, i.e. for any (arbitrarily small) positive value of the radius of a physical particle, as well as by a ”hard” transition when radius of the physical particle must exceed some critical strictly positive value. Such transitions may change a phase portrait of a mathematical billiard locally as well as completely (globally). These results are somewhat unexpected because for all standard examples of billiards their dynamics remains absolutely the same after transition from a point particle to a finite size (”physical”) particle. Moreover we show that a character of dynamics may change several times when the size of the particle is increasing. This approach already demonstrated a sensational result that quantum system could be more chaotic than its classical counterpart.

In an optimal design problem, we are given a set of linear experiments v1,...,vn \in R^d and k >= d, and our goal is to select a set or a multiset S subseteq [n] of size k such that Phi((\sum_{i \in [n]} v_i v_i^T )^{-1}) is minimized. When Phi(M) = det(M)^{1/d}, the problem is known as the D-optimal design problem, and when Phi(M) = tr(M), it is known as the A-optimal design problem. One of the most common heuristics used in practice to solve these problems is the local search heuristic, also known as the Fedorov's exchange method. This is due to its simplicity and its empirical performance. However, despite its wide usage no theoretical bound has been proven for this algorithm. In this paper, we bridge this gap and prove approximation guarantees for the local search algorithms for D-optimal design and A-optimal design problems. We show that the local search algorithms are asymptotically optimal when $\frac{k}{d}$ is large. In addition to this, we also prove similar approximation guarantees for the greedy algorithms for D-optimal design and A-optimal design problems when k/d is large.

Computing the eigenvalues and eigenvectors of a large matrix is a basic task in high dimensional data analysis with many applications in computer science and statistics. In practice, however, data is often perturbed by noise. A natural question is the following: How much does a small perturbation to the matrix change the eigenvalues and eigenvectors? In this talk, I will consider the case where the perturbation is random. I will discuss perturbation results for the eigenvalues and eigenvectors as well as for the singular values and singular vectors.  This talk is based on joint work with Van Vu, Ke Wang, and Philip Matchett Wood.