Abstract

We study the first-order stochastic methods that can be utilized to solve the optimization problems derived from parameter estimation in statistics. The stochastic algorithm has a low cost per iteration and is more suitable for a large-size dataset. The first-order method only involves gradients; therefore, its implementation is more straightforward than other methods. Within the first-order stochastic methods, the techniques of variance reduction and generalization to mirror descent are often used to enhance their performance. In the first part of this talk, we describe a discovery that by using a particular design of the first-order stochastic method, utilizing variance reduction and mirror descent, one can show that the outcome automatically has an implicit regularization property. That is, the result of the stochastic algorithm automatically is a minimum-norm solution, while no penalty has explicitly involved. Based on this property, one can build some high probability exact recovery property for the corresponding parameter estimate. In the second part, we show that the stochastic algorithm can be used to solve the optimal transport problem, which is rooted in the applications of the Wasserstein distance and has found many applications in machine learning and artificial intelligence. We show that our stochastic algorithm has the best convergence rate among all the known methods. This talk is based on joint work with Ms. Yiling Luo.

About the Speaker

Dr. Huo received the B.S. degree in mathematics from the University of Science and Technology, China, in 1993, and the M.S. degree in electrical engineering and the Ph.D. degree in statistics from Stanford University, Stanford, CA, in 1997 and 1999, respectively. Since August 1999, he has been an Assistant/Associate/Full Professor with the School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta. He represented China in the 30th International Mathematical Olympiad (IMO), which was held in Braunschweig, Germany, in 1989, and received a golden prize. From August 2013 to August 2015, he served the US National Science Foundation as a Program Director in the Division of Mathematical Sciences (DMS).

Dr. Huo has presented keynote talks in major conferences (including, The 2nd IEEE Global Conference on Signal and Information Processing, Atlanta, GA, and the IMA-HK-IAS Joint program on statistics and computational interfaces to big data, The Hong Kong University of Science and Technology, Hong Kong, etc.) and numerous invited colloquia and seminar presentations in the US, Asia, and Europe. He is the Specialty Chief Editor in Frontiers in Applied Mathematics and Statistics - Statistics, April 2021 – present.  

Huo is now the Executive Director of TRIAD (Transdisciplinary Research Institute for Advancing Data Science), http://triad.gatech.edu, an NSF funded research center located at Georgia Tech. Dr. Huo is an Associate Director in the program of the Master of Science in Analytics -- https://analytics.gatech.edu/ -- overseeing creating a new branch in the Shenzhen-China campus of Georgia Institute of Technology. Dr. Huo is the Associate Director for Research of Institute for Data Engineering and Science (https://research.gatech.edu/data).