• Speaker: Raymond Chan, City University of Hong Kong
• Title: TBA; zoom link @ https://sites.google.com/view/minds-seminar/home
• Date: 01/14/2021
• Time: 3:30 PM - 4:30 PM
• Place:  (Virtual Meeting Link)
• Contact: Olga Turanova (turanova@msu.edu)

TBA; (Note the unusual time: 4:30pm Shanghai, 10:30am Paris.)

• Speaker: Zuowei Shen, National University of Singapore
• Title: Deep Approximation via Deep Learning
• Date: 01/28/2021
• Time: 3:30 AM - 4:30 AM
• Place: Online (virtual meeting) (Virtual Meeting Link)
• Contact: Olga Turanova (turanova@msu.edu)

The primary task of many applications is approximating/estimating a function through samples drawn from a probability distribution on the input space. The deep approximation is to approximate a function by compositions of many layers of simple functions, that can be viewed as a series of nested feature extractors. The key idea of deep learning network is to convert layers of compositions to layers of tuneable parameters that can be adjusted through a learning process, so that it achieves a good approximation with respect to the input data. In this talk, we shall discuss mathematical theory behind this new approach and approximation rate of deep network; how this new approach differs from the classic approximation theory, and how this new theory can be used to understand and design deep learning network.

• Speaker: Massimo Fornasier , Technische Universität München
• Title: Consensus-based Optimization on the Sphere
• Date: 02/11/2021
• Time: 2:30 PM - 3:30 PM
• Place: Online (virtual meeting) (Virtual Meeting Link)
• Contact: Olga Turanova (turanova@msu.edu)

I present new stochastic multi-particle models for global optimization of nonconvex functions on the sphere. These models belong to the class of Consensus-Based Optimization methods. In fact, particles move over the manifold driven by a drift towards an instantaneous consensus point, computed as a combination of the particle locations weighted by the cost function according to Laplace's principle. The consensus point represents an approximation to a global minimizer. The dynamics is further perturbed by a random vector field to favor exploration, whose variance is a function of the distance of the particles to the consensus point. In particular, as soon as the consensus is reached, then the stochastic component vanishes. In the first part of the talk, I present the well-posedness of the model on the sphere and we derive rigorously its mean-field approximation for large particle limit. In the second part I address the proof of convergence of numerical schemes to global minimizers provided conditions of well-preparation of the initial datum. The proof combines the mean-field limit with a novel asymptotic analysis, and classical convergence results of numerical methods for SDE. We present several numerical experiments, which show that the proposed algorithm scales well with the dimension and is extremely versatile. To quantify the performances of the new approach, we show that the algorithm is able to perform essentially as good as ad hoc state of the art methods in challenging problems in signal processing and machine learning, namely the phase retrieval problem and the robust subspace detection. Joint work with H. Huang, L. Pareschi, and P. Sünnen

• Speaker: Zaiwen Wen, Peking University, China
• Title: Stochastic Second-Order Methods For Deep Learning
• Date: 02/25/2021
• Time: 3:30 AM - 4:30 AM
• Place: Online (virtual meeting) (Virtual Meeting Link)
• Contact: Olga Turanova (turanova@msu.edu)

Stochastic methods are widely used in deep learning. In this talk, we first review the state-of-the-art methods such as KFAC. Then we present a structured stochastic quasi-Newton method and a sketchy empirical natural gradient method. Numerical results on deep convolution networks illustrate that our methods are quite competitive to SGD and KFAC.

• Speaker: Jianfeng Cai, Hong Kong University of Science and Technology
• Title: Landscape analysis of non-convex optimizations in phase retrieval
• Date: 03/11/2021
• Time: 3:30 AM - 4:30 AM
• Place: Online (virtual meeting) (Virtual Meeting Link)
• Contact: Olga Turanova (turanova@msu.edu)

Non-convex optimization is a ubiquitous tool in scientific and engineering research. For many important problems, simple non-convex optimization algorithms often provide good solutions efficiently and effectively, despite possible local minima. One way to explain the success of these algorithms is through the global landscape analysis. In this talk, we present some results along with this direction for phase retrieval. The main results are, for several of non-convex optimizations in phase retrieval, a local minimum is also global and all other critical points have a negative directional curvature. The results not only explain why simple non-convex algorithms usually find a global minimizer for phase retrieval, but also are useful for developing new efficient algorithms with a theoretical guarantee by applying algorithms that are guaranteed to find a local minimum.

• Speaker: Rachel Ward , University of Texas at Austin
• Title: Function Approximation via Sparse Random Features
• Date: 03/25/2021
• Time: 2:30 PM - 2:30 PM
• Place: Online (virtual meeting) (Virtual Meeting Link)
• Contact: Olga Turanova (turanova@msu.edu)

Random feature methods have been successful in various machine learning tasks, are easy to compute, and come with theoretical accuracy bounds. They serve as an alternative approach to standard neural networks since they can represent similar function spaces without a costly training phase. However, for accuracy, random feature methods require more measurements than trainable parameters, limiting their use for data-scarce applications or problems in scientific machine learning. This paper introduces the sparse random feature method that learns parsimonious random feature models utilizing techniques from compressive sensing. We provide uniform bounds on the approximation error for functions in a reproducing kernel Hilbert space depending on the number of samples and the distribution of features. The error bounds improve with additional structural conditions, such as coordinate sparsity, compact clusters of the spectrum, or rapid spectral decay. We show that the sparse random feature method outperforms shallow networks for well-structured functions and applications to scientific machine learning tasks.

• Speaker: Mahdi Soltanolkotabi, University of Southern California
• Title: Precise Tradeoffs for Adversarial Training
• Date: 04/08/2021
• Time: 2:30 PM - 3:30 PM
• Place: Online (virtual meeting) (Virtual Meeting Link)
• Contact: Olga Turanova (turanova@msu.edu)

Despite breakthrough performance, modern learning models are known to be highly vulnerable to small adversarial perturbations in their inputs. While a wide variety of recent adversarial training methods have been effective at improving robustness to perturbed inputs (robust accuracy), often this benefit is accompanied by a decrease in accuracy on benign inputs (standard accuracy), leading to a tradeoff between often competing objectives. Complicating matters further, recent empirical evidence suggests that a variety of other factors (size and quality of training data, model size, etc.) affect this tradeoff in somewhat surprising ways. In this talk we will provide a precise and comprehensive understanding of the role of adversarial training in the context of linear regression with Gaussian features and binary classification in a mixture model. We precisely characterize the standard/robust accuracy and the corresponding tradeoff achieved by a contemporary mini-max adversarial training approach in a high-dimensional regime where the number of data points and the parameters of the model grow in proportion to each other. Our theory for adversarial training algorithms also facilitates the rigorous study of how a variety of factors (size and quality of training data, model overparametrization etc.) affect the tradeoff between these two competing accuracies.

• Speaker: Shahar Mendelson, Australian National University
• Title: TBA
• Date: 04/22/2021
• Time: 4:30 AM - 5:30 AM
• Place: Online (virtual meeting) (Virtual Meeting Link)
• Contact: Olga Turanova (turanova@msu.edu)

No abstract available.