• Speaker: Daniel Spielman, Yale
• Title: Balancing covariates in randomized experiments using the Gram–Schmidt walk
• Date: 05/21/2020
• Time: 2:30 PM - 3:30 PM
• Place: 

(Part of the One World MINDS series: https://sites.google.com/view/minds-seminar/home) In randomized experiments, such as a medical trials, we randomly assign the treatment, such as a drug or a placebo, that each experimental subject receives. Randomization can help us accurately estimate the difference in treatment effects with high probability. We also know that we want the two groups to be similar: ideally the two groups would be similar in every statistic we can measure beforehand. Recent advances in algorithmic discrepancy theory allow us to divide subjects into groups with similar statistics. By exploiting the recent Gram-Schmidt Walk algorithm of Bansal, Dadush, Garg, and Lovett, we can obtain random assignments of low discrepancy. These allow us to obtain more accurate estimates of treatment effects when the information we measure about the subjects is predictive, while also bounding the worst-case behavior when it is not. We will explain the experimental design problem we address, the Gram-Schmidt walk algorithm, and the major ideas behind our analyses. This is joint work with Chris Harshaw, Fredrik Sävje, and Peng Zhang. Paper: https://arxiv.org/abs/1911.03071 Code: https://github.com/crharshaw/GSWDesign.jl

• Speaker: Ben Adcock, Simon Fraser University
• Title: The troublesome kernel: instabilities in deep learning for inverse problems; zoom link @ https://sites.google.com/view/minds-seminar/home
• Date: 06/04/2020
• Time: 2:30 PM - 3:30 PM
• Place: 

(Part of One World MINDS seminar: https://sites.google.com/view/minds-seminar/home) \[\] Due to their stunning success in traditional machine learning applications such as classification, techniques based on deep learning have recently begun to be actively investigated for problems in computational science and engineering. One of the key areas at the forefront of this trend is inverse problems, and specifically, inverse problems in imaging. The last few years have witnessed the emergence of many neural network-based algorithms for important imaging modalities such as MRI and X-ray CT. These claim to achieve competitive, and sometimes even superior, performance to current state-of-the-art techniques. However, there is a problem. Techniques based on deep learning are typically unstable. For example, small perturbations in the data can lead to a myriad of artefacts in the recovered images. Such artifacts can be hard to dismiss as obviously unphysical, meaning that this phenomenon has potentially serious consequences for the safe deployment of deep learning in practice. In this talk, I will first showcase the instability phenomenon empirically in a range of examples. I will then focus on its mathematical underpinnings, the consequences of these insights when it comes to potential remedies, and the future possibilities for computing genuinely stable neural networks for inverse problems in imaging. This is joint work with Vegard Antun, Nina M. Gottschling, Anders C. Hansen, Clarice Poon, and Francesco Renna Papers: https://www.pnas.org/content/early/2020/05/08/1907377117 https://arxiv.org/abs/2001.01258

• Speaker: Gitta Kutyniok, TU Berlin
• Title: Understanding Deep Neural Networks: From Generalization to Interpretability; zoom link @ https://sites.google.com/view/minds-seminar/home
• Date: 06/18/2020
• Time: 2:30 PM - 3:30 PM
• Place: 

(Part of One World MINDS seminar: https://sites.google.com/view/minds-seminar/home) \[ \] Deep neural networks have recently seen an impressive comeback with applications both in the public sector and the sciences. However, despite their outstanding success, a comprehensive theoretical foundation of deep neural networks is still missing. For deriving a theoretical understanding of deep neural networks, one main goal is to analyze their generalization ability, i.e. their performance on unseen data sets. In case of graph convolutional neural networks, which are today heavily used, for instance, for recommender systems, already the generalization capability to signals on graphs unseen in the training set, typically coined transferability, was not rigorously analyzed. In this talk, we will prove that spectral graph convolutional neural networks are indeed transferable, thereby also debunking a common misconception about this type of graph convolutional neural networks. If such theoretical approaches fail or if one is just given a trained neural network without knowledge of how it was trained, interpretability approaches become necessary. Those aim to "break open the black box" in the sense of identifying those features from the input, which are most relevant for the observed output. Aiming to derive a theoretically founded approach to this problem, we introduced a novel approach based on rate-distortion theory coined Rate-Distortion Explanation (RDE), which not only provides state-of-the-art explanations, but in addition allows first theoretical insights into the complexity of such problems. In this talk we will discuss this approach and show that it also gives a precise mathematical meaning to the previously vague term of relevant parts of the input.

• Speaker: Stephane Mallat, Ecole Normal Superieure
• Title: Beyond Sparsity: Non-Linear Harmonic Analysis with Phase for Deep Networks; zoom link @ https://sites.google.com/view/minds-seminar/home
• Date: 07/02/2020
• Time: 2:30 PM - 3:30 PM
• Place: 

(Part of One World MINDS seminar: https://sites.google.com/view/minds-seminar/home) \[ \] Understanding the properties of deep neural networks is not just about applying standard harmonic analysis tools with a bit of optimization. It is shaking our understanding of non-linear harmonic analysis and opening new horizons. By considering complex image generation and classification problems of different complexities, I will show that sparsity is not always the answer and phase plays an important role to capture important structures including symmetries within multiscale representations. This talk will raise more questions than answers.

• Speaker: Caroline Uhler, MIT & ETH Zurich
• Title: Multi-Domain Data Integration: From Observations to Mechanistic Insights; zoom link @ https://sites.google.com/view/minds-seminar/home
• Date: 07/16/2020
• Time: 2:30 PM - 3:30 PM
• Place: 

(Part of One World MINDS seminar: https://sites.google.com/view/minds-seminar/home) \[ \] Massive data collection holds the promise of a better understanding of complex phenomena and ultimately, of better decisions. An exciting opportunity in this regard stems from the growing availability of perturbation / intervention data (manufacturing, advertisement, education, genomics, etc.). In order to obtain mechanistic insights from such data, a major challenge is the integration of different data modalities (video, audio, interventional, observational, etc.). Using genomics and in particular the problem of identifying drugs for the repurposing against COVID-19 as an example, I will first discuss our recent work on coupling autoencoders in the latent space to integrate and translate between data of very different modalities such as sequencing and imaging. I will then present a framework for integrating observational and interventional data for causal structure discovery and characterize the causal relationships that are identifiable from such data. We end by a theoretical analysis of autoencoders linking overparameterization to memorization. In particular, I will characterize the implicit bias of overparameterized autoencoders and show that such networks trained using standard optimization methods implement associative memory. Collectively, our results have major implications for planning and learning from interventions in various application domains.

• Speaker: Mary Wooters, Stanford University
• Title: Sparse Recovery for Orthogonal Polynomial Transforms; zoom link @ https://sites.google.com/view/minds-seminar/home
• Date: 07/30/2020
• Time: 2:30 PM - 3:30 PM
• Place: 

(Part of One World MINDS seminar: https://sites.google.com/view/minds-seminar/home) \[ \] Consider the following sparse recovery problem. We have query access to a vector x in R^N such that \hat{x} = Fx is k-sparse (or nearly k-sparse) for some orthogonal transform F. The goal is to output an approximation to \hat{x} in sublinear time. This problem has been well-studied in the special case that F is the Discrete Fourier Transform (DFT), and a long line of work has resulted in sparse Fast Fourier Transforms that run in time O(k polylog(N)). However, for transforms F other than the DFT (or closely related transforms like the Discrete Cosine Transform), the question is much less settled. In this talk I'll describe sublinear-time algorithms---running in time poly(k log N)---for solving the sparse recovery problem for orthogonal transforms F that arise from orthogonal polynomials. More precisely, our algorithm works for any F that is an orthogonal polynomial transform derived from Jacobi polynomials. The Jacobi polynomials are a large class of classical orthogonal polynomials (and include Chebyshev and Legendre polynomials as special cases), and show up extensively in applications like numerical analysis and signal processing. One caveat of our work is that we require an assumption on the sparsity structure of the sparse vector, although we note that vectors with random support have this property with high probability. Our approach is to give a very general reduction from the k-sparse sparse recovery problem to the 1-sparse sparse recovery problem, which works for any "flat" orthogonal polynomial transform. This is joint work with Anna Gilbert, Albert Gu, Chris Re, and Atri Rudra.