Feb 24 (Thu) @ 10:00am: "Randomized Algorithms for Efficient Machine Learning Systems," Beidi Chen, Postdoctoral Fellow, Stanford

https://ucsb.zoom.us/j/84106492648

Abstract

Machine learning (ML) has demonstrated great promise in scientific discovery, healthcare, and education, especially with the rise of large neural networks. However, large models trained on complex and rapidly growing data consume enormous computational resources. In this talk, I will describe my work on exploiting model sparsity with randomized algorithms to accelerate large ML systems on current hardware.

I will start by describing SLIDE, an open-source system for efficient sparse neural network training on CPUs that has been deployed by major technology companies and academic labs. It blends Locality Sensitive Hashing with multi-core parallelism and workload optimization to drastically reduce computations. SLIDE trains industry-scale recommendation models on a 44 core CPU 3.5x faster than TensorFlow on V100 GPU.

Next, I will present Pixelated Butterfly, a simple yet efficient sparse training framework on GPUs. It uses a simple static block-sparse pattern based on butterfly and low-rank matrices, taking into account GPU block-oriented efficiency. Pixelated Butterfly trains up to 2.5x faster (wall-clock) than the dense Vision Transformer and GPT-2 counterparts with no drop in accuracy.

I will conclude by outlining future research directions for further accelerating ML pipelines and making ML more accessible to the general community, such as software-hardware co-design and sparse models for scientific computing and medical imaging.

Bio

Beidi Chen is a postdoctoral scholar in the CS department at Stanford University, working with Christopher Ré. Her research focuses on large-scale machine learning and deep learning. Specifically, she designs and optimizes randomized algorithms (algorithm-hardware co-design) to accelerate large machine learning systems for real-world problems. Prior to joining Stanford, she received her Ph.D. from the CS department at Rice University, advised by Anshumali Shrivastava. She received a BS in EECS from UC Berkeley in 2015. She has held internships in Microsoft Research, NVIDIA Research, and Amazon AI. Her work has won Best Paper awards at LISA and IISA. She was selected as a Rising Star in EECS by MIT and UIUC.

Hosted by: ECE Computer Engineering

Submitted by: Libby Straight <libby@ece.ucsb.edu>