ECE Seminar Series – Mar 10 (Fri) @ 2:00pm: "How Designing an Application-specific Algorithm Led to a New Form of Reinforcement Learning," Urbashi Mitra, Prof., ECE, USC
Come at 1:30p for Cookies, Coffee and Conversation!
DISTINGUISHED LECTURE at the ECE SEMINAR SERIES
Abstract
Designing optimized policies in large scale wireless networks is challenging due to unknown or time-varying dynamics. While wireless communication networks can be well-modeled by Markov Decision Processes (MDPs), this approach induces a large state space which challenges policy optimization. Herein, we review strategies exploiting graph signal processing for network optimization including new representations for wireless network behavior. Our new representations effectively capture the influences of multiple hops in the network graph. We show that the novel representations allow for efficient graph reduction by projecting onto a lower dimensional subspace that accurately captures the behavior of the network while strongly reducing complexity for policy optimization. A novel on-line/off-line Q-learning methodology is proposed based on the new graph representation for wireless networks. The graph representations allow for the efficient creation of synthetic MDPs, or digital cousins, that accurately capture network behavior without the need for excessive trajectory sampling of the actual network. The approach enables the ensemble learning of multiple policies which can be efficiently fused. The proposed hybrid strategy offers significantly improved convergence rates and performance. Preliminary results on convergence and optimal weighting are provided. The method can be generalized for other directed graphs.
Bio
Urbashi Mitra received the B.S. and the M.S. degrees from the University of California at Berkeley and her Ph.D. from Princeton University. Dr. Mitra is currently the Gordon S. Marshall Professor in Engineering at the University of Southern California with appointments in Electrical & Computer Engineering and Computer Science. She was the inaugural Editor-in-Chief for the IEEE Transactions on Molecular, Biological and Multi-scale Communications. She has been a member of the IEEE Information Theory Society's Board of Governors (2002-2007, 2012-2017), the IEEE Signal Processing Society’s Technical Committee on Signal Processing for Communications and Networks (2012-2016), the IEEE Signal Processing Society’s Awards Board (2017-2018), and the Chair/Vice-Chair of the IEEE Communication Theory Technical Committee (2017-2020). Dr. Mitra is a Fellow of the IEEE. She is the recipient of: the 2021 USC Viterbi School of Engineering Senior Research Award, the 2017 IEEE Women in Communications Engineering Technical Achievement Award, a 2015 UK Royal Academy of Engineering Distinguished Visiting Professorship, a 2015 US Fulbright Scholar Award, a 2015-2016 UK Leverhulme Trust Visiting Professorship, IEEE Communications Society Distinguished Lecturer, 2012 Globecom Signal Processing for Communications Symposium Best Paper Award, 2012 US National Academy of Engineering Lillian Gilbreth Lectureship, the 2009 DCOSS Applications & Systems Best Paper Award, 2001 Okawa Foundation Award, 2000 Ohio State University’s College of Engineering Lumley Award for Research, and a 1996 National Science Foundation CAREER Award. Her research interests are in wireless communications, structured statistical methods, communication and sensor networks, biological communication systems, detection and estimation and the interface of communication, sensing and control.
Hosted by: ECE Seminar Series
Submitted by: Haewon Jeong <haewon@ucsb.edu>
