Online Learning for Orchestrating Deep Learning Inference at Edge

Name: Sina Shahhosseini

Chair: Nikil Dutt

Date: February 22, 2023

Time: 10:30 AM

Location: 2011 DBH

Committee: Amir Rahmani, Fadi Kurdahi

Title:  Online Learning for Orchestrating Deep Learning Inference at Edge

Abstract: 

Deep-learning-based intelligent services have become prevalent in cyber-physical applications including smart cities and health-care. Resource-constrained end-devices must be carefully managed in order to meet the latency and energy requirements of computationally-intensive deep learning services. Collaborative end-edge-cloud computing for deep learning provides a range of performance and efficiency that can address application requirements through computation offloading. The decision to offload computation is a communication-computation co-optimization problem that varies with both system parameters (e.g., network condition) and workload characteristics. On the other hand, deep learning model optimization provides another source of tradeoff between latency and model accuracy. An end-to-end decision-making solution that considers such computation-communication problem is required to synergistically find the optimal offloading policy and model for deep learning services. To this end, we propose a reinforcement-learning-based computation offloading solution that learns optimal offloading policy considering deep learning model selection techniques to minimize response time while providing sufficient accuracy. In addition, we present a Hybrid Learning approach that reduces the number of interactions with the system environment by combining model-based and model-free reinforcement learning. We demonstrate the efficacy of our strategies through experimental comparison with state-of-the-art RL-based inference orchestration.

Short Bio:

Sina Shahhosseini received his B.S. and M.S degrees in Electrical Engineering from the University of Tehran and Sharif University of Technology, in 2013 and 2015, respectively. Since Fall 2016, he has been working towards his Ph.D. degree in Computer Engineering at University of California, Irvine, with a focus on the application of reinforcement learning for task orchestration in computer systems. He interned with Cruise in the summer of 2022 as a Machine Learning Engineer Intern, where he worked on optimizing Machine Learning frameworks for autonomous vehicles.