The Case for Hierarchical Deep Learning Inference at the Network Edge

Part of ISBN 9798400702129QC 20230823
Resource-constrained Edge Devices (EDs), e.g., IoT sensors and microcontroller units, are expected to make intelligent decisions using Deep Learning (DL) inference at the edge of the network. Toward this end, developing tinyML models is an area of active research - DL models with reduced computation and memory storage requirements - that can be embedded on these devices. However, tinyML models have lower inference accuracy. On a different front, DNN partitioning and inference offloading techniques were studied for distributed DL inference between EDs and Edge Servers (ESs). In this paper, we explore Hierarchical Inference (HI), a novel approach proposed in [19] for performing distributed DL inference at the edge. Under HI, for each data sample, an ED first uses a local algorithm (e.g., a tinyML model) for inference. Depending on the application, if the inference provided by the local algorithm is incorrect or further assistance is required from large DL models on edge or cloud, only then the ED offloads the data sample. At the outset, HI seems infeasible as the ED, in general, cannot know if the local inference is sufficient or not. Nevertheless, we present the feasibility of implementing HI for image classification applications. We demonstrate its benefits using quantitative analysis and show that HI provides a better trade-off between offloading cost, throughput, and inference accuracy compared to alternate approaches.

Fresa, AndreaIMDEA Networks Institute, Madrid, Spain (författare)
Behera, Adarsh PrasadIMDEA Networks Institute, Madrid, Spain (författare)
Moothedath, Vishnu NarayananKTH,Teknisk informationsvetenskap(Swepub:kth)u1m3dons (författare)
Gross, James,Professor,1975-KTH,Teknisk informationsvetenskap(Swepub:kth)u1zrx6q1 (författare)
Champati, Jaya PrakashIMDEA Networks Institute, Madrid, Spain (författare)
IMDEA Networks Institute, Madrid, SpainTeknisk informationsvetenskap (creator_code:org_t)

Ingår i:NetAISys 2023 - Proceedings of the 1st International Workshop on Networked AI Systems, Part of MobiSys 2023: Association for Computing Machinery (ACM), s. 13-18

Av författaren/redakt...: Al-Atat, Ghina; Fresa, Andrea; Behera, Adarsh P ...; Moothedath, Vish ...; Gross, James, Pr ...; Champati, Jaya P ...

Om ämnet

TEKNIK OCH TEKNOLOGIER: TEKNIK OCH TEKNO ...; och Elektroteknik oc ...; och Datorsystem

Kungliga biblioteket hanterar dina personuppgifter i enlighet med EU:s dataskyddsförordning (2018), GDPR. Läs mer om hur det funkar här.
Så här hanterar KB dina uppgifter vid användning av denna tjänst.