Sökning: id:"swepub:oai:DiVA.org:kth-334523" >
The Case for Hierar...
The Case for Hierarchical Deep Learning Inference at the Network Edge
-
- Al-Atat, Ghina (författare)
- IMDEA Networks Institute, Madrid, Spain
-
- Fresa, Andrea (författare)
- IMDEA Networks Institute, Madrid, Spain
-
- Behera, Adarsh Prasad (författare)
- IMDEA Networks Institute, Madrid, Spain
-
visa fler...
-
- Moothedath, Vishnu Narayanan (författare)
- KTH,Teknisk informationsvetenskap
-
- Gross, James, Professor, 1975- (författare)
- KTH,Teknisk informationsvetenskap
-
- Champati, Jaya Prakash (författare)
- IMDEA Networks Institute, Madrid, Spain
-
visa färre...
-
(creator_code:org_t)
- Association for Computing Machinery (ACM), 2023
- 2023
- Engelska.
-
Ingår i: NetAISys 2023 - Proceedings of the 1st International Workshop on Networked AI Systems, Part of MobiSys 2023. - : Association for Computing Machinery (ACM). ; , s. 13-18
- Relaterad länk:
-
https://doi.org/10.1...
-
visa fler...
-
https://urn.kb.se/re...
-
https://doi.org/10.1...
-
visa färre...
Abstract
Ämnesord
Stäng
- Resource-constrained Edge Devices (EDs), e.g., IoT sensors and microcontroller units, are expected to make intelligent decisions using Deep Learning (DL) inference at the edge of the network. Toward this end, developing tinyML models is an area of active research - DL models with reduced computation and memory storage requirements - that can be embedded on these devices. However, tinyML models have lower inference accuracy. On a different front, DNN partitioning and inference offloading techniques were studied for distributed DL inference between EDs and Edge Servers (ESs). In this paper, we explore Hierarchical Inference (HI), a novel approach proposed in [19] for performing distributed DL inference at the edge. Under HI, for each data sample, an ED first uses a local algorithm (e.g., a tinyML model) for inference. Depending on the application, if the inference provided by the local algorithm is incorrect or further assistance is required from large DL models on edge or cloud, only then the ED offloads the data sample. At the outset, HI seems infeasible as the ED, in general, cannot know if the local inference is sufficient or not. Nevertheless, we present the feasibility of implementing HI for image classification applications. We demonstrate its benefits using quantitative analysis and show that HI provides a better trade-off between offloading cost, throughput, and inference accuracy compared to alternate approaches.
Ämnesord
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Sciences (hsv//eng)
- TEKNIK OCH TEKNOLOGIER -- Elektroteknik och elektronik -- Datorsystem (hsv//swe)
- ENGINEERING AND TECHNOLOGY -- Electrical Engineering, Electronic Engineering, Information Engineering -- Computer Systems (hsv//eng)
Nyckelord
- deep learning
- edge computing
- hierarchical inference
Publikations- och innehållstyp
- ref (ämneskategori)
- kon (ämneskategori)