Search: onr:"swepub:oai:DiVA.org:ri-65998" >
Efficient Decoding ...
Efficient Decoding of Compositional Structure in Holistic Representations
-
- Kleyko, Denis (author)
- RISE,Datavetenskap,University of California, USA
-
- Bybee, Connor (author)
- University of California, USA
-
- Huang, P-C (author)
- University of California, USA
-
show more...
-
- Kymn, Christopher (author)
- University of California, USA
-
- Olshausen, Bruno (author)
- University of California, USA
-
- Frady, E Paxon (author)
- Intel Labs, USA
-
- Sommer, Friedrich (author)
- University of California, USA; Intel Labs, USA
-
show less...
-
(creator_code:org_t)
- MIT Press Journals, 2023
- 2023
- English.
-
In: Neural Computation. - : MIT Press Journals. - 0899-7667 .- 1530-888X. ; 35:7, s. 1159-1186
- Related links:
-
https://doi.org/10.1...
-
show more...
-
https://urn.kb.se/re...
-
https://doi.org/10.1...
-
show less...
Abstract
Subject headings
Close
- We investigate the task of retrieving information from compositional distributed representations formed by hyperdimensional computing/vector symbolic architectures and present novel techniques that achieve new information rate bounds. First, we provide an overview of the decoding techniques that can be used to approach the retrieval task. The techniques are categorized into four groups. We then evaluate the considered techniques in several settings that involve, for example, inclusion of external noise and storage elements with reduced precision. In particular, we find that the decoding techniques from the sparse coding and compressed sensing literature (rarely used for hyperdimensional computing/vector symbolic architectures) are also well suited for decoding information from the compositional distributed representations. Combining these decoding techniques with interference cancellation ideas from communications improves previously reported bounds (Hersche et al., 2021) of the information rate of the distributed representations from 1.20 to 1.40 bits per dimension for smaller codebooks and from 0.60 to 1.26 bits per dimension for larger codebooks.
Subject headings
- TEKNIK OCH TEKNOLOGIER -- Elektroteknik och elektronik -- Telekommunikation (hsv//swe)
- ENGINEERING AND TECHNOLOGY -- Electrical Engineering, Electronic Engineering, Information Engineering -- Telecommunications (hsv//eng)
Keyword
- Architecture
- Computer architecture
- Codebooks
- Compositional structure
- Decoding techniques
- Distributed representation
- External noise
- Four-group
- Information rates
- Novel techniques
- Reduced precision
- Storage elements
- article
- information retrieval
- noise
- Decoding
Publication and Content Type
- ref (subject category)
- art (subject category)
Find in a library
To the university's database