Search: onr:"swepub:oai:DiVA.org:kth-292658" >
Decentralized Word2...
Decentralized Word2Vec Using Gossip Learning
-
- Alkathiri, Abdul Aziz (author)
- KTH,Skolan för elektroteknik och datavetenskap (EECS)
-
- Giaretta, Lodovico, 1995- (author)
- KTH,Programvaruteknik och datorsystem, SCS
-
- Girdzijauskas, Sarunas (author)
- KTH,Programvaruteknik och datorsystem, SCS
-
show more...
-
Sahlgren, Magnus (author)
-
show less...
-
(creator_code:org_t)
- 2021
- 2021
- English.
-
In: Proceedings of the 23<sup>rd</sup> Nordic Conference on Computational Linguistics (NoDaLiDa 2021).
- Related links:
-
https://nodalida2021...
-
show more...
-
https://kth.diva-por... (primary) (Raw object)
-
https://urn.kb.se/re...
-
show less...
Abstract
Subject headings
Close
- Advanced NLP models require huge amounts of data from various domains to produce high-quality representations. It is useful then for a few large public and private organizations to join their corpora during training. However, factors such as legislation and user emphasis on data privacy may prevent centralized orchestration and data sharing among these organizations. Therefore, for this specific scenario, we investigate how gossip learning, a massively-parallel, data-private, decentralized protocol, compares to a shared-dataset solution. We find that the application of Word2Vec in a gossip learning framework is viable. Without any tuning, the results are comparable to a traditional centralized setting, with a reduction in ground-truth similarity scores as low as 4.3%. Furthermore, the results are up to 54.8% better than independent local training.
Subject headings
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Sciences (hsv//eng)
Keyword
- Computer Science
- Datalogi
- Informations- och kommunikationsteknik
- Information and Communication Technology
Publication and Content Type
- ref (subject category)
- kon (subject category)
To the university's database