Cross-encoded meta embedding towards transfer learning

• In this paper we generate word meta-embeddings from already existing embeddings using cross-encoding. Previous approaches can only work with words that exist in each source embedding, while the architecture presented here drops this requirement. We demonstrate the method using two pre-trained embeddings, namely GloVE and FastText. Furthermore, we propose additional improvements to the training process of the metaembedding. Results on six standard tests for word similarity show that the meta-embedding trained outperforms the original embeddings. Moreover, this performance can be further increased with the proposed improvements, resulting in a competitive performance with those reported earlier.

Ämnesord

NATURVETENSKAP  -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Computer Sciences (hsv//eng)

TEKNIK OCH TEKNOLOGIER  -- Elektroteknik och elektronik -- Annan elektroteknik och elektronik (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Electrical Engineering, Electronic Engineering, Information Engineering -- Other Electrical Engineering, Electronic Engineering, Information Engineering (hsv//eng)

TEKNIK OCH TEKNOLOGIER  -- Maskinteknik -- Teknisk mekanik (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Mechanical Engineering -- Applied Mechanics (hsv//eng)

Nyckelord

Embeddings

Intelligent computing

Learning systems

Transfer learning

Competitive performance

Standard tests

Training process

Word similarity

Neural networks

Maskininlärning

Elektroniksystem

Experimentell mekanik

Publikations- och innehållstyp

ref (ämneskategori)

kon (ämneskategori)