A Distributional Semantic Online Lexicon for Linguistic Explorations of Societies

• Linguistic Explorations of Societies (LES) is an interdisciplinary research project with scholars from the fields of political science, computer science, and computational linguistics. The overarching ambition of LES has been to contribute to the survey-based comparative scholarship by compiling and analyzing online text data within and between languages and countries. To this end, the project has developed an online semantic lexicon, which allows researchers to explore meanings and usages of words in online media across a substantial number of geo-coded languages. The lexicon covers data from approximately 140 language-country combinations and is, to our knowledge, the most extensive free research resource of its kind. Such a resource makes it possible to critically examine survey translations and identify discrepancies in order to modify and improve existing survey methodology, and its unique features further enable Internet researchers to study public debate online from a comparative perspective. In this article, we discuss the social scientific rationale for using online text data as a complement to survey data, and present the natural language processing-based methodology behind the lexicon including its underpinning theory and practical modeling. Finally, we engage in a critical reflection about the challenges of using online text data to gauge public opinion and political behavior across the world.

Subject headings

SAMHÄLLSVETENSKAP  -- Annan samhällsvetenskap (hsv//swe)

SOCIAL SCIENCES  -- Other Social Sciences (hsv//eng)

NATURVETENSKAP  -- Data- och informationsvetenskap -- Språkteknologi (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Language Technology (hsv//eng)

SAMHÄLLSVETENSKAP  -- Medie- och kommunikationsvetenskap -- Kommunikationsvetenskap (hsv//swe)

SOCIAL SCIENCES  -- Media and Communications -- Communication Studies (hsv//eng)

Keyword

distributional semantics

natural language processing

word2vec

comparative surveys

language use

semantic similarities

democracy

satisfaction

Computer Science

Information Science & Library Science

Social Sciences

- Other Topics

comparative surveys

Publication and Content Type

ref (subject category)

art (subject category)