Sökning: onr:"swepub:oai:DiVA.org:ri-72876" >
ELOQUENT CLEF Share...
ELOQUENT CLEF Shared Tasks for Evaluation of Generative Language Model Quality
-
- Karlgren, Jussi (författare)
- Silo AI, Finland
-
- Dürlich, Luise (författare)
- RISE,Datavetenskap
-
- Gogoulou, Evangelia (författare)
- RISE,Datavetenskap
-
visa fler...
-
- Guillou, Liane (författare)
- RISE,Datavetenskap
-
- Nivre, Joakim, 1962- (författare)
- RISE,Datavetenskap
-
- Sahlgren, Magnus (författare)
- Silo AI, Finland; AI Sweden, Sweden
-
- Talman, Aarne (författare)
- Silo AI, Finland
-
visa färre...
-
(creator_code:org_t)
- Springer Science and Business Media Deutschland GmbH, 2024
- 2024
- Engelska.
-
Ingår i: Lecture Notes in Computer Science. - : Springer Science and Business Media Deutschland GmbH. - 0302-9743 .- 1611-3349. ; 14612 LNCS, s. 459-465
- Relaterad länk:
-
https://urn.kb.se/re...
-
visa fler...
-
https://doi.org/10.1...
-
visa färre...
Abstract
Ämnesord
Stäng
- ELOQUENT is a set of shared tasks for evaluating the quality and usefulness of generative language models. ELOQUENT aims to bring together some high-level quality criteria, grounded in experiences from deploying models in real-life tasks, and to formulate tests for those criteria, preferably implemented to require minimal human assessment effort and in a multilingual setting. The selected tasks for this first year of ELOQUENT are (1) probing a language model for topical competence; (2) assessing the ability of models to generate and detect hallucinations; (3) assessing the robustness of a model output given variation in the input prompts; and (4) establishing the possibility to distinguish human-generated text from machine-generated text.
Ämnesord
- NATURVETENSKAP -- Data- och informationsvetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences (hsv//eng)
Nyckelord
- Benchmarking; CLEF; Generative language model; Human assessment; Language model; LLM; Modeling quality; Multilinguality; Quality benchmark; Quality criteria; Shared task; Computational linguistics
Publikations- och innehållstyp
- ref (ämneskategori)
- art (ämneskategori)
Hitta via bibliotek
Till lärosätets databas