| 1. |
- Berendsen, Richard, et al.
(författare)
-
First Report on Alternative Evaluation Methodology PROMISE Deliverable 4.1
- 2011. - 16
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- The first report on alternative evaluation methodology summarizes work done within the PROMISE environment and especially within Work package 4 - Evaluation Metrics and Methodologies. The report outlines efforts to develop and support alternative, automated evaluation methodologies, with a special focus on generating ground truth from existing data sources like Log files or annotations. Events like LogCLEF 2011, PatOlympics 2011 or the CHiC2011 workshop are presented and reviewed on their impact on the three main uses case domains.
|
|
| 2. |
- Ferro, Nicola, et al.
(författare)
-
PROMISE Retreat Report Prospects and Opportunities for Information Access Evaluation
- 2013
-
Ingår i: ACM SIGIR Forum. - : Association for Computing Machinery (ACM). - 0163-5840 .- 1558-0229. ; 46:2, s. 60-84
-
Tidskriftsartikel (övrigt vetenskapligt/konstnärligt)abstract
- The PROMISE network of excellence organized a two-days brainstorming workshop on 30th and 31st May 2012 in Padua, Italy, to discuss and envisage future directions and perspectives for the evaluation of information access and retrieval systems in multiple languages and multiple media. This document reports on the outcomes of this event and provides details about the six envisaged research lines: search applications; contextual evaluation; challenges in test collection design and exploitation; component-based evaluation; ongoing evaluation; and signal-aware evaluation. The ultimate goal of the PROMISE retreat is to stimulate and involve the research community along these research lines and to provide funding agencies with effective and scientifically sound ideas for coordinating and supporting information access research.
|
|
| 3. |
- Karlgren, Jussi, et al.
(författare)
-
Initial specification of the evaluation tasks "Use cases to bridge validation and benchmarking" PROMISE Deliverable 2.1
- 2011. - 7
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- Evaluation of multimedia and multilingual information access systems needs to be performed from a usage oriented perspective. This document outlines use cases from the three use case domains of the PROMISE project and gives some initial pointers to how their respective characteristics can be extrapolated to determine and guide evaluation activities, both with respect to benchmarking and to validation of the usage hypotheses. The use cases will be developed further during the course of the evaluation activities and workshops projected to occur in coming CLEF conferences.
|
|
