Challenges for IF performance evaluation in practice

• This study presents a literature analysis of 52 publications on the topic of information fusion performance evaluation. The aim is to identify whether information fusion researchers are aware of the problematic nature of performance evaluation in practice, which is substantially different from performance evaluation in test environments. As expected, only 6% of the publications do actually report on studies conducted in practice, and therefore more such studies are needed. However, 58% of the publications do critically reflect on the problematic nature of performance evaluation in practice. As ground truth is lacking in practice, there is a need for (a set of) performance measures which are both comprehensive and adaptable to domain or situation context and changing circumstances over time. More studies need to address the overall question how multiple performance measures can function in concert and how exactly can be judged whether adaptation has been an improvement or not.

Subject headings

NATURVETENSKAP  -- Data- och informationsvetenskap (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences (hsv//eng)

Keyword

Performance evaluation

literature survey

Teknik

Technology

Publication and Content Type

ref (subject category)

kon (subject category)