Ex-Post Evaluation of Data-Driven Decisions: Conceptualizing Design Objectives

• This paper addresses a need for developing ex-post evaluation for data-driven decisions resulting in collaboration between humans and machines. As a first step of a design science project, we propose four design objectives for an ex-post evaluation solution, from the perspectives of both theory (concepts from the literature) and practice (through a case of industrial production planning): (1) incorporate multi-faceted decision evaluation criteria across the levels of environment, organization, and decision itself and (2) acknowledge temporal requirements of the decision contexts at hand, (3) define applicable mode(s) of collaboration between humans and machines to pursue collaborative rationality, and (4) enable a (potentially automated) feedback loop for learning from the (discrete or continuous) evaluations of past decisions. The design objectives contribute by supporting development of solutions for the observed lack of ex-post methods for evaluating data-driven decisions to enhance human-machine collaboration in decision making. Our future research involves design and implementation efforts through on-going industry-academia cooperation.

Ämnesord

TEKNIK OCH TEKNOLOGIER  -- Elektroteknik och elektronik -- Datorsystem (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Electrical Engineering, Electronic Engineering, Information Engineering -- Computer Systems (hsv//eng)

Nyckelord

Data-driven decisions

ex-post evaluation

design objectives

collaborative rationality

human-machine collaboration

Information systems

Informationssystem

Publikations- och innehållstyp

ref (ämneskategori)

kon (ämneskategori)