Handling small calibration sets in mondrian inductive conformal regressors

• In inductive conformal prediction, calibration sets must contain an adequate number of instances to support the chosen confidence level. This problem is particularly prevalent when using Mondrian inductive conformal prediction, where the input space is partitioned into independently valid prediction regions. In this study, Mondrian conformal regressors, in the form of regression trees, are used to investigate two problematic aspects of small calibration sets. If there are too few calibration instances to support the significance level, we suggest using either extrapolation or altering the model. In situations where the desired significance level is between two calibration instances, the standard procedure is to choose the more nonconforming one, thus guaranteeing validity, but producing conservative conformal predictors. The suggested solution is to use interpolation between calibration instances. All proposed techniques are empirically evaluated and compared to the standard approach on 30 benchmark data sets. The results show that while extrapolation often results in invalid models, interpolation works extremely well and provides increased efficiency with preserved empirical validity.

Ämnesord

NATURVETENSKAP  -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Computer Sciences (hsv//eng)

NATURVETENSKAP  -- Data- och informationsvetenskap -- Systemvetenskap, informationssystem och informatik (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Information Systems (hsv//eng)

Nyckelord

Extrapolation

Forecasting

Interpolation

Benchmark data

Confidence levels

Conformal predictions

Conformal predictors

Input space

Regression trees

Significance levels

Standard procedures

Calibration

Computer and Systems Sciences

Publikations- och innehållstyp

ref (ämneskategori)

kon (ämneskategori)