Search: onr:"swepub:oai:DiVA.org:kau-94905" >
A domain-region bas...
A domain-region based evaluation of ML performance robustness to covariate shift
-
- Bayram, Firas (author)
- Karlstads universitet,Institutionen för matematik och datavetenskap (from 2013)
-
- Ahmed, Bestoun S., 1982- (author)
- Karlstads universitet,Institutionen för matematik och datavetenskap (from 2013),Czech Technical University in Prague, Czech Republic
-
(creator_code:org_t)
- Springer, 2023
- 2023
- English.
-
In: Neural Computing & Applications. - : Springer. - 0941-0643 .- 1433-3058. ; 35:24, s. 17555-17577
- Related links:
-
https://doi.org/10.1...
-
show more...
-
https://kau.diva-por... (primary) (Raw object)
-
https://urn.kb.se/re...
-
https://doi.org/10.1...
-
show less...
Abstract
Subject headings
Close
- Most machine learning methods assume that the input data distribution is the same in the training and testing phases.However, in practice, this stationarity is usually not met and the distribution of inputs differs, leading to unexpectedperformance of the learned model in deployment. The issue in which the training and test data inputs follow differentprobability distributions while the input–output relationship remains unchanged is referred to as covariate shift. In thispaper, the performance of conventional machine learning models was experimentally evaluated in the presence of covariateshift. Furthermore, a region-based evaluation was performed by decomposing the domain of probability density function ofthe input data to assess the classifier’s performance per domain region. Distributional changes were simulated in a twodimensional classification problem. Subsequently, a higher four-dimensional experiments were conducted. Based on theexperimental analysis, the Random Forests algorithm is the most robust classifier in the two-dimensional case, showing thelowest degradation rate for accuracy and F1-score metrics, with a range between 0.1% and 2.08%. Moreover, the resultsreveal that in higher-dimensional experiments, the performance of the models is predominantly influenced by the complexity of the classification function, leading to degradation rates exceeding 25% in most cases. It is also concluded that themodels exhibit high bias toward the region with high density in the input space domain of the training samples.
Subject headings
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Sciences (hsv//eng)
Keyword
- Degradation
- Function evaluation
- Input output programs
- Machine learning
- Probability distributions
- Classifier evaluation
- Concept drifts
- Covariate shifts
- Input datas
- Machine-learning
- Model degradations
- Performance
- Region-based
- Robust machine learning
- Two-dimensional
- Probability density function
- Computer Science
- Datavetenskap
Publication and Content Type
- ref (subject category)
- art (subject category)
Find in a library
To the university's database