Sökning: id:"swepub:oai:DiVA.org:kth-290868" >
Time travel and pro...
Time travel and provenance for machine learning pipelines
-
- Ormenisan, Alexandru-Adrian (författare)
- KTH,Programvaruteknik och datorsystem, SCS,Logical Clocks AB, Sweden
-
Meister, M. (författare)
-
Buso, F. (författare)
-
visa fler...
-
Andersson, R. (författare)
-
Haridi, S. (författare)
-
- Dowling, Jim (författare)
- KTH,Programvaruteknik och datorsystem, SCS
-
visa färre...
-
(creator_code:org_t)
- USENIX Association, 2020
- 2020
- Engelska.
-
Ingår i: OpML 2020 - 2020 USENIX Conference on Operational Machine Learning. - : USENIX Association.
- Relaterad länk:
-
https://urn.kb.se/re...
Abstract
Ämnesord
Stäng
- Machine learning pipelines have become the defacto paradigm for productionizing machine learning applications as they clearly abstract the processing steps involved in transforming raw data into engineered features that are then used to train models. In this paper, we use a bottom-up method for capturing provenance information regarding the processing steps and artifacts produced in ML pipelines. Our approach is based on replacing traditional intrusive hooks in application code (to capture ML pipeline events) with standardized change-data-capture support in the systems involved in ML pipelines: the distributed file system, feature store, resource manager, and applications themselves. In particular, we leverage data versioning and time-travel capabilities in our feature store to show how provenance can enable model reproducibility and debugging.
Ämnesord
- TEKNIK OCH TEKNOLOGIER -- Elektroteknik och elektronik (hsv//swe)
- ENGINEERING AND TECHNOLOGY -- Electrical Engineering, Electronic Engineering, Information Engineering (hsv//eng)
Nyckelord
- File organization
- Machine learning
- Metadata
- Pipeline processing systems
- Application codes
- Bottom up methods
- Data versioning
- Distributed file systems
- Machine learning applications
- Processing steps
- Reproducibilities
- Resource managers
- Pipelines
Publikations- och innehållstyp
- ref (ämneskategori)
- kon (ämneskategori)