Predictive performance of internal and external validation procedures

Baverel, Paul GUppsala universitet,Institutionen för farmaceutisk biovetenskap,Farmakometri (författare)

Purpose: To compare estimates of predictive performance between internal (IV) and external data-splitting (EV) validation procedures. Methods: Datasets of different study size (n=6, 12, 24, 48, 96, 192, or 384 individuals) were simulated from a one compartment, first-order absorption, pharmacokinetic model and both parametric (FOCE), and nonparametric (NONP) parameter estimates were obtained in NONMEM. From these, three different validation procedures (IV, EV, and a population validation (PV)) were undertaken by means of numerical predictive checks (NPCs) to provide estimates of predictive performance, the PV procedure serving as a reference to assess performance of IV and EV. The predictive performance of NONP versus FOCE estimates was further assessed. Results: Estimates of predictive performance for predicting the median of the population distribution had in general significantly lower imprecision for IV than EV, with little bias for both procedures. For small study sizes, n=6-12 (FOCE) or n=6-24 (NONP), the tails of the population distribution were significantly more biased with IV than EV, but similar imprecision was obtained. The predictive performance for FOCE was similar or superior to that of NONP. Conclusions: Data-splitting is inferior to IV when evaluating predictive models to retain sufficient precision both in predictions and in estimates of predictive performance.

Karlsson, Kristin EUppsala universitet,Institutionen för farmaceutisk biovetenskap,Farmakometri(Swepub:uu)krkar932 (författare)
Karlsson, Mats OUppsala universitet,Institutionen för farmaceutisk biovetenskap,Farmakometri(Swepub:uu)matskarl (författare)
Uppsala universitetInstitutionen för farmaceutisk biovetenskap (creator_code:org_t)

Av författaren/redakt...: Baverel, Paul G; Karlsson, Kristi ...; Karlsson, Mats O

Kungliga biblioteket hanterar dina personuppgifter i enlighet med EU:s dataskyddsförordning (2018), GDPR. Läs mer om hur det funkar här.
Så här hanterar KB dina uppgifter vid användning av denna tjänst.