Open-source Python library for post-deployment monitoring that estimates a model's performance when ground-truth labels are delayed or missing, using confidence-based estimation for classification and direct loss estimation for regression, and links univariate and multivariate drift alerts to performance impact.
open_source · generally available · Research snapshot 2026-09-06
Demonstrate estimated versus realised ROC AUC on a tabular classifier with delayed labels, and show which drift alerts were linked to the performance change.
Capabilities and evidence
Support labels reflect the supplied research. Documentation and vendor claims are not independent product tests. “Not established” means the researcher did not find support; it does not prove a capability is absent.
Documented by provider
The NannyML repository states the library estimates post-deployment model performance without access to targets, detects data drift, links drift alerts to changes in model performance, supports tabular classification and regression, and implements confidence-based performance estimation (CBPE) and direct loss estimation (DLE), including metrics such as ROC AUC and RMSE.
Limit: Documented scope is tabular ML; the repository documents no registry, approvals or lifecycle governance features.
Multiple repository paths exist for the project (NannyML/NannyML plus forks such as nnansters/nannyml), so the canonical source should be confirmed before adoption.
Limit: Only establishes that several repository names surface publicly; ownership history was not investigated.