Objective
Maintain classification quality as input distributions and category definitions change, without updating a production model in place or forgetting historical classes.
Components
- versioned taxonomy contract;
- champion classifier and rollback package;
- drift detector;
- failure-cluster and label-review workflow;
- replay dataset with retention rules;
- candidate trainer and calibrator;
- frozen historical and current evaluation suites;
- shadow and canary release controller.
Adaptation trigger
Trigger only after persistent drift, a new approved category, or a material failure cluster. Distinguish data drift from taxonomy drift: a new label changes the contract, not only the model.
Descendant training
PROCEDURE adapt_classifier(champion, new_labels, replay, taxonomy)
train_set <- BALANCE_AND_DEDUPLICATE(new_labels + replay)
candidates <- TRAIN_VARIANTS(champion, train_set, bounded_recipes)
FOR each candidate IN candidates
candidate <- CALIBRATE(candidate, calibration_split)
evidence <- EVALUATE(
candidate,
suites = [historical_suite, current_suite, critical_slices]
)
RECORD(candidate, evidence)
END FOR
RETURN SELECT_IF_NO_FORGETTING_AND_NET_GAIN()
END PROCEDUREForgetting controls
Maintain replay coverage for historical categories, track per-class recall and confusion, and compare representation or score drift where useful. A candidate that improves new classes but materially harms protected historical classes is rejected or routed only to a narrower contract.
Taxonomy migration
Version label definitions and provide mapping from old to new categories. Downstream systems must opt into the new contract. Do not silently reinterpret historical labels.
Metrics
Macro and weighted F1, critical-class recall, calibration, abstention, drift detection delay, human-label volume, forgetting score, adaptation cost, and rollback time.
Safety
High-impact categories require human confirmation. Model output cannot directly trigger irreversible action without a separate policy gate. Training examples retain source and reviewer provenance.
Source reports used for this guide
These reports are preserved verbatim in the site archive. The guide above is an editorial synthesis and may narrow, qualify, or reorganize claims from the source material.