Why quality alone is insufficient
Selecting only the highest aggregate score can eliminate specialists that are essential for rare tasks, different hardware, or future adaptation. Quality-diversity methods retain strong candidates across a behavior space.
Behavior descriptors
Descriptors define niches. Useful descriptors for model systems include:
- task family or domain;
- language or modality;
- latency and memory class;
- calibration or abstention profile;
- robustness to specific perturbations;
- explanation style or output structure;
- data jurisdiction;
- tool or network permission level;
- complementarity with current population.
Descriptors should be measurable and relevant. Arbitrary high-dimensional embeddings can create niches that are difficult to interpret or govern.
MAP-Elites style archive
archive <- MAP_FROM_NICHE_TO_ELITE()
FOR each candidate IN evaluated_candidates
niche <- DESCRIBE_BEHAVIOR(candidate.evidence)
incumbent <- archive[niche]
IF incumbent IS NULL OR candidate.viability > incumbent.viability
archive[niche] <- candidate
END IF
END FORArchive versus production
The archive is not the active runtime population. It preserves stepping stones and alternatives. The production registry contains only approved packages. Moving an archived elite into production still requires current evaluation and release gates.
Diversity metrics
Measure behavioral distance on relevant slices, disagreement entropy, error correlation, niche coverage, lineage distance, and resilience after removing one specialist. Parameter distance alone may not correspond to useful behavioral diversity.
Cost controls
Quality-diversity archives can grow indefinitely. Limit niche resolution, retain top-k per niche only when justified, age out obsolete environments, and compress evidence. Use a separate cold archive for scientific reproducibility.
Avoiding novelty theater
Novel output is not inherently valuable. A candidate should enter the archive only if it is valid, safe, and sufficiently competent within a defined niche. Novelty can guide exploration, but hard invariants still apply.
Population seeding
Archived elites can seed new experiments when workloads shift. Select parents based on niche complementarity and lineage risk rather than only top aggregate score.
Source reports used for this guide
These reports are preserved verbatim in the site archive. The guide above is an editorial synthesis and may narrow, qualify, or reorganize claims from the source material.