Multi-Parent Merge Workflow

Answer first

Multi-parent merging turns model breeding from a pairwise experiment into a portfolio search. Instead of asking whether parent A and parent B can be blended, the lab asks which compatible set of parents, adapter deltas, layer choices, and weights creates the best descendant for a specific niche.

Compatibility comes first

A parameter-level merge should start only when parents share the same base family, tensor schema, tokenizer contract, and artifact identity rules. When those fields do not match, the right constructive path is usually distillation, routing, or a new student model rather than raw parameter mixing.

The workflow

1. Select a compatible parent pool.
2. Normalize metadata into a shared Genome record.
3. Generate candidate merge recipes.
4. Score each candidate on utility, cost, latency, local-fit, and novelty.
5. Preserve a small archive of diverse high performers.
6. Emit a release packet only for candidates with a clear scope and rollback target.

PROCEDURE n_way_merge(parent_pool, niche_contract)
    compatible <- FILTER(parent_pool, SAME_BASE_AND_TOKENIZER)
    recipes <- SAMPLE_MERGE_RECIPES(compatible, operators = [average, sign_merge, layer_mix])
    candidates <- []
    FOR recipe IN recipes
        child <- APPLY_RECIPE(recipe)
        child.genome <- RECORD_PARENTS_OPERATOR_AND_WEIGHTS(recipe)
        child.fitness <- EVALUATE(child, niche_contract)
        candidates.ADD(child)
    END FOR
    archive <- QUALITY_DIVERSITY_SELECT(candidates, descriptor = niche_contract.behavior_descriptor)
    RETURN archive
END PROCEDURE

Why this belongs in ModelBreeder

The source reports repeatedly point toward model soups, LoRA/adapters, task vectors, evolutionary model merging, novelty search, and Quality-Diversity archives. This workflow turns those reports into a builder sequence: choose parents, make descendants, score the descendants, keep diversity, and promote only useful artifacts.