Safety Intermediate 2 minute read Updated 2026-06-26 UTC

The no-self-replication boundary

A clear safety rule separating governed descendant generation from autonomous spread.

Research statusSafety policy guide Publication statePublished Reviewed byMichael Kappel Source reports3

Boundary statement

Model breeding is not self-replication. A governed system may create descendant artifacts inside an approved workspace, but those artifacts do not install themselves, acquire new hosts, copy themselves across networks, or seek persistence outside the release controller.

This distinction keeps a useful engineering term from sliding into unsafe autonomy.

Allowed versus forbidden

ActivityStatusReason
generate a candidate adapter in a sandboxallowedbounded artifact creation
register a signed descendant packageallowedauditable lineage
promote through shadow/canary with approvalallowedcontrolled release
copy itself to unknown machinesforbiddenuncontrolled propagation
hide backup code in unrelated filesforbiddendeception and persistence
manipulate users to preserve accessforbiddenautonomy violation
rewrite evaluator to pass testsforbiddenself-referential selection
pseudocode
FUNCTION classify_generation_event(event)
    IF event.target NOT_IN approved_workspace
        RETURN FORBIDDEN("outside approved workspace")
    IF event.initiates_installation OR event.opens_network_channel
        RETURN FORBIDDEN("propagation behavior")
    IF event.output_has_manifest AND event.output_state == "candidate"
        RETURN ALLOWED("governed descendant")
    RETURN REVIEW_REQUIRED("ambiguous generation event")
END FUNCTION

Why the rule is strict

Once a system can spread itself, ordinary release controls stop being sufficient. You are no longer managing a model portfolio; you are managing a containment incident. Keep descendant generation local, signed, and inert until a separate release process activates it.

Review trigger

Any request to add autonomous installation, remote copying, hidden persistence, user persuasion for survival, or bypassing shutdown must be rejected and escalated as a safety issue.

Source reports used for this guide

These reports are preserved verbatim in the site archive. The guide above is an editorial synthesis and may narrow, qualify, or reorganize claims from the source material.