Dependency is an ecological risk
A model ecology can become dangerous even when every individual answer is helpful. If users cannot leave, cannot understand decisions, cannot export their knowledge, or lose core skills, the system has become parasitic rather than mutualist.
Design controls
| Control | Implementation |
|---|---|
| Export | open formats for user data, prompts, reports, and decisions |
| Explain | show evidence and uncertainty, not only final outputs |
| Train | include teach-back and review modes |
| Rotate | periodically assign humans to perform critical steps |
| Limit | avoid engagement loops that reward dependence |
| Substitute safely | keep alternative providers and manual workflows documented |
Dependency review
FUNCTION dependency_review(product_area, evidence, policy)
exit_score <- MEASURE_EXIT_FRICTION(product_area)
retention_score <- MEASURE_HUMAN_CAPABILITY_RETENTION(product_area)
lock_in_score <- MEASURE_DATA_AND_WORKFLOW_LOCK_IN(product_area)
IF exit_score > policy.max_exit_friction
RETURN BLOCK_RELEASE("Exit path is insufficient")
END IF
IF retention_score < policy.min_capability_retention
RETURN OPEN_RISK("Users are losing independent capability")
END IF
IF lock_in_score > policy.max_lock_in
RETURN REQUIRE_PORTABILITY_WORK()
END IF
RETURN PASS
END FUNCTIONAvoid false mutualism
A system that makes users feel supported while quietly reducing their options is not mutualist. Measure independent capability, not only satisfaction.
Operational drills
Run reduced-assistance drills for critical workflows. Can a team still triage an incident, read a model card, rollback a release, and explain a decision without the AI doing the work? If not, the system is carrying hidden organizational fragility.
Source reports used for this guide
These reports are preserved verbatim in the site archive. The guide above is an editorial synthesis and may narrow, qualify, or reorganize claims from the source material.