Why this matters
A browser-local LLM runtime is the clearest technical expression of ModelBreeder.com's positive side: private work stays local, latency falls, small models become useful, and model packages can be copied, hashed, evaluated, and improved without central cloud dependency.
The zero-dependency Rust direction makes this more credible. A small trusted runtime can load a .slm model, validate tensor layout, apply adapter deltas, run deterministic sampling, emit diagnostics, and expose a handwritten WASM boundary without requiring a large JavaScript or Python inference stack.
Improvement roadmap
| Layer | Positive improvement | Why it helps model breeding |
|---|---|---|
| Math kernels | Add WebAssembly SIMD128 matvec paths for f32, q8, and q4 storage. | Faster evaluation lets more candidates be tested locally. |
| Quantization | Extend flat q4/q8 into hierarchical block formats when feasible. | Better quality per byte keeps specialists small. |
| Adapter deltas | Preserve raw, sparse, and low-rank delta packages with compatibility digests. | Descendants can be compact and auditable. |
| Tokenization | Keep tokenizer sections embedded in .slm packages. | Model artifacts remain self-contained. |
| Sampling | Use deterministic seeds, top-k, top-p, and fixed candidate buffers. | Experiments can be replayed exactly. |
| KV cache | Move toward paged or prefix-aware cache records. | Long sessions become faster and more memory-aware. |
| Diagnostics | Track tokens/sec, cache length, adapter count, and assembly checksum. | Fitness vectors can include real runtime evidence. |
| WASM boundary | Keep zero-copy typed-array transfer and narrow exports. | Browser tools remain fast and easy to audit. |
Candidate evaluation loop
PROCEDURE evaluate_browser_candidate(model_package, adapter_package, eval_cases)
runtime <- INIT_TINY_RUST_LM()
model_result <- runtime.LOAD_MODEL(model_package)
REQUIRE model_result == OK
IF adapter_package EXISTS
REQUIRE runtime.VALIDATE_ADAPTER(adapter_package) == OK
REQUIRE runtime.APPLY_ADAPTER(adapter_package) == OK
END IF
runtime.CONFIGURE_SAMPLING(temperature: 0, top_k: 1, top_p: 1, seed: 1)
evidence <- []
FOR case IN eval_cases
output <- runtime.GENERATE(case.prompt, case.max_new_tokens)
diagnostics <- runtime.READ_DIAGNOSTICS()
evidence.ADD(COMPARE(case.expected, output, diagnostics))
END FOR
RETURN BUILD_FITNESS_VECTOR(evidence)
END PROCEDUREDesign principle
The runtime should make local experimentation joyful: load a package, apply a delta, run cases, inspect diagnostics, build a release packet, and keep the whole evidence trail portable.
Source reports used for this guide
These reports are preserved verbatim in the site archive. The guide above is an editorial synthesis and may narrow, qualify, or reorganize claims from the source material.