docs(whaleflow): define external memory cutline (#2842)

This commit is contained in:
Hunter Bown
2026-06-05 23:18:34 -07:00
committed by GitHub
parent 7fc4ec820a
commit a7052751e4
4 changed files with 81 additions and 1 deletions
+3
View File
@@ -89,6 +89,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
stored on teacher candidates, and a deterministic PromotionGate compares stored on teacher candidates, and a deterministic PromotionGate compares
baseline-vs-candidate replay deltas, required tests, policy violations, baseline-vs-candidate replay deltas, required tests, policy violations,
staleness, and cost constraints before marking a candidate promotable (#2675). staleness, and cost constraints before marking a candidate promotable (#2675).
The external-memory cutline now documents that Aleph-style memory stays
optional, explicit, visible, and clear/export-capable for v0.9.0 rather than
becoming a hidden default context substrate (#2677).
Thanks @AdityaVG13 for the WhaleFlow draft and cost-tracking direction. Thanks @AdityaVG13 for the WhaleFlow draft and cost-tracking direction.
- Added a state-store v2 schema migration for WhaleFlow trace tables covering - Added a state-store v2 schema migration for WhaleFlow trace tables covering
workflow, branch, leaf, control-node, and teacher-candidate runs. The workflow, branch, leaf, control-node, and teacher-candidate runs. The
+3
View File
@@ -89,6 +89,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
stored on teacher candidates, and a deterministic PromotionGate compares stored on teacher candidates, and a deterministic PromotionGate compares
baseline-vs-candidate replay deltas, required tests, policy violations, baseline-vs-candidate replay deltas, required tests, policy violations,
staleness, and cost constraints before marking a candidate promotable (#2675). staleness, and cost constraints before marking a candidate promotable (#2675).
The external-memory cutline now documents that Aleph-style memory stays
optional, explicit, visible, and clear/export-capable for v0.9.0 rather than
becoming a hidden default context substrate (#2677).
Thanks @AdityaVG13 for the WhaleFlow draft and cost-tracking direction. Thanks @AdityaVG13 for the WhaleFlow draft and cost-tracking direction.
- Added a state-store v2 schema migration for WhaleFlow trace tables covering - Added a state-store v2 schema migration for WhaleFlow trace tables covering
workflow, branch, leaf, control-node, and teacher-candidate runs. The workflow, branch, leaf, control-node, and teacher-candidate runs. The
+3 -1
View File
@@ -280,7 +280,9 @@ clearable, and scoped**:
`finalize`/`FINAL` is an *in-kernel Python function*, not a tool). `finalize`/`FINAL` is an *in-kernel Python function*, not a tool).
6. **Cached-main overlay** — promoted lessons from the cached main branch 6. **Cached-main overlay** — promoted lessons from the cached main branch
(`/overlay`, §9). (`/overlay`, §9).
7. **External memory (Aleph)** — large local data via the `aleph` skill. 7. **External memory (Aleph)** — large local data via the `aleph` skill;
see `docs/WHALEFLOW_EXTERNAL_MEMORY.md` for the v0.9.0 cutline that keeps
this optional, explicit, inspectable, and out of the default path.
**Why it helps weaker models.** The model never has to *guess* where a fact **Why it helps weaker models.** The model never has to *guess* where a fact
should live or *re-derive* context it already established. Each layer has a should live or *re-derive* context it already established. Each layer has a
+72
View File
@@ -0,0 +1,72 @@
# WhaleFlow External Memory Cutline
This note resolves the v0.9.0 cutline for Aleph-style external memory in
WhaleFlow. It is a design boundary, not a runtime implementation.
## Decision
External memory should be optional and explicit for v0.9.0. Normal CodeWhale
operation must not depend on it, and WhaleFlow must not silently enable it for
long-running runs.
For v0.9.0, external memory can appear only as:
- an explicit workflow node whose inputs, outputs, scope, and permissions are
visible in the typed WhaleFlow IR;
- an optional plugin or skill-backed tool that the user enables deliberately;
- a documented experiment whose state can be inspected, cleared, and exported.
It should not be a hidden context substrate, a replacement for repo search, or a
default backing store for every workflow run.
## Layer Boundaries
External memory is separate from the existing memory and replay layers:
| Layer | Scope | v0.9.0 rule |
| --- | --- | --- |
| User memory | Small durable user preferences and facts surfaced by `/memory` | Opt-in, user-owned, not workflow evidence |
| Repo search / codemap | Derived repo structure and search results | Rebuildable from the workspace; not a memory log |
| ARMH/RLM memo | In-session working memory and exact-context memoization | Visible hit/miss telemetry; not durable replay evidence |
| TraceStore | Recorded workflow, branch, leaf, and control results | Source of deterministic replay; no live model calls during replay |
| Cached-main overlay | Promoted lessons after review and replay | Inspectable and reversible; never mutates Git main |
| External memory | Large local or plugin-backed data outside normal context | Explicit node/plugin only; visible state and clear/export required |
## Visibility Requirements
Any future external-memory implementation must show:
- when it is active;
- which workflow node or plugin owns it;
- where its state is stored;
- what repo or run scope it can read;
- whether it is included in replay, export, or promotion evidence;
- how to inspect, clear, pin, and export it.
The UI should treat this like an active context layer, not like invisible model
intuition. If a run cannot explain why a fact came from external memory, the
feature is not ready for default use.
## Permissions And Privacy
External memory must inherit the strictest relevant scope:
- it must not cross repo/workspace boundaries without explicit approval;
- project-local config must not silently enable broad external-memory reads;
- replay must record external-memory inputs as evidence or mark replay as
unavailable/diverged;
- exports must make external-memory references visible without dumping private
raw state by default.
## Deferred Work
The following remain out of scope for the v0.9.0 cutline:
- default-on Aleph-style memory for all WhaleFlow runs;
- automatic promotion from external memory into cached-main overlay;
- hidden retrieval behind ordinary prompts;
- hosted or shared external-memory services;
- treating external memory as a substitute for TraceStore replay.
Future implementation should start with a read-only typed workflow node and a
mock replay fixture before adding any plugin-backed or live retrieval path.