Memory without provenance is a liability
Embeddings must not erase source, trust, sensitivity, permission, expiry, or the right to say no.
- problem
- Embedding similarity can retrieve a claim while erasing whether the claim is trusted, sensitive, current, or permitted for a side effect.
- scope
- A typed memory-policy boundary covering provenance, trust, sensitivity, allowed use, expiry, invalidation, and autonomous-action prohibitions.
- environment
- Agent memory pipelines that retrieve personal context and may later propose tools, messages, publication, code, or device actions.
Assumptions
- Provenance metadata remains attached through ingestion, retrieval, and promotion.
- Retrieval relevance and action permission are evaluated as separate decisions.
Limitations
- Private memory content and deployment topology are outside this public note.
- Policy metadata reduces ambiguity; it does not prove that every downstream tool is correctly sandboxed.
Table of contents 4 sections
Preserve provenance before similarity
A memory without provenance is not memory. It is a rumor with good embeddings. Similarity can retrieve a claim; it cannot tell me whether the claim was user-stated, inferred, externally sourced, stale, sensitive, or permitted for the action under consideration.
The ingestion boundary therefore records an immutable source identity, observed time, subject, producer, extraction method, and lineage to the exact material that produced the candidate. Corrections do not overwrite the old fact in place: they create a superseding record and preserve the contradiction edge. That gives retrieval enough context to say “this was once believed, later corrected” instead of presenting the most recent vector hit as timeless truth.
Represent policy as data, not prompt prose
Those concerns are represented as policy, not prompt prose. New candidates default to weak trust, internal sensitivity, retrieval-only use, and explicit prohibitions on autonomous side effects. Confirmation can strengthen a decision without rewriting the source that earned it.
The schema uses closed enums and explicit dates for trust class, sensitivity, retention, review status, and allowed action families. Unknown does not collapse into false, and absence never upgrades permission. A policy evaluator returns both the decision and machine-readable reasons, so logs and tests can show whether a memory was blocked because it expired, lacked confirmation, crossed a sensitivity boundary, or prohibited the requested side effect.
Separate retrieval from permission
Retrieval and permission are separate gates. A highly relevant memory may still be forbidden from sending a message, running code, publishing content, or controlling a device. Personalization is not ambient authority.
The first gate ranks candidate context for the current question. The second evaluates the proposed use against source policy, current user intent, tool risk, and execution scope. A third execution gate can still require confirmation immediately before an external effect. Keeping these decisions distinct prevents a retrieval-score refactor from silently changing authorization and lets a memory remain useful for conversation while staying unavailable to autonomous tools.
Make the boundary executable
The proof surface is executable: typed policy, conservative lifecycle defaults, adapters, and tests for weak-trust effects and explicit prohibitions. If the boundary matters, it belongs in code.
The adversarial test matrix pairs high semantic relevance with low authority: inferred personal data, stale operational facts, corrected preferences, sensitive source material, expired instructions, and explicit “never do this” constraints. The expected outcome is not merely denial; the evaluator must preserve the prohibition through retrieval, summarization, promotion, and tool planning, while emitting an audit reason that an operator can inspect without exposing the protected content itself.
Split relevance from permission
- observation
- A relevant memory can still be stale, sensitive, inferred, or explicitly forbidden from driving an external action.
- hypothesis
- A separate permission gate with conservative defaults will prevent similarity score from becoming ambient authority.
- change
- Attach provenance and policy metadata to each memory, then evaluate action permission after retrieval instead of inside the retrieval score.
- measurement
- Lifecycle and side-effect tests verify weak-trust defaults, expiry behavior, and explicit prohibitions across high-relevance retrieval cases.
- mechanism
- The second gate reasons over authority and sensitivity rather than semantic similarity, so relevance cannot silently upgrade permission.
- decision
- Keep retrieval and action permission as independent typed gates.
- implemented Typed policy model
Source, trust, sensitivity, allowed and prohibited use, expiry, lineage, and audit gates are represented as data rather than prompt prose.
- implemented Lifecycle and side-effect tests
Weak-trust memories retain conservative defaults, while explicit prohibitions remain effective even when retrieval relevance is high.
No public artifact is attached to this record. The limitation is explicit rather than replaced by a decorative link.