Introduction
When you call client.retrieve(query="..."), the query does not simply hit a vector index. It passes through a 10-phase cognitive retrieval cycle designed to mimic how biological memory systems process recall requests.
This article explains each phase, why it exists, and how it affects the results you get back.
The 10 phases
Query → Encode → Route → Activate → Spread → Decay → Cause → Rank → Filter → Score → Return
Each phase transforms the query or the candidate set, progressively narrowing from "all possible memories" to "the most cognitively relevant results."
Phase 1: Encoding
The raw query text is transformed into a multi-dimensional representation:
- Semantic embedding — captures meaning via transformer model
- Intent classification — factual, causal, temporal, or exploratory
- Entity extraction — identifies named entities and concepts
- Temporal markers — detects time references ("last week", "before the deploy")
The encoding phase determines what kind of question is being asked, which directly affects pathway routing in Phase 2.
Input: "What caused the outage last Tuesday?"
Output: {
embedding: [0.23, -0.41, ...], // 1536 dimensions
intent: "causal",
entities: ["outage"],
temporal: { reference: "last Tuesday", direction: "before" }
}
Phase 2: Pathway routing
Based on the encoded query, the engine selects one of three retrieval pathways:
| Pathway | Triggered by | Optimized for |
|---|---|---|
| Exact | Factual queries, specific entities | Precision — finds the exact memory |
| Energy | Broad/exploratory queries | Recall — surfaces related patterns |
| Attention | Complex/multi-part queries | Focus — weighs multiple signals |
When you set pathway="auto", the router makes this decision based on the encoding. You can override it, but auto-routing is correct roughly 90% of the time.
The routing decision is logged and visible via the explain endpoint:
explanation = client.explain(query="What caused the outage?")
print(explanation.pathway_scores)
# {"exact": 0.2, "energy": 0.3, "attention": 0.5} → attention pathway selected
Phase 3: Activation
The selected pathway activates a candidate set from the memory store. This is where the underlying vector search happens — but filtered through the pathway lens:
- Exact pathway: narrow ANN search with high precision threshold
- Energy pathway: broad ANN search with lower threshold, more candidates
- Attention pathway: multiple sub-queries generated from decomposed intent
The activation phase typically selects 50–200 candidate patterns from potentially millions stored.
Phase 4: Spreading activation
Inspired by Collins & Loftus (1975), spreading activation expands the candidate set by following connections:
- Causal links (caused_by, leads_to)
- Temporal proximity (events within the same time window)
- Shared metadata (same tags, same category)
A memory pattern that was not directly activated by vector similarity can enter the candidate set because it is connected to an activated pattern.
Activated: "Deploy failed at 3pm"
Spread to: "Config PR merged at 2:45pm" (temporal proximity)
Spread to: "Missing env variable" (causal link: caused_by)
The spread depth is configurable but defaults to 2 hops. Each hop reduces the activation strength by 50%.
Phase 5: Temporal decay
Every candidate is penalized based on age. The decay function follows a power law:
decay_factor = 1 / (1 + age_days * decay_rate)
Recent memories receive minimal penalty. Memories older than 30 days receive significant decay unless they have been reinforced (accessed recently or marked high-importance).
Reinforcement resets the decay clock:
effective_age = days_since_last_access (not days_since_creation)
This means a 6-month-old memory that was accessed yesterday decays like a 1-day-old memory.
Phase 6: Causal traversal
For queries classified as "causal" (intent detection in Phase 1), the engine performs directed graph traversal on causal links:
- Start from activated candidates
- Follow
caused_bylinks backward (finding causes) - Follow
leads_tolinks forward (finding effects) - Score each traversed node by path length and link confidence
This phase is what enables questions like "Why did X happen?" to return the chain of events, not just memories containing the word "why."
If the query is not causal, this phase is skipped (no compute wasted).
Phase 7: Ranking
All candidates (from activation, spreading, and causal traversal) are merged and ranked by a composite score:
score = (
semantic_similarity * 0.35 +
temporal_relevance * 0.20 +
causal_relevance * 0.20 +
importance * 0.15 +
access_frequency * 0.10
)
The weights are pathway-dependent:
- Exact: similarity weight increased to 0.55
- Energy: importance and access frequency weighted higher
- Attention: causal and temporal weights increased
Phase 8: Filtering
Post-ranking filters remove candidates that do not meet threshold criteria:
min_confidence— removes anything below the thresholdmetadata_filter— enforces exact metadata matches- Deduplication — removes near-duplicate patterns (cosine similarity > 0.95)
This phase ensures the final result set is clean and non-redundant.
Phase 9: Confidence scoring
Each surviving candidate receives a final confidence score — a normalized value between 0 and 1 representing how certain the engine is that this memory answers the query.
Confidence is not the same as similarity. A memory can be highly similar but low-confidence (e.g., it is old and unreinforced). Conversely, a memory with moderate similarity but strong causal links and recent reinforcement can score high confidence.
results = client.retrieve(query="deploy schedule", min_confidence=0.7)
for r in results:
print(f"{r.confidence:.0%} — {r.text}")
# 94% — The deployment window is Tuesday 2-4pm UTC
# 82% — Deploy freeze starts Friday for release cut
# 71% — Last deploy was delayed due to staging issues
Phase 10: Response assembly
The final phase assembles the response:
- Results ordered by confidence (descending)
- Pathway metadata attached (which pathway was used)
- Timing data included (how long each phase took)
- Truncated to
top_kresults
The complete response includes everything needed for transparency:
results = client.retrieve(query="...", top_k=5)
# Each result contains:
# - text, pattern_id, confidence, pathway
# - metadata, created_at, last_accessed_at
# - causal_links (if traversed)
Performance characteristics
| Phase | Typical latency | Compute |
|---|---|---|
| Encoding | 5–15ms | Transformer inference |
| Routing | under 1ms | Rule-based classification |
| Activation | 10–50ms | ANN search |
| Spreading | 5–20ms | Graph traversal |
| Temporal decay | under 1ms | Mathematical function |
| Causal traversal | 5–30ms | Directed graph BFS |
| Ranking | 2–5ms | Score computation |
| Filtering | under 1ms | Threshold checks |
| Confidence scoring | under 1ms | Normalization |
| Response assembly | under 1ms | Serialization |
Total p50 latency: 35–80ms for most queries. Causal queries with deep traversal can reach 120ms.
Inspecting the cycle
The explain endpoint exposes the full cycle for any query:
explanation = client.explain(query="What caused the outage?", level="debug")
print(f"Intent: {explanation.intent}")
print(f"Pathway: {explanation.pathway_selected}")
print(f"Candidates activated: {explanation.candidates_activated}")
print(f"After spreading: {explanation.candidates_after_spread}")
print(f"After filtering: {explanation.candidates_final}")
print(f"Phase timings: {explanation.phase_timings}")
This is invaluable for debugging retrieval quality — if results are poor, the explain output tells you exactly which phase is dropping relevant candidates.
Tuning the cycle
You can influence the cycle through query parameters:
| Parameter | Affects phase | Effect |
|---|---|---|
pathway | Phase 2 | Override automatic routing |
temporal_weight | Phase 5, 7 | Increase/decrease recency bias |
causal_weight | Phase 6, 7 | Increase/decrease causal traversal depth |
min_confidence | Phase 8 | Stricter or looser filtering |
top_k | Phase 10 | More or fewer results |
For most use cases, the defaults are correct. Tune when you have specific retrieval quality feedback.
Conclusion
The 10-phase cycle is why Engramma returns different results than a vector database for the same query. Similarity search is one phase (Phase 3) out of ten. The other nine phases add temporal awareness, causal reasoning, importance weighting, and cognitive relevance scoring.
The cycle is transparent (via explain), tunable (via parameters), and fast (sub-100ms for most queries). It represents a fundamentally different approach to retrieval — one designed for memory, not search.
Tags
Share