# Knowledge representation — JSON-LD, Postgres graph, promotion ladder

Company OS keeps **operational truth in Postgres**. JSON-LD is the **portable, human-editable view** agents and external tools (Fluree, SPARQL pipelines, doc generators) can share without replacing the ledger.

Canonical context document: [`hsm-ii-context-v1.jsonld`](./hsm-ii-context-v1.jsonld)  
Published IRI: `https://permute.ai/hsm-ii/context/v1` (same terms as the file above).

Related: [world-model-and-intelligence.md](./world-model-and-intelligence.md), [self-operating-software-loop.md](./self-operating-software-loop.md).

---

## Why JSON-LD here

| Need | Markdown alone | Postgres alone | JSON-LD |
|------|----------------|----------------|---------|
| Human edit | Good | Poor (SQL) | Good (JSON) |
| Stable IDs | Weak | Strong (UUID) | Strong (`@id` / URN) |
| Relations | Manual links | `memory_edges` | `fromMemory` / `toMemory` |
| Agent + graph tooling | Weak | API-specific | RDF-ready |

**Rule:** Postgres is **system of record**; JSON-LD is **export / interchange / intent envelope**. Promote **into** Postgres first, then export JSON-LD for federation or archival.

---

## Four knowledge layers (recommended policy)

Same spirit as [world-model-and-intelligence.md](./world-model-and-intelligence.md)—made explicit for docs, agents, and JSON-LD. **Do not collapse layers** (e.g. pasting constitution into every prompt, or treating chat as operational truth).

| Layer | What | Who writes | Who reads | Format |
|-------|------|------------|-----------|--------|
| **Constitution** | Vision, principles, rarely changing company narrative | Humans (operators, DRIs) | All agents via `companies.context_markdown` + `docs/company-os/*` | Markdown |
| **Operational facts** | Decisions, policies, KPIs, playbooks, durable company knowledge | Agents + humans; **verified promote** into memory | `company_memory_search`, `GET …/tasks/:id/llm-context`, scoped tools | Postgres (`company_memory_entries`, `memory_edges`) + optional **`export-jsonld`** |
| **Procedure** | SOPs, eval packs, harness contracts, capability wiring | Engineers | Harness + eval runners + `capability_refs` on tasks | YAML / JSON (+ shared **`@context`** for interchange) |
| **Evidence** | Traces, runs, governance rows, promotion ledgers | Automation (runs, verifiers, gates) | Verifiers, auto-revert, audits, DRI inbox | JSONL artifacts + `governance_events` + file ledgers |

### Layer ↔ JSON-LD types (v1)

| Layer | Primary store | JSON-LD `@type` (when exported) |
|-------|---------------|----------------------------------|
| Constitution | `context_markdown`, git docs | (markdown; no required `@type` today) |
| Operational | `company_memory_entries`, `memory_edges` | `MemoryEntry`, `MemoryEdge` |
| Procedure | eval pack files, harness `*.filled.json`, `gate_contract` | `HarnessPromotion` (ledger); pack schema is JSON Schema |
| Evidence | `governance_events`, `agent_runs`, `improvement_runs`, `turns_*.jsonl` | `GovernanceEvent`, `ImprovementRun` (export TBD); ledger uses `HarnessPromotion` |

**Write path:** Evidence → Verify → **Promote** → Operational (memory row + `supersedes` edge). Constitution and Procedure change on human/engineer review, not on a single successful agent turn.

---

## DRI, verifier, and ledger

| Role | Responsibility | Where it lives |
|------|----------------|----------------|
| **DRI** | Owns outcome; resolves conflicts with constitution; may override when stakes demand | `dri_assignments`, task ownership, `requires_human` escalation |
| **Verifier** | Distinct persona/process from the subject that produced the change | `separate_verifier.rs`, eval-pack disagree/override, harness `approved_by` / `verified_by` |
| **Ledger** | Append-only record that promotion happened with provenance | Postgres `governance_events`, harness `promotion_*.json` (`@id`, `source_trace_id` / `source_run_id`), `improvement_runs` |

Promotion without a ledger row (or without trace/run id when gates are strict) is **integration debt**, not shipped intelligence. See [intelligence-layer-dri-alignment.md](./intelligence-layer-dri-alignment.md).

---

## Agent context, review, and compaction

Engineers should spend more time on **specs and gates**, less on re-typing context for each agent.

| Concern | Policy | Implementation hooks |
|---------|--------|----------------------|
| **Context management** | Compose from **`llm-context`** + **scoped search** (`context_key`, `artifact_type`)—not bigger static prompts | `GET …/tasks/:id/llm-context`; `company_memory_search` query params |
| **Review** | Separate verifier personas; eval-pack **disagree / override**; human inbox via **`requires_human`** | `separate_verifier.rs`, eval-pack review API, `company_task_requires_human` |
| **Compaction** | Default retrieval **L0/L1** summaries; full `body` only when needed | `surface=l0_l1` (default env `HSM_MEMORY_RETRIEVAL_SURFACE=l0_l1`); `surface=full` or `path=deep` for exhaustive hybrid |

**Anti-patterns:** stuffing constitution + all memory bodies into system prompt; one agent both proposes and approves promotion; promoting on metric regression without `allow_regression` and explicit operator id.

---

## `@context` (v1)

Terms live under `https://permute.ai/hsm-ii/vocab/`. Core types:

| JSON-LD `@type` | Meaning |
|-----------------|--------|
| `MemoryEntry` | Row in `company_memory_entries` |
| `MemoryEdge` | Row in `memory_edges` |
| `HarnessPromotion` | File ledger under `candidate/promotions/promotion_*.json` |
| `GovernanceEvent` | Row in `governance_events` (export TBD) |
| `ImprovementRun` | Row in `improvement_runs` (export TBD) |

**URN pattern (stable, offline-friendly):**

| Entity | `@id` pattern |
|--------|----------------|
| Company | `urn:hsm:company:{uuid}` |
| Memory | `urn:hsm:memory:{uuid}` |
| Memory edge | `urn:hsm:memory-edge:{uuid}` |
| Company agent | `urn:hsm:company-agent:{uuid}` |
| Harness promotion | `urn:hsm:harness-promotion:{candidate_id}:{timestamp}` |

---

## Postgres tables → JSON-LD types

| Postgres | JSON-LD type | Key properties |
|----------|--------------|----------------|
| `company_memory_entries` | `MemoryEntry` | `title`, `summaryL0`, `summaryL1`, `body`, `scope`, `contextKey`, `artifactType`, `kind`, `version`, `isLatest`, `supersedes` |
| `memory_edges` | `MemoryEdge` | `relationType`, `fromMemory`, `toMemory`, `confidence` |
| `memory_artifacts` / `memory_chunks` | (nested in inspect API; optional future `Artifact` type) | provenance for ingest |
| `governance_events` | `GovernanceEvent` | `action`, actor, subject, payload |
| `improvement_runs` | `ImprovementRun` | baseline/candidate/metrics meta |
| `tasks` | (future `Task` type) | spec, `capability_refs`, `context_notes` |

### `artifact_type` taxonomy (memory)

Distinct from `kind` (`general` | `broadcast`):

`general` · `playbook` · `kpi` · `policy_snippet` · `briefing` · `runbook`

Set via column, tag `artifact:playbook`, or ingest metadata.

### `context_key` (partition)

Business context slice (e.g. `sales`, `finance`). Filter retrieval with `?context_key=` or tag `ctx:sales` on append.

### `memory_edges.relation_type`

| DB value | Promotion ladder role |
|----------|------------------------|
| `supersedes` | **Promote** — new fact replaces old (`is_latest`) |
| `supports` | **Verify** — evidence for a candidate |
| `contradicts` | **Block** — triggers human / DRI review |
| `derives` | **Observe** — trace-derived link |
| `extends` | **Bind** — same topic, more detail |
| `updates` | **Expose** — revision without full supersession |
| `related` | Weak association |

---

## Promotion ladder ↔ graph ↔ gates

From [world-model-and-intelligence.md](./world-model-and-intelligence.md):

```text
Observe → Bind → Expose → Verify → Promote
```

| Step | Postgres / artifacts | Graph edge | Enforcement (`promotion_gates.rs`) |
|------|----------------------|------------|-----------------------------------|
| **Observe** | `agent_runs`, traces, `run_failure_events` | `derives` | — |
| **Bind** | task_id, `capability_refs`, `entity_type` | `extends` / `related` | typed ship `meta` on overlay |
| **Expose** | `llm-context`, memory L0/L1 search | — | `GET …/export-jsonld` |
| **Verify** | `company_eval_reviews`, replay, separate verifier | `supports` | `require_distinct_verifier` |
| **Promote** | `improvement_runs`, harness `promotion_*.json`, memory `is_latest` | `supersedes` | metrics + `retest_passed` + trace id |

**Do not promote** on prose-only overlay text: harness apply requires `artifact_kind`, `schema_version`, and `source_trace_id` or `source_run_id` in `meta` (server) and in promotion ledger JSON (CLI).

---

## HTTP APIs

| Endpoint | Role |
|----------|------|
| `GET /api/company/companies/{company_id}/memory/{memory_id}/export-jsonld` | Canonical JSON-LD for one memory + incident `memory_edges` |
| `GET …/memory/{memory_id}/inspect` | Full inspector; includes `jsonld_export_path` hint |
| `GET …/memory/{memory_id}/edges` | Raw edge list (JSON) |

Example:

```bash
curl -s -H "Authorization: Bearer $HSM_COMPANY_API_BEARER" \
  "$HSM_CONSOLE_URL/api/company/companies/$CID/memory/$MID/export-jsonld" | jq .
```

Response shape:

```json
{
  "@context": "https://permute.ai/hsm-ii/context/v1",
  "@id": "urn:hsm:memory:…",
  "@type": "MemoryEntry",
  "@graph": [
    { "@id": "urn:hsm:memory:…", "@type": "MemoryEntry", "title": "…", "summaryL1": "…" },
    { "@id": "urn:hsm:memory-edge:…", "@type": "MemoryEdge", "relationType": "supports", … }
  ]
}
```

---

## Harness promotion ledger (file)

Written by `scripts/meta-harness/company_os_promote_delta.py` to  
`{candidate_dir}/promotions/promotion_{timestamp}.json`.

**Required fields (non-dry-run):**

| Field | Purpose |
|-------|---------|
| `@id` | Stable promotion URN (auto-generated if omitted) |
| `source_trace_id` **or** `source_run_id` | Provenance (from filled delta or CLI flags) |
| `approved_by` | Distinct verifier (human/operator id) |
| `before_metrics` / `after_metrics` | Metric gate (e.g. `finalize_rate`) |

Optional JSON-LD aliases: `promotion_id` (= `@id`), `trace_id` (= `source_trace_id`).

Aligns with server `promotion_gates::validate_ship_artifact_meta` and improvement-run `require_promotion_evidence`.

---

## Maintenance checklist

1. **Constitution** — edit `context_markdown` / docs; do not duplicate into every agent prompt.
2. **Operational** — promote to memory; add edges; `export-jsonld` after promote if federating.
3. **Procedure** — version eval packs and harness contracts in git; wire `capability_refs` on tasks.
4. **Evidence** — every promote leaves trace/run id + governance or file ledger; verifiers are distinct from subjects.

**Anti-patterns:** authoritative state only in chat; promote without trace; ship overlay without typed `meta`; two competing ledgers without sync; retrieval `surface=full` as the default for all queries.

---

## Fluree (optional downstream)

If you adopt Fluree or another RDF store:

- **Sync direction:** Postgres → JSON-LD export → Fluree ingest (on promote or schedule).
- **Keep** Company OS SQL APIs for console, spend, tasks, and `llm-context`.
- Use SPARQL for cross-company analytics; use hybrid memory search for agent retrieval.

No Fluree dependency is required to use JSON-LD export today.