Skip to content

Context Compiler

If you are trying to understand StatePlane as a user, read Getting Started first. This page explains what happens under the hood after that onboarding example calls StatePlaneSession(...).transform(...).

The current compiler slice answers a narrow but important question:

Given a set of candidate context items and a token budget, what should actually be sent to the model, and why?

In the onboarding example, five candidate items go in. The compiler keeps four of them and excludes the irrelevant large file before provider rendering.

Current Pipeline

  1. Normalization turns known inputs into typed ContextItem objects.
  2. Validation checks ids, provenance, stable hashes, and request budget invariants.
  3. Selection chooses required items first, then optional items by deterministic priority.
  4. Reporting joins the resulting decisions back to source items and produces inspectable receipts.

What Is Deterministic

  • stable JSON serialization
  • stable hashing for item identity
  • deterministic id helpers
  • latest-user selection rules
  • explicit required-item inclusion
  • required constraint inclusion
  • optional-item sort order
  • report rendering and content previews

What Is Required Today

The current selection slice always requires:

  • every context item marked required=True
  • the latest user_msg
  • every constraint

If required items are expired or exceed the usable budget, compilation fails with structured issues instead of silently dropping them.

What Is Not Implemented Yet

The current compiler does not yet handle:

  • deduplication
  • compression
  • redaction
  • dependency expansion
  • relevance scoring beyond the current deterministic sort
  • provider rendering inside context_core
  • persistence or replay execution

Core Entry Points

  • prepare_openai_request() uses these steps under the SDK surface
  • normalize_user_message()
  • normalize_text_context()
  • normalize_json_context()
  • validate_context_items()
  • validate_compilation_request()
  • compile_context()
  • build_compilation_report()

See Context Items and Receipts for the data model behind those steps.