Home > intention-kernel/testing

intention-kernel/testing package

Declarative scenario evaluation, assertions, portable reports, and in-memory test helpers.

Remarks

Import these APIs from intention-kernel/testing in the consuming application. Evaluations run through a compiled agent or an application-owned transport and return structured results for each case and acceptance criterion.

Classes

Class	Description
EvaluationError	Stable public failure for invalid definitions, reports or execution options.

Functions

Function	Description
createAgentEvaluationAdapter(options)	Adapt a compiled agent to the shared evaluation runner.
createEventCollector()	Create an in-memory sink for trace inspection and conformance assertions.
createMemoryDurability(options)	Create an in-memory durability adapter for conformance tests and local examples. It implements serialization, replay and effect semantics, but it does not survive process restarts and is not production persistence.
evaluateAssertions(assertions, context, evaluators)	Evaluate every assertion and preserve the evidence for each outcome.
parseEvaluationReport(value)	Validate an imported execution report before display or resume.
parseEvaluationSuite(value)	Parse a portable suite, rejecting unknown fields, duplicate IDs and unsafe paths.
runEvaluation(options)	Execute isolated multi-turn cases using an application-owned adapter.

Interfaces

Interface	Description
AgentEvaluationAdapterOptions	Direct-agent transport dependencies; no HTTP or provider coupling.
EvaluationAdapter	Host boundary for direct agents, HTTP chat or another conversation protocol.
EvaluationAssertionResult	Evidence and outcome of a single assertion.
EvaluationCaseResult	One scenario variant and repetition, isolated from every other case.
EvaluationContext	Immutable input to registered domain or semantic evaluators.
EvaluationEvent	Event emitted after a corresponding snapshot has been persisted.
EvaluationOptions	Runner dependencies and bounded execution controls.
EvaluationReport	Portable incremental run snapshot. Store in a host-authorized location.
EvaluationScenario	Portable, independently executable conversation definition.
EvaluationSession	Isolated transport; credentials remain here, never in report/checkpoint data.
EvaluationSessionContext	Context passed once to the transport when opening an isolated case.
EvaluationStep	One sequential exchange in a scenario.
EvaluationSuite	Versioned suite shared by CLI, services and management interfaces.
EvaluationTurnResult	One evaluated exchange, including complete host-approved evidence.
EvaluationVariant	Alternative input for one step. Variants replace the full input, not individual fields.
EventCollector	Event sink that retains emitted events for local inspection and conformance tests.

Variables

Variable	Description
TESTING_VERSION	Package version of these testing helpers, identical to the runtime's VERSION.

Type Aliases

Type Alias	Description
EvaluationAssertion	A declarative assertion or explicitly registered host evaluator.
EvaluationCaseStatus	Execution state of a matrix case.
EvaluationData	Named JSON fields supplied by the host; never executable source code.
EvaluationEvaluator	Explicitly injected evaluator; the framework never evaluates source strings or calls a model itself.
EvaluationEvaluators	Separate domain evaluators keyed by public identifiers.
EvaluationOperator	Built-in operators compare structured evidence, not model-generated judgments.
EvaluationPath	Safe own-property path. Numeric segments address array indexes.
EvaluationSummary	Counts cover every scheduled matrix case; omitted/error/blocked cases cannot inflate passes.
EvaluationValue	JSON-safe value stored in definitions, observations and portable reports.