Structured Answers from Conversational Agents
Context
Callers want an agent's answer in a JSON shape they choose: fields read from an attached document, a classification, a summary with a fixed structure, answered as JSON instead of prose (#2342, #2343). Plain models already do this through OpenAI's response_format and forced function calls. Agents did not:
- the answer travelled only as text in
LLMStopEvent.output_messages; UserMessageEventhad no field for the request;- the OpenAI-compatible endpoint accepted
response_format,toolsandtool_choicefor agents and dropped them without a word.
The runtime schemas of record extraction (ADR 2026_10_02_runtime_record_schemas_for_structured_extraction) cover flat primitive fields only, because jambo reads nothing but properties. A caller's schema has nested objects and arrays.
Decision Drivers
- Native: the model that writes the answer is asked for the shape, the way it is asked for tools. No second pipeline next to the conversation.
- Every chat blueprint, without each one opting in: a blueprint must not be able to forget the shape and answer prose.
- A caller who asked for a shape gets it or an error, never prose.
- One request in the protocol, whatever API the caller used.
- Visible in the trace with its cost, like every other model call.
Decision
- The request is
UserMessageEvent.response_schema, a coreResponseSchema(name,description,json_schema,strict). Ajson_schemaofNonemeans any JSON object, OpenAI'sjson_object, asked of the model as the schema{"type": "object"}, since providers retirejson_object(Infomaniak refuses it). The schema is passed to the model as is and validated withjsonschema, so nested objects and arrays work. A schema may reference only its own definitions (#/...) and is validated against an empty registry, since resolving any other$refwould make the agent worker fetch a URL the caller chose.NaN,Infinityand numbers beyond a float are violations: Python parses them, but strict JSON clients do not. The schema's patterns run on theregexengine against one time budget per check (ResponseSchemaValidator), not on Python'sre: the caller writes both the pattern and, through the prompt, the answer, and one backtracking pattern onrefroze the agent's event loop for every run. A schema may therefore not combineunevaluatedPropertieswithpatternProperties, which jsonschema matches onreitself. There is no separate forced-function variant: whether the result comes back ascontentor as a tool call is decided by the OpenAI endpoint, which remembers how it was asked. - The result is
LLMStopEvent.structured_output, the parsed and validated JSON. The answer text holds the same JSON, so clients that read text keep working. RAG and Expert RAG end on their own stop events and have no place for it. - The answering model call is asked for the shape.
EventDisplayer.display_llm_stream(..., response_schema=...)sendsresponse_formatnext totoolswhen the model declaressupports_response_schema.- LLM Wrapping, Web Fetch and Few-Shot pass the schema in their answering step.
- The tool loop passes it only on a turn without tools. Sent next to the tools on a live check, the shape decided the first turn: the Universal Agent answered at once from the schema alone instead of reading the attached file. A reply given with tools is put into the shape by the completion.
- Nothing is added to the prompt. How a value is filled, including whether a missing one is null, is the caller's to say in the schema's descriptions and their message, as with OpenAI itself. An instruction such as "take values only from the documents" was tried and dropped: it suits extraction and contradicts schemas that ask the model to derive values.
- The schema reaches these steps through the run context, read with
RequestedResponseSchema, a sibling ofRequestedFeatures.
Conversation.completevalidates. With a schema requested, the completion parses the answer instead of generating follow-up questions.- When the answer does not match,
StructuredAnswerasks the model once more, non-streamed and with thinking off (ReasoningFreeChat), from the same prompt with the validation error and the schema. A second miss fails the run. - The same prompt is kept in the run context by the call that asked for the shape (
StructuredAnswer.keep_prompt): the completion carries only the conversation (LLMEvent.for_hand_back), since the documents the model read made the events ending a turn too large for the broker, and a correction without them had to invent the values it was asked to fix. The run context is not bound by that limit and goes with the run. A tool loop keeps the prompt with its tool calls as text, and without a kept prompt the conversation is used with its result-less tool calls taken out, which providers refuse. - A refusal handed over ends the run as it is, without follow-ups.
- A stop the blueprint hands over is the answer validated. Its
answermay be only the summary the follow-ups are grounded on: Knowledge Extraction's table and Release Notes' text in every language sit on the stop. - This covers a model without structured outputs, one that ignored them, and a blueprint that did not pass the schema (MCP ReAct, Knowledge Extraction).
- When the answer does not match,
- The meta-question gate is skipped for a message with a schema. Its answer is prose that ends the run before the completion. A refusal handed to the completion stays a refusal.
- Support is derived, never declared.
Agent.supports_response_schema()holds when a step returnsCompleteConversationEventand every declared completion stop is anLLMStopEvent. Discovery announces it, the class entity and the agent DTOs carry it, and the per-agent endpoints answer a schema sent to an agent without support with 400.
Rejected alternatives:
- A second structuring call after every text answer: it works for every blueprint unchanged, but it doubles the input tokens on every structured call, and the answer the caller sees is not the one the model wrote.
- A dedicated
StructuredStopEvent: RAG's typed stops would need parallel structured subclasses, and the API would have to know them all. - A separate result event ahead of the stop: the result would no longer be on the event that ends the run, which is what the per-agent endpoints return.
Consequences
- One structured answer costs one model call when the model honours
response_format, and two when it does not. A tool-loop answer usually costs the correction call too, since its last turn still offers tools. - The answering step streams the JSON as chunks, so the chat shows the JSON as it is written. A corrected answer shows a thought, and the corrected JSON arrives only on the stop event; API clients must read the result from the stop event.
- The answering call keeps the model's thinking. A reasoning model whose provider breaks JSON with thinking on is corrected by the completion's second call, which runs without streaming and without thinking.
UserMessageEventgrows by one optional field. Chat clients ignore it.- The API's model rebuild (
ModelCreationService) now also restores free-form maps announced as optional (anyOfmap or null) and maps inside optional nested models. Before, the caller'sjson_schemalost every key butproperties.
