An assistant message with 'tool_calls' must be followed by tool messages

Any AI coding tool Last checked 2026-11-01

What you saw

You're writing your own agent loop against the OpenAI (or Azure OpenAI, or an OpenAI-compatible) chat completions API, and the very next call after a tool-calling turn throws a 400 with this message.

It often appears only intermittently — the loop works for dozens of turns, then one request has two or three parallel tool_calls in the assistant message and only some of them get a tool response appended before the next request fires.

It also shows up after adding conversation history truncation or compaction (to save tokens or stay under the context window): the trimmer drops an older message and happens to cut a tool response while leaving the assistant message that requested it.

Streaming is a common trigger: if you build the assistant message from streamed deltas and a tool_call chunk arrives but your code doesn't accumulate all of its id/name/arguments before you execute it, the tool_call_id saved in history can end up blank or mismatched.

Retrying a failed tool execution by re-sending the conversation sometimes duplicates the assistant message's tool_calls array without duplicating the paired tool responses, so the second copy has none.

What it actually means

OpenAI's chat completions format is strict about this one pairing: an assistant message that contains a tool_calls array creates a contract — every single tool_call_id in that array must be answered by exactly one message with role 'tool' and a matching tool_call_id, before you send the next request.

This isn't a soft validation — the API checks message order and IDs on every call, including ones that worked fine many turns ago in the same conversation. If any later edit to your history-building code breaks the pairing for one old turn, every subsequent request in that conversation fails the same way.

Parallel tool calls make this easy to get wrong: a single assistant turn can request three tools at once, and it is easy to write code that executes and appends only the tools your code recognizes, silently skipping an unknown or failed one instead of still appending a tool response for it.

Truncating or summarizing conversation history to manage context length is the other major cause — if your trimming logic removes messages by age or token count without the specific rule "never split an assistant tool_calls message from its tool responses," it will eventually cut through the middle of a pair.

This is distinct from a model producing a malformed tool call (bad JSON arguments) — that fails differently, inside your tool-execution code. This error fires from the API itself, rejecting the request before the model ever sees it, because your message array is structurally invalid.

The fix

1. Find the offending tool_call_id from the error message and locate it in the conversation history you're about to send — confirm whether its tool response message is missing, duplicated, or has a mismatched tool_call_id string.

   ids_called = {tc["id"] for m in messages if m.get("tool_calls") for tc in m["tool_calls"]}
   ids_answered = {m["tool_call_id"] for m in messages if m.get("role") == "tool"}
   print("missing responses:", ids_called - ids_answered)

2. If a tool call failed or your code doesn't handle its name, you still must append a tool response — return the error as the tool's content instead of skipping the message entirely.

   for call in assistant_msg["tool_calls"]:
       try:
           result = run_tool(call)
       except Exception as err:
           result = f"error: {err}"
       messages.append({
           "role": "tool",
           "tool_call_id": call["id"],
           "content": str(result),
       })

3. If you truncate or summarize history for context length, treat an assistant tool_calls message and its tool responses as one atomic unit — trim whole turns, never individual messages inside a turn.

   # wrong: trims by raw message count / age, can split a pair
   messages = messages[-N:]
   
   # right: group into turns first, then drop whole turns from the oldest end
   turns = group_into_turns(messages)  # each turn keeps its tool_calls + tool responses together
   messages = flatten(turns[-K:])

4. If you build the assistant message from a streamed response, accumulate every tool_call delta (id, name, and the full arguments string) before you execute anything or append the message to history — don't act on a partial tool_call.

   tool_calls = accumulate_stream_deltas(stream)  # wait for stream to finish this turn
   assert all(tc["id"] and tc["function"]["name"] for tc in tool_calls)
   messages.append({"role": "assistant", "tool_calls": tool_calls, "content": None})

5. If you retry a failed turn by resending history, strip the previous attempt's assistant tool_calls message and its (incomplete) responses together, rather than leaving a half-paired turn in place and appending a second attempt on top.

   if last_turn_failed:
       messages = messages[:-len(last_turn_messages)]  # drop the whole broken turn
       # then retry tool execution and append a fresh, complete turn

How to stop it happening again

Write one function that appends a completed tool-call turn to history — the assistant message plus every matching tool response — as a single atomic operation, and never let other code paths append either half separately.

Add an assertion before every API call in development: every tool_call_id in the message array that appears in an assistant message must appear exactly once in a later tool message. Fail loudly in dev instead of discovering it in production.

If you use an agent framework (LangChain, LangGraph, the OpenAI Agents SDK, Vercel AI SDK) instead of raw API calls, let the framework manage message history rather than hand-editing it — most of these bugs come from custom truncation or retry code layered on top.

When trimming history for context length, trim by whole conversational turn (grouped by response id or a turn counter you assign yourself), never by raw message index or token count alone.

Log the full messages array (or at least the role/tool_call_id shape of it) right before any 400 from the chat completions endpoint — this error is much faster to diagnose from the actual payload than from a stack trace.

Related questions

Does this happen with Anthropic's Claude API too?

The same underlying rule applies — Claude's Messages API expects a tool_use content block to be followed by a matching tool_result block with the same tool_use_id — but the error text differs. OpenAI's chat completions API is the one that produces this exact 'must be followed by tool messages' string.

Can I just delete the orphaned assistant message instead of adding a tool response?

Yes, that also fixes it, and it's sometimes the right call if the tool call was genuinely invalid and you don't want the model to see it was ever asked. But deleting messages after the fact means your stored conversation no longer matches what actually happened, which makes later debugging harder — appending an honest 'error' tool response is usually the better default.

Why did this only start happening after I added parallel tool calls?

Single sequential tool calls are easy to pair correctly almost by accident — one assistant message, one tool response, in order. Parallel tool calls put multiple IDs in one assistant message's tool_calls array, and any code that loops over 'the tools I know how to run' rather than 'every tool_call_id in this array' will silently skip the ones it doesn't recognize, leaving them unanswered.

Is there a way to validate my message array before sending it?

Yes — walk the list, and for every message with role 'assistant' that has a non-empty tool_calls array, confirm the very next N messages (role 'tool') cover every id in that array with no gaps and no role other than 'tool' or another tool_calls assistant message in between. Run that check in a test against a few real conversation logs, not just happy-path unit tests.