json.decoder.JSONDecodeError: Unterminated string starting at (truncated tool-call arguments)

Any AI coding tool Last checked 2026-10-25

What you saw

Your code calls json.loads() on a tool call's arguments and it throws partway through — the string just stops, there is no closing quote or brace.

In JavaScript the same failure shows as SyntaxError: Unexpected end of JSON input or Unexpected non-whitespace character after JSON at position N.

It is intermittent: short arguments parse fine for weeks, then one call with a long file path, a big code block or a long search query breaks. The model was not wrong about *what* to call — the JSON encoding of the call itself is incomplete.

It happens most with tools whose arguments contain large free-text fields — a content or code parameter, a long query, an entire file being written — rather than tools with small scalar arguments.

What it actually means

Tool-call arguments are generated token by token, as JSON text, inside the same per-response token budget as everything else the model writes. If that budget runs out before the closing " and } are emitted, the API still returns what it has — a syntactically incomplete JSON string — because from the model's side nothing "errored"; it just ran out of room.

This is different from the model inventing wrong values (a schema problem). Here the values would have been correct; the text was simply amputated. Parsing fails because JSON has no tolerance for a partial string — one missing quote and the whole document is invalid, even though everything before the cut point was well-formed.

It is also different from a provider-side truncation marker. Most chat completion responses that got cut for length set finish_reason: "length" (OpenAI) or stop_reason: "max_tokens" (Anthropic) on the *message*, but if your code only looks at the tool call's arguments field and tries to parse it blind, you never check that flag and the JSONDecodeError is the first sign anything went wrong.

Streaming makes it worse if you parse eagerly: reading tool_call.function.arguments deltas and calling json.loads() on the partial buffer after every chunk will throw on nearly every chunk except the last one, because every chunk before completion is, correctly, incomplete JSON.

The fix

1. Check the finish/stop reason before you touch the arguments. If the model ran out of tokens, treat it as a retry case, not a parse bug.

   # OpenAI
   if resp.choices[0].finish_reason == "length":
       raise ToolCallTruncated(resp.choices[0].message.tool_calls)
   
   # Anthropic
   if resp.stop_reason == "max_tokens":
       raise ToolCallTruncated(resp.content)

2. Raise the output token limit for calls that use large-payload tools (write_file, search with long queries, anything with a free-text body parameter). This fixes the large majority of cases outright.

   # OpenAI
   resp = client.chat.completions.create(
       model="gpt-4.1", messages=messages, tools=tools,
       max_tokens=4096,  # was the default / too low
   )
   
   # Anthropic
   resp = client.messages.create(
       model="claude-opus-4", messages=messages, tools=tools,
       max_tokens=4096,
   )

3. If you are parsing a streamed buffer, only call json.loads() once the stream signals the tool call is complete — never on every delta.

   buf = ""
   for event in stream:
       if event.type == "content_block_delta" and event.delta.type == "input_json_delta":
           buf += event.delta.partial_json
       if event.type == "content_block_stop":
           args = json.loads(buf)  # parse exactly once, at the end

4. Wrap the parse so a truncation produces a specific, retryable error instead of an unhandled exception — then have the agent loop ask the model to re-emit the call, usually with a smaller payload.

   try:
       args = json.loads(raw_arguments)
   except json.JSONDecodeError as e:
       if raw_arguments.count('"') % 2 == 1 or not raw_arguments.rstrip().endswith("}"):
           return retry_tool_call(tool_name, reason="truncated_arguments")
       raise

5. For tools that write large content (a whole file, a long document), split the payload: have the tool take a path/handle and a small number of chunked append calls instead of one call with the entire body as a single JSON string argument.

   code: write_file(path, chunk, append=True)  # called N times
   # instead of: write_file(path, entire_10kb_file_as_one_string)

How to stop it happening again

Set max_tokens explicitly and generously for any call where a tool schema includes a large free-text field — do not rely on the provider default, which is tuned for prose, not for embedding whole files as JSON string values.

Log finish_reason / stop_reason on every tool-call response in development. A truncation that gets silently retried in production is invisible until someone finally reads a stack trace.

Design tool schemas so large payloads are the last argument in the JSON object, not the first — a cut mid-payload at least leaves every preceding field complete and parseable with a tolerant partial-JSON parser if you need to salvage a retry.

If your framework supports it, use a streaming JSON parser (e.g. a "partial JSON" parser) for progress UI, but keep a strict json.loads() as the gate before you actually execute the tool — never execute a tool call built from a partially-parsed buffer.

Related questions

Why does json.loads() fail even though the error message says the string just stops?

Because JSON strings must be closed with a matching quote and the overall document with a matching brace. If generation stopped mid-string, neither exists yet, so the parser reaches the end of the buffer still inside an open string literal — hence "unterminated string".

Is this the same bug as the agent calling a tool with invalid arguments?

No. Invalid-arguments means the JSON parsed fine but the values are wrong or made up — a schema and prompting problem. Truncated JSON means the response was cut off before the arguments finished encoding — a token-budget problem. The fix is different: schema tightening for one, a higher max_tokens and a finish-reason check for the other.

Will increasing max_tokens cost more?

Only if the model actually uses the extra room — you are raising a ceiling, not a floor. The calls that were truncating will now complete; calls that already finished well under the old limit are unaffected.

My framework (LangChain, Vercel AI SDK, etc.) hides the raw tool call. How do I check finish_reason?

Most wrap the provider response in a normalized object that still exposes it: LangChain's AIMessage.response_metadata['finish_reason'], the Vercel AI SDK's result.finishReason. If your wrapper truly hides it, drop to the underlying client for that one call so you can see the flag.