Tianshu Libraries 1.0.6

This release focuses on the low-level LLM runtime used by upper Tianshu modules. It adds safer context planning, split MTP draft model support, structured thinking output, request-level tool template support, and clearer token usage accounting.

Highlights

Context Planning Before Model Load

• Added a dry-run context budget API so upper modules can estimate a safe plannedContextSize before loading a model.
• The library no longer silently reduces contextSize during startup. If the requested context is not safe, startup fails and asks the caller to use the dry-run result explicitly.
• Runtime context budget snapshots are now observational data, not a late correction after the model has already loaded.

Split MTP Draft Models

• Added explicit support for external split MTP draft models, such as Gemma MTP draft files.
• Upper modules can pass an MTP draft model path to enable it, or omit the path to keep it disabled.
• Removed implicit draft model discovery by file name or folder layout.
• MTP remains request-controlled through InferenceOptions.
• Successful calibration and runtime MTP trials are cached under config/Tianshu/libs/mtp, so the best draftMax can be reused automatically.
• The default uncalibrated draftMax is conservative.

Structured Thinking / COT Output

• Added request-level thinking capture through InferenceOptions.captureThinkingContent(true).
• Normal replies and thinking content are now separated:
  • text() contains the final visible answer.
  • thinkingContent() contains captured reasoning text when enabled.
  • Streaming calls can use a separate thinking callback.
• Token usage now separates prompt tokens, final answer tokens, and thinking tokens.
• Cancelled streaming requests report the usage known at the time of cancellation, including captured thinking tokens.

Request-Level Tool Template Support

• Added InferenceOptions.toolsJson(String).
• Supports OpenAI-style function tools:
  • a single function tool object,
  • an array of function tools,
  • or a { "tools": [...] } wrapper.
• If the model chat template supports tools, Tianshu Libraries passes the tools into the native chat template tools slot.
• If the model does not support tools, or the JSON cannot be parsed as OpenAI-style function tools, the raw JSON is appended to the prompt in a <tools>...</tools> fallback block.
• The library does not execute tools or parse tool calls. Tool orchestration remains the responsibility of the upper module.

Token Budget Semantics

• maxTokens > 0 now means a caller-defined maximum answer length, still clipped by the real context window.
• maxTokens == 0 now means no caller-defined answer limit; generation is only limited by the remaining context budget.
• Removed the old output reserve semantics.
• If the prompt already fills the available context budget, the request finishes cleanly with known usage instead of attempting unbounded generation.

Compatibility and Migration Notes

• Upper modules should call the new context budget dry-run API before loading large context sizes, then explicitly pass the returned plannedContextSize into contextSize(...).
• To use split MTP, pass the draft model path explicitly. If no draft path is provided, no external draft model is loaded.
• To capture thinking output, enable captureThinkingContent(true) and read the structured thinking result or streaming thinking callback.
• To provide tools, pass OpenAI-style function tools JSON through toolsJson(...). The upper module still needs to parse the model output and execute the selected tool.
• Existing simple chat/task APIs still return only the final visible answer text.

Cleanups

Removed or intentionally did not include several old or overly broad APIs:

• embedding token counting,
• embedding-with-usage,
• tokenizer/model id exposure in runtime capability,
• implicit MTP draft discovery,
• output reserve,
• the old include-thinking flag,
• request/config-level taskSuspendOnChat.

The current scheduling behavior remains: chat requests may suspend task-lane inference.