Tianshu Library
A comprehensive local AI library integrating llama.cpp (JJML JNI), sherpa-onnx, and ONNX Runtime to power fully offline LLMs, speech recognition, and text-to-speech directly in-game.
Список изменений
Release Notes
This update upgrades the underlying JJML runtime and improves compatibility with the latest local GGUF model workflow, especially for Qwen3.5, Vulkan GPU inference, and MTP speculative decoding.
Highlights
- Updated the bundled JJML runtime and native loading path.
- Improved compatibility with Qwen3.5 GGUF models.
- Text-only inference remains the default; vision/mmproj paths are not enabled automatically.
- Added request-level MTP speculative decoding support.
- Added MTP capability detection and automatic calibration.
- Added safe fallback to standard decoding when MTP is unsupported or fails to initialize.
- Improved cold-resume behavior for background TASK inference interrupted by CHAT or higher-priority TASK requests.
- Cold-resume now stores generated token IDs instead of re-tokenizing raw text, improving recovery accuracy.
- Integrated the new JJML restored-target resume path for MTP recovery.
- Added Vulkan inference priority controls for future-friendly GPU scheduling.
- Improved GPU device selection support for Vulkan environments.
- Strengthened long-context testing and MTP calibration with token-budget-aware workloads.
Compatibility
Existing chat, chatStream, task, and taskStream calls continue to work without changes. New runtime options are optional and only needed when callers want to enable MTP or Vulkan priority controls.
contextSize is still fixed when the model is loaded. Per-request maxTokens controls the maximum response length, so long-prompt use cases should keep enough context budget for both input and output.
Validation
Tested with Qwen3.5-2B-Q4_K_M.gguf on Vulkan GPU inference, including standard generation, MTP generation, CHAT interruption, TASK preemption, MTP cold resume, and an 8k-context long-prompt recovery test.