inclusionAI/ZwZ2-4B-GGUF 模型权重发布

GGUF-format weights for ZwZ2-4B, part of Zooming without Zooming v2. The model is split into two components:

Language model: FP16, Q8_0, Q4_K_M.
Vision encoder and projector (mmproj): FP16, Q8_0, Q4_K_M.

Precision	Language model	Vision component
FP16	`ZwZ2-4B-F16.gguf`	`mmproj-ZwZ2-4B-F16.gguf`
Q8_0	`ZwZ2-4B-Q8_0.gguf`	`mmproj-ZwZ2-4B-Q8_0.gguf`
Q4_K_M	`ZwZ2-4B-Q4_K_M.gguf`	`mmproj-ZwZ2-4B-Q4_K_M.gguf`

Choose a language model and a matching vision component from this repository. Different precision levels can be combined. Quantization can affect accuracy; see the original model page for the published evaluation results.

How to Use

Use a recent llama.cpp build with Qwen3.5 multimodal support.

CLI Inference

llama-mtmd-cli \
  -m ZwZ2-4B-Q8_0.gguf \
  --mmproj mmproj-ZwZ2-4B-Q8_0.gguf \
  --image /path/to/image.jpg \
  -p "Describe this image." \
  --temp 0 -n 2048

Local API and Web Chat

llama-server \
  -m ZwZ2-4B-Q8_0.gguf \
  --mmproj mmproj-ZwZ2-4B-Q8_0.gguf \
  --host 127.0.0.1 --port 8080

Replace the filenames to use FP16 or Q4_K_M weights. For image input, always provide the corresponding mmproj component.

Paper

Zooming without Zooming v2

License

These converted weights retain the terms of the original model and its backbone. llama.cpp is distributed under the MIT license.