v0.40.8-rc0: mlx: add Flint model support (#19181)
v0.40.8-rc0: mlx: add Flint model support (#19181)
Ollama v0.40.8-rc0 has been released, adding support for the Flint model. It retains the source tool JSON and decoding API values to maintain numeric types and pattern order in prompts, while keeping the existing API serialization unchanged.
Preserve source tool JSON alongside decoded API values so Flint can retain
numeric types and schema order in prompts. Existing API marshaling is unchanged.
Assemble ModelOpt expert banks during safetensors import, and reject direct
Flint NVFP4 creation before conversion. Other models retain their existing
import and quantization paths.
Add optional learned sinks and device causal masks to shared MLX attention.
Existing callers retain the no-sink and host-mask paths; Flint opts into the
new behavior.
Source: Ollama releases · github.com