Hacker News · 300+· joshcsimmons·· 6 days agoFeatured78
Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
Brief overview
llama-modchip releases a set of ggml/llama.cpp core patches and quantization transcoding tools, compressing the 284.33B DeepSeek-V4-Flash to 56.56 GiB (1.71 bpw), which can run on a pure CPU machine with 64 GB memory, with a peak memory of 57.61 GiB under 256K context and no need for swap.
Why it matters
The project provides a quantization scheme and core patches for running 284B MoE on 64 GB memory, along with measured data for each hardware, which can be used to determine the feasible boundary for pure CPU deployment.
Source: Hacker News · 300+ · github.com