Running the Kimi K3 model
The story so far
User u/IndividualSurvival successfully ran the 2.8 trillion parameter Kimi K3 model using the discontinued Intel Optane Persistent Memory without a graphics card, with an initial generation speed of approximately 0.85 tokens per second, which increased to 1.50 tokens per second after disabling the DAX feature. The project was shared in the Reddit community, using a 40-core third-generation Intel Xeon Scalable processor and llama.cpp software for inference, but the token output speed was not smooth. The 3-bit quantization test only reached 0.75 tokens per second, and generating 50,000 tokens took 18.5 hours.
AI-written from the reports · Updated yesterday
Coverage timeline
Follow the coverage to see every side of the story.
- ITHome · AIPlayers use retired Intel Optane memory to run the 2.8 trillion parameter Kimi K3 model, outputting 1.5 tokens per second
Users have successfully run the Kimi K3 model with Intel Optane Persistent Memory, which has been discontinued, without a graphics card, achieving 2.8 trillion parameters. The initial generation speed was approximately 0.85 tokens per second, and it increased to 1.50 tokens per second after disabling the DAX function. The 3-bit quantization test only reached 0.75 tokens per second, and generating 50,000 tokens took 18.5 hours.
Heat over time
- Now
- 4
- Peak
- 1013 Nov, 16:00
- 24h change
- -49%
Hover or tap the chart to see each hour; use the arrow keys on a keyboard.