Skip to content
Live
Trending storyDeveloping

K2-Horizon Release

1 reports1 outlets9 hours agoUpdate

The story so far

AI overview

Hugging Face Released Open Source Large Language Model K2-Horizon, Including Six Models from 0.9B to 375B-A23B, Used for General Inference and Agent Work. Its 36B-A4B Model Adopts Innovative Attention Mechanism, Parameters Are Only One Seventh of 32B Dense Model. The Release Follows LLM360 Open Source Definition, Providing Complete Development History, Data, Code, and Training Framework, and Uses About 10T Synthetic Data for Pre-training, Among Which Four Models Share 22T Pre-training Corpus, Showing Scale Effects.

AI-written from the reports · Updated 1 hour ago

Coverage timeline

Follow the coverage to see every side of the story.

17 Nov
  1. Hugging Face Daily Papers
    K2-Horizon: A Frontier Fleet of Open Source Language Models

    K2-Horizon has released six open-source large language models, ranging from 0.9B dense models to 375B-A23B hybrid expert models, demonstrating excellent performance in general reasoning and agent tasks. Its 36B-A4B model combines Mixture-of-Value Attention and soft plus attention gating, with parameters only one-seventh of those in a 32B dense model.

Heat over time

Not enough data yet to draw a trend.