MMDM benchmark released

Hugging Face Daily Papers released the MMDM evaluation benchmark on November 11, 2026, evaluating 28 decision models. Most models performed well on common knowledge but poorly on interdependent options, constrained decision-making, and multiple-choice tasks. For example, Perplexity Decider-V1.2 achieved 78.75% accuracy on single-choice knowledge, but only 53.95% and 47.95% on option logic and constrained planning, respectively. The best accuracy for all models on hard multiple-choice tasks was only 36.81%. Perplexity then released the open-source decision model Decider v1.3 and its lightweight version pplx-decider-flash-v1.3 on November 14, 2026, adding a Locate feature to extract precise text fragments. The new models were released under Apache 2.0, supporting Perplexity Decisions API and OpenRouter, with an API input limit of 256K tokens and requiring CUDA GPU support for operation.

AI-written from the reports · Updated yesterday

1. 14 Nov, 00:10 · 1 reports
   Perplexity Releases Decider v1.3
   MarkTechPost：Perplexity releases Decider v1.3: Open-source decision model adds Locate feature
2. 11 Nov, 08:00 · 1 reports
   MMDM: Measuring Massive Multitask Decision Making
   Hugging Face Daily Papers：MMDM: Large-scale Multi-task Decision Model Evaluation Benchmark