Skip to content
Live
Official sources#Engineering
Oct 21Wed
  1. IBM Research Blog

    IBM vLLM-Hook MIA Plugin enables rapid runtime monitoring and control

    IBM releases vLLM-Hook MIA Plugin, implementing runtime monitoring and control in the vLLM inference engine, 20 times faster than its predecessor vLLM-Hook and 14 times faster than similar methods inserting code inside the model, with only a 2%-6% increase in latency. The plugin allows developers to observe and adjust model states in real-time, such as rejecting malicious prompts, marking hallucinatory facts, or editing response tone and accuracy, while almost not reducing performance.

  2. NVIDIA Blog55

    NVIDIA Confidential Computing expands, providing secure extension capabilities for cutting-edge model builders across global infrastructure

    NVIDIA Confidential Computing protects models and data execution through hardware root isolation and encrypted authentication, adding Confidential Bare Metal deployment options, enabling cutting-edge model builders to run sensitive workloads on third-party infrastructure while retaining model data control.

  3. NVIDIA Blog38

    NVIDIA cuDSS is now generally available, accelerating engineering and scientific simulations.

    NVIDIA cuDSS is generally available, migrating large sparse system of equations solvers to GPUs as part of the CUDA-X library, shortening simulation runtimes and improving throughput. Applied Materials achieves up to 10x acceleration in Ginestra, Dassault Systèmes reports 8-10x improvement in Abaqus, and ZEISS SMT completes an EUV lithography plasma simulation benchmark in COMSOL 4x faster.

    Same story · 2 outlets

    How the accounts differ:NVIDIA Blog states ZEISS SMT achieved a 4x speedup with COMSOL for EUV lithography plasma simulation; NVIDIA Newsroom mentions a 10x speedup for COMSOL with cuDSS.

  4. NVIDIA Blog58

    NVIDIA European partners expand computing capabilities, building sovereign, open, and secure AI agent cloud ecosystems.

    NVIDIA European partners are expanding regional computing capabilities to support model training and inference, helping developers build agentic and physical AI applications. Nscale achieves 37% more GPUs and 50% higher inference throughput using the NVIDIA DSX platform, Crusoe develops a liquid-cooled AI factory for low-latency inference, and Nebius adopts the NVIDIA Groq 3 LPX to accelerate sensitive applications.

  5. NVIDIA Blog72

    Google Cloud and NVIDIA announce Vera Rubin infrastructure now available

    Google Cloud and NVIDIA announce A5X instances based on NVIDIA Vera Rubin now available, opening next month and planned to expand to Europe.

    Why it matters: The original text provides the availability time of A5X instances, customer testing, and confidential computing support, allowing readers to judge the implementation rhythm of large-scale reinforcement learning infrastructure.

  6. NVIDIA Blog

    NVIDIA showcased its full lineup of local AI hardware: RTX Spark, DGX Spark, and DGX Station, helping to power next-generation intelligent agents.

    NVIDIA showcased its full lineup of local AI hardware at GTC Berlin, with RTX Spark now available to support personal agents on Windows PCs, DGX Spark enabling developers to build and scale Linux applications, and DGX Station designed for Linux enterprise teams to run cutting-edge workloads, with the Windows version launching in December.

  7. Databricks Blog

    Databricks AI Runtime officially released

    Databricks releases AI Runtime (AIR), bringing managed GPU infrastructure directly into its platform, allowing customers to train high-demand models at their enterprise data location. AIR provides GPU infrastructure, developer tools, and data governance features, supporting large-scale training, rapid iteration, and load safety, with customers not needing to manage a separate GPU stack.

  8. NVIDIA Blog67

    NVIDIA GTC Berlin 2026: Huang Renxun announces multiple updates including DSX POD, OpenShell, and Nemotron.

    NVIDIA CEO Huang Renxun announced multiple updates at the GTC Berlin 2026 keynote speech, including the Vera Rubin AI factory reference design DSX POD delivered in modular container format, reducing on-site deployment time from six months to 14 days.

    Why it matters: Huang Renxun's keynote speech at GTC Berlin revealed the latest developments in three product lines: AI factories, secure runtime for agents, and open-source models, providing insights into NVIDIA's focus in Europe.

  9. AWS Machine Learning Blog58

    vLLM enhances the TTFT performance of multi-round agent workloads through bidirectional KV transmission.

    vLLM's bidirectional KV transmission mechanism reduces redundant calculations by allowing the decoding node to return previously computed KV cache to the pre-filled node in subsequent dialogue turns. In multi-round benchmark tests on AWS EC2 P5en instances, it improved the TTFT of Qwen3-32B by 1.6x and zai-org/GLM-5-FP8 by 3.3x. This feature is implemented by extending the NIXL KV connector and utilizing a KV cache-aware router to coordinate KV reuse across dialogue turns.

  15. AWS Machine Learning Blog38

    How Amazon Quick and Amazon Bedrock AgentCore implement conversational intelligent decision-making in supply chains

    Amazon Quick combined with Amazon Bedrock AgentCore provides supply chain teams with real-time data access across systems. Users can get cross-system answers in seconds through conversational queries, such as identifying high-income SKUs and costs affected by port closures. Quick serves as the interface layer, and AgentCore as the intelligence layer, connecting backend systems like ERP, monitoring, and procurement via the MCP protocol to achieve a complete process from problem discovery to generating emergency purchase requests, all operations requiring manual confirmation.

  17. GitHub Blog · AI & ML33

    How GitHub Copilot continues a session in VS Code

    GitHub Copilot users can directly open a GitHub Copilot app session in VS Code to continue their conversation, code changes, and context. Users can review Copilot's modifications line by line, make manual edits directly in VS Code, and run and validate the project locally. The session and code workspace remain connected, supporting interactive iterative optimization with Copilot, and finally submitting updates to the associated pull request.

  18. Databricks Blog

    Deploy RFD4-Proteina on Databricks

    Databricks has integrated the RFD4-Proteina model into Genesis Workbench, supporting execution in AI runtime environments and connecting research data and workflows. Developers can build and adjust pipelines using Genie Code and the NVIDIA BioNeMo Agent Toolkit, leveraging serverless NVIDIA GPU computing for protein design.

  19. vLLM Blog60

    vLLM supported the Recursive Flint model on the first day.

    vLLM announced support on the day Flint was released, with both NVFP4 and BF16 checkpoints capable of running on a single GPU, decoding approximately 500 tokens/s per request on a single GB200 with a context of 100,000 tokens.

    Why it matters: The original text provided Flint's day-0 support on vLLM, single-card throughput data, and the step-by-step kernel optimization process, allowing readers to judge the engineering trade-offs of small-batch inference based on this.

  20. Google Developers Blog

    Agents CLI Extension System: Supports the intelligent agent development lifecycle across frameworks.

    Agents CLI 1.5.0 version introduces an extension system, allowing developers to override or add commands without modifying the tool, supporting cross-framework usage; the official LangChain extension replaces framework-bound commands while retaining other lifecycle functionalities; developers can create custom extensions, such as adding deployment commands for software bill of materials (SBOM) scanning, enabling enterprise platform teams to enforce governance policies.

  21. AWS Machine Learning Blog38

    Smartsheet MCP on AWS has launched the Universal Model Context Protocol (MCP) server.

    Smartsheet launches the Universal Model Context Protocol (MCP) server, enabling AI assistants to read and write manage Smartsheet data through a standardized protocol; supports Amazon Quick and Kiro connections, Quick for knowledge workers and operations teams to naturally query and update Smartsheet with natural language, Kiro allows developers to refer to project specifications and update task status without leaving the coding environment.

  22. Databricks Blog

    Databricks introduces Agent Memory to enhance intelligent agent capabilities

    Databricks introduces Agent Memory, which improves agent quality, reduces costs and latency by storing key facts, preferences, and processes, and supports running on Lakebase. It is currently in Beta version. Agent Memory is based on Databricks' latest memory research and uses the MemCurrent architecture. In internal benchmark tests, it has tripled the pass rate of agent tasks while reducing token usage by 31%.

Oct 20Tue
  1. IBM Research Blog

    llm-d 1.0 Released: Strengthening Open Source AI Inference Production Environment

    The llm-d project announces the release of version 1.0, which is a Kubernetes-native distributed inference service network that supports deploying any model's inference stack on any accelerator in any cloud. This version enhances multi-tenant quality of service, zero-downtime operations, and agent-aware services to achieve continuous performance in common production operations. Through techniques such as prefix cache-aware routing and KV cache offloading, llm-d 1.0 can capture 97% of available cache reuse, reducing the cost of large-scale AI services.

  2. Hugging Face Blog58

    AnyLanguageModel 1.0 is released: supporting local model advancements on Apple platforms for a year.

    AnyLanguageModel 1.0 is released, supporting local model advancements on Apple platforms for a year. New support added for cloud models such as OpenAI Chat Completions, Anthropic Messages, and Google Gemini. Local model support includes llama.cpp, MLX, and Core ML. Tool calling and guided generation features are expanded. Cloud models have the same functionality as local models, and API keys support dynamic acquisition through closures.

  3. Anthropic Research

    How does Claude design protein binders for scorpion neurotoxins?

    Claude autonomously generated and screened 318,484 protein designs in 64 hours, selecting 180 for testing, and collaborated with Adaptyv Bio and Twist Bioscience to experimentally validate 211 designs, of which 17 met the binding criteria. All design data has been made public. Experts consider this the first AI design attempt for scorpion neurotoxins, which could accelerate drug discovery for neglected diseases.

  5. Hugging Face Blog62

    PyTorch 2.15 brings FP8 and faster LLM decoding to the MPS backend.

    The Hugging Face PyTorch team introduces updates to the MPS backend in PyTorch 2.15, adding the tensor torch.float8_e4m3fn and F.scaled_mm, with FP8 tensors occupying half the space of BF16.

    Why it matters: The original text provides specific numbers for the MPS backend in FP8, MoE decoding, and memory usage, which can be used to judge the actual changes in running models locally on Mac.

  9. Claude · YouTube

    Claude has launched Claude Dashboards and Claude Motion

    Claude has launched Claude Dashboards and Claude Motion, with the former supporting the real-time conversion of data platform or CRM data into dashboards in all paid plans, and the latter supporting the generation of animated explanatory videos from reports or charts in Team and Enterprise plans, both generated in code form without a video model, with content that can be modified and exported as MP4 or edited.

  10. Meta Engineering

    Meta introduces GenRecs Tokenization to reshape ad recommendations with a stable model vocabulary.

    GenRecs Tokenization enhances conversion rates by 0.35% for advertisers by representing items like ads and products with a stable semantic vocabulary, combined with Meta's Generative Ad Model (GEM) and adaptive ranking model. This technology leverages the world knowledge of pre-trained large models to enable recommendation systems to understand preferences of new items or users with no historical interaction, accelerates learning through shared concepts, achieves cross-activity transfer, and improves performance as large model scale increases, forming a feedback loop for performance improvement.

Oct 19Mon
  2. LangChain Blog

    How Nu company builds AI health coaches using LangGraph and LangSmith and prioritizes evaluation

    Nu company, while developing AI health coach Nu, prioritized building evaluation criteria to avoid boundary-crossing medical diagnoses. Using the LangGraph framework and LangSmith tools, it ensures the system meets health guidance needs while complying with regulatory requirements through LLM judges scored by category and open-ended behavior tests. The evaluation framework is designed separately for single responses and complete conversations, supporting rapid migration between two leading model providers. Safety probes and guard nodes review conversation content in real-time, ensuring the product stays within compliance boundaries.

  3. IBM Research Blog22

    Coursera's new course 'AI Data Stream Architecture: From Principles to Practice' teaches the principles of modern accelerator operation

    Coursera's new course 'AI Data Stream Architecture: From Principles to Practice' teaches engineers how to work across GPUs, TPUs, Cerebras WSE, Groq LPU, and IBM Spyre accelerators. The course is taught by Kaoutar El Maghraoui, a senior research scientist at IBM Research, covering execution models, compilation, memory systems, debugging, and performance modeling through six courses and hands-on experiments, emphasizing how different architectures affect model performance and the key role of compiler optimization.

  5. Together AI Blog

    Together Post-Training: Flexible and efficient model fine-tuning service

    Together AI has launched Post-Training services, supporting the construction of custom models through reinforcement learning and supervised fine-tuning without complex infrastructure. The service provides flexible training loop definition capabilities and optimizes the underlying system and infrastructure to improve model quality and training speed. It supports mainstream open-source models such as Muse Spark Open, GLM 5.3, Kimi K3, and Qwen 3.8-27B, running on dedicated GPU resources to ensure efficient and numerically consistent training.

Oct 18Sun
  1. NVIDIA Blog

    NVIDIA RTX Spark laptops are officially launched, featuring Blackwell RTX GPUs and an AI platform.

    NVIDIA RTX Spark laptops are now available, equipped with Blackwell RTX GPUs, up to 128GB unified memory, and the full NVIDIA AI platform, starting at $2399. They are launched by ASUS, Dell, HP, Lenovo, Microsoft, and MSI, bringing powerful local AI, content creation, and gaming capabilities to portable devices. NVIDIA held global launch events, including one in Berlin, where founder Jensen Huang made a surprise appearance.