Skip to content
Live

Deployment & infra · Page 4

Oct 21Wed61–66
  1. vLLM Blog60

    vLLM supported the Recursive Flint model on the first day.

    vLLM announced support on the day Flint was released, with both NVFP4 and BF16 checkpoints capable of running on a single GPU, decoding approximately 500 tokens/s per request on a single GB200 with a context of 100,000 tokens.

    Why it matters: The original text provided Flint's day-0 support on vLLM, single-card throughput data, and the step-by-step kernel optimization process, allowing readers to judge the engineering trade-offs of small-batch inference based on this.

Oct 20Tue
  1. Hugging Face Blog62

    PyTorch 2.15 brings FP8 and faster LLM decoding to the MPS backend.

    The Hugging Face PyTorch team introduces updates to the MPS backend in PyTorch 2.15, adding the tensor torch.float8_e4m3fn and F.scaled_mm, with FP8 tensors occupying half the space of BF16.

    Why it matters: The original text provides specific numbers for the MPS backend in FP8, MoE decoding, and memory usage, which can be used to judge the actual changes in running models locally on Mac.

Sep 14Mon
  1. LangChain Blog71

    How to build paid media agents with LangChain

    LangChain publicly shares its construction method for paid media intelligence agents, which run on Slack, summarize ad platform and warehouse data weekly, and generate reports. Paid media increased from contributing 0 to 20% of the marketing pipeline within 6 months, and the cost per qualified lead decreased by 30% from June to August.

    Why it matters: LangChain publicly shares the complete engineering details of its paid media intelligence agents, including context hierarchy, on-demand tool discovery, and sub-agent isolation, which can serve as a reference for building long-running intelligent agents.

Jul 30Thu
  1. LangChain Blog62

    LangSmith has launched the public beta version of LLM Gateway, providing runtime control for production agents.

    LangSmith LLM Gateway has entered public beta, serving as a centralized governance layer between agents and models, uniformly executing spending limits, rate limiting, model fallback, and sensitive data deidentification. Spending and rate limiting can be set at four levels: organization, workspace, API key, and user. When the limit is hit, the agent receives a 402 response; fallback rules can take effect uniformly across models and hosting providers.

    Why it matters: The original text listed the specific control levels of the gateway in terms of cost, rate limiting, fallback, and data masking, which can help determine at which level the governance of the production intelligent agent should be placed.

May 14Thu
  1. LangChain Blog73

    LangChain is conducting a public beta of Managed Deep Agents on LangSmith Cloud.

    LangChain launches Managed Deep Agents on LangSmith Cloud. Users write intelligent agents as Python projects consisting of files like agent.py, instructions.md, skills/, and deploy them with a single command mda deploy, provided with persistent threads and streaming execution by LangSmith Agent Server.

    Why it matters: The original text explains how to organize the agent project by files and deploy it to LangSmith with one click, as well as the infrastructure managed runtime that takes on the basics for the team.

Apr 8Wed
  1. Anthropic Research74

    Anthropic launched Claude Managed Agents, decoupling the agent's brain from its hands

    Anthropic has launched Managed Agents on the Claude Platform, splitting the agents into three replaceable interfaces - session, harness, and sandbox - for running long-term tasks.

    Why it matters: Anthropic has disclosed the architectural trade-offs of hosting agents, showing readers how latency and security boundaries change when sessions, harnesses, and sandboxes are separated.